2024-02-18T11:20:35.236 INFO:root:teuthology version: 0.0.1.dev260+gb1dac55 2024-02-18T11:20:35.236 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/vshankar-2024-02-17_05:57:56-fs-wip-vshankar-testing-20240217.015652-testing-default-smithi/7564362 --verbose --owner scheduled_vshankar@teuthology --name vshankar-2024-02-17_05:57:56-fs-wip-vshankar-testing-20240217.015652-testing-default-smithi --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/no 5-workunit/suites/iogen}} -- /home/teuthworker/archive/vshankar-2024-02-17_05:57:56-fs-wip-vshankar-testing-20240217.015652-testing-default-smithi/7564362/orig.config.yaml 2024-02-18T11:20:35.293 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-18T11:20:35.390 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/vshankar-2024-02-17_05:57:56-fs-wip-vshankar-testing-20240217.015652-testing-default-smithi/7564362 branch: wip-vshankar-testing-20240217.015652 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/no 5-workunit/suites/iogen}} email: vshankar@redhat.com first_in_suite: false job_id: '7564362' kernel: client: sha1: distro flavor: default kdb: true sha1: 8d59ab0bd93c806231018a6729893bf309624262 ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: vshankar-2024-02-17_05:57:56-fs-wip-vshankar-testing-20240217.015652-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-vshankar-testing-20240217.015652 ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: 1c920afa8316ea21877f3186850254947531ea22 subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported dry_run: true install: ceph: flavor: default sha1: 1c920afa8316ea21877f3186850254947531ea22 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - wsync syntax: v2 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-vshankar-testing-20240217.015652 sha1: 1c920afa8316ea21877f3186850254947531ea22 owner: scheduled_vshankar@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 218 sha1: 1c920afa8316ea21877f3186850254947531ea22 sleep_before_teardown: 0 subset: 793/1024 suite: fs suite_branch: wip-vshankar-testing-20240217.015652 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_1c920afa8316ea21877f3186850254947531ea22/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 1c920afa8316ea21877f3186850254947531ea22 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi012.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPrh55wigUo1PV8Sb+YAd/BcDe3NzW1EBCcCr++pGUugPTHIrEASfgvqt8wLiFvHCFlwb40zj+IIA7WFOtaQ3y4= smithi158.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDJBuLtLzONUSej4uAtVBleRAiTjKIFNpfuX5dLDM0m0j1l2nOCx+EdZqn08ocQh6JQufNhmMjal/fpjbV3J+Fk= smithi188.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBamJ0stpfhNmYvSj0ZmQp6uM1kWXi3t6wSf7VETIvnMOdClFcvgahcP/hcCcctGcvr6rYRthK9ke8H0znZLiWE= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - workunit: clients: all: - suites/iogen.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: b1dac5519c57c269ea98a22fb7729016a1d0e4d2 timestamp: 2024-02-17_05:57:56 tube: smithi user: vshankar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-02-18T11:20:35.390 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_1c920afa8316ea21877f3186850254947531ea22/qa; will attempt to use it 2024-02-18T11:20:35.391 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_1c920afa8316ea21877f3186850254947531ea22/qa/tasks 2024-02-18T11:20:35.391 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-02-18T11:20:35.393 INFO:teuthology.task.internal:Checking packages... 2024-02-18T11:20:35.419 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '1c920afa8316ea21877f3186850254947531ea22' 2024-02-18T11:20:35.420 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-02-18T11:20:35.420 INFO:teuthology.packaging:ref: None 2024-02-18T11:20:35.420 INFO:teuthology.packaging:tag: None 2024-02-18T11:20:35.420 INFO:teuthology.packaging:branch: wip-vshankar-testing-20240217.015652 2024-02-18T11:20:35.420 INFO:teuthology.packaging:sha1: 1c920afa8316ea21877f3186850254947531ea22 2024-02-18T11:20:35.420 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-vshankar-testing-20240217.015652 2024-02-18T11:20:35.644 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1555.g1c920afa 2024-02-18T11:20:35.646 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-02-18T11:20:35.653 INFO:teuthology.task.internal:no buildpackages task found 2024-02-18T11:20:35.653 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-02-18T11:20:35.678 INFO:teuthology.task.internal:Saving configuration 2024-02-18T11:20:35.700 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-02-18T11:20:35.706 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-18T11:20:35.734 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi012.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-02-17_05:57:56-fs-wip-vshankar-testing-20240217.015652-testing-default-smithi/7564362', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-18 11:14:53.493738', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPrh55wigUo1PV8Sb+YAd/BcDe3NzW1EBCcCr++pGUugPTHIrEASfgvqt8wLiFvHCFlwb40zj+IIA7WFOtaQ3y4='} 2024-02-18T11:20:35.759 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi158.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-02-17_05:57:56-fs-wip-vshankar-testing-20240217.015652-testing-default-smithi/7564362', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-18 11:14:53.492560', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDJBuLtLzONUSej4uAtVBleRAiTjKIFNpfuX5dLDM0m0j1l2nOCx+EdZqn08ocQh6JQufNhmMjal/fpjbV3J+Fk='} 2024-02-18T11:20:35.777 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi188.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-02-17_05:57:56-fs-wip-vshankar-testing-20240217.015652-testing-default-smithi/7564362', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-18 11:14:53.490714', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBamJ0stpfhNmYvSj0ZmQp6uM1kWXi3t6wSf7VETIvnMOdClFcvgahcP/hcCcctGcvr6rYRthK9ke8H0znZLiWE='} 2024-02-18T11:20:35.777 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-02-18T11:20:35.784 INFO:teuthology.task.internal:roles: ubuntu@smithi012.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-02-18T11:20:35.784 INFO:teuthology.task.internal:roles: ubuntu@smithi158.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-02-18T11:20:35.784 INFO:teuthology.task.internal:roles: ubuntu@smithi188.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-02-18T11:20:35.784 INFO:teuthology.run_tasks:Running task console_log... 2024-02-18T11:20:35.924 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f74ea816160>, signals=[15]) 2024-02-18T11:20:35.925 INFO:teuthology.run_tasks:Running task internal.connect... 2024-02-18T11:20:35.931 INFO:teuthology.task.internal:Opening connections... 2024-02-18T11:20:35.931 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi012.front.sepia.ceph.com 2024-02-18T11:20:35.933 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-18T11:20:36.053 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi158.front.sepia.ceph.com 2024-02-18T11:20:36.054 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-18T11:20:36.144 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi188.front.sepia.ceph.com 2024-02-18T11:20:36.145 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-18T11:20:36.239 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-02-18T11:20:36.246 DEBUG:teuthology.orchestra.run.smithi012:> uname -m 2024-02-18T11:20:36.266 INFO:teuthology.orchestra.run.smithi012.stdout:x86_64 2024-02-18T11:20:36.267 DEBUG:teuthology.orchestra.run.smithi012:> cat /etc/os-release 2024-02-18T11:20:36.325 INFO:teuthology.orchestra.run.smithi012.stdout:NAME="CentOS Stream" 2024-02-18T11:20:36.325 INFO:teuthology.orchestra.run.smithi012.stdout:VERSION="9" 2024-02-18T11:20:36.325 INFO:teuthology.orchestra.run.smithi012.stdout:ID="centos" 2024-02-18T11:20:36.325 INFO:teuthology.orchestra.run.smithi012.stdout:ID_LIKE="rhel fedora" 2024-02-18T11:20:36.325 INFO:teuthology.orchestra.run.smithi012.stdout:VERSION_ID="9" 2024-02-18T11:20:36.325 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM_ID="platform:el9" 2024-02-18T11:20:36.325 INFO:teuthology.orchestra.run.smithi012.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-18T11:20:36.325 INFO:teuthology.orchestra.run.smithi012.stdout:ANSI_COLOR="0;31" 2024-02-18T11:20:36.325 INFO:teuthology.orchestra.run.smithi012.stdout:LOGO="fedora-logo-icon" 2024-02-18T11:20:36.325 INFO:teuthology.orchestra.run.smithi012.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-18T11:20:36.325 INFO:teuthology.orchestra.run.smithi012.stdout:HOME_URL="https://centos.org/" 2024-02-18T11:20:36.325 INFO:teuthology.orchestra.run.smithi012.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-18T11:20:36.326 INFO:teuthology.orchestra.run.smithi012.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-18T11:20:36.326 INFO:teuthology.orchestra.run.smithi012.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-18T11:20:36.326 INFO:teuthology.lock.ops:Updating smithi012.front.sepia.ceph.com on lock server 2024-02-18T11:20:36.361 DEBUG:teuthology.orchestra.run.smithi158:> uname -m 2024-02-18T11:20:36.378 INFO:teuthology.orchestra.run.smithi158.stdout:x86_64 2024-02-18T11:20:36.379 DEBUG:teuthology.orchestra.run.smithi158:> cat /etc/os-release 2024-02-18T11:20:36.439 INFO:teuthology.orchestra.run.smithi158.stdout:NAME="CentOS Stream" 2024-02-18T11:20:36.439 INFO:teuthology.orchestra.run.smithi158.stdout:VERSION="9" 2024-02-18T11:20:36.440 INFO:teuthology.orchestra.run.smithi158.stdout:ID="centos" 2024-02-18T11:20:36.440 INFO:teuthology.orchestra.run.smithi158.stdout:ID_LIKE="rhel fedora" 2024-02-18T11:20:36.440 INFO:teuthology.orchestra.run.smithi158.stdout:VERSION_ID="9" 2024-02-18T11:20:36.440 INFO:teuthology.orchestra.run.smithi158.stdout:PLATFORM_ID="platform:el9" 2024-02-18T11:20:36.440 INFO:teuthology.orchestra.run.smithi158.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-18T11:20:36.440 INFO:teuthology.orchestra.run.smithi158.stdout:ANSI_COLOR="0;31" 2024-02-18T11:20:36.440 INFO:teuthology.orchestra.run.smithi158.stdout:LOGO="fedora-logo-icon" 2024-02-18T11:20:36.440 INFO:teuthology.orchestra.run.smithi158.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-18T11:20:36.440 INFO:teuthology.orchestra.run.smithi158.stdout:HOME_URL="https://centos.org/" 2024-02-18T11:20:36.440 INFO:teuthology.orchestra.run.smithi158.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-18T11:20:36.440 INFO:teuthology.orchestra.run.smithi158.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-18T11:20:36.440 INFO:teuthology.orchestra.run.smithi158.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-18T11:20:36.441 INFO:teuthology.lock.ops:Updating smithi158.front.sepia.ceph.com on lock server 2024-02-18T11:20:36.469 DEBUG:teuthology.orchestra.run.smithi188:> uname -m 2024-02-18T11:20:36.487 INFO:teuthology.orchestra.run.smithi188.stdout:x86_64 2024-02-18T11:20:36.488 DEBUG:teuthology.orchestra.run.smithi188:> cat /etc/os-release 2024-02-18T11:20:36.544 INFO:teuthology.orchestra.run.smithi188.stdout:NAME="CentOS Stream" 2024-02-18T11:20:36.545 INFO:teuthology.orchestra.run.smithi188.stdout:VERSION="9" 2024-02-18T11:20:36.545 INFO:teuthology.orchestra.run.smithi188.stdout:ID="centos" 2024-02-18T11:20:36.545 INFO:teuthology.orchestra.run.smithi188.stdout:ID_LIKE="rhel fedora" 2024-02-18T11:20:36.545 INFO:teuthology.orchestra.run.smithi188.stdout:VERSION_ID="9" 2024-02-18T11:20:36.545 INFO:teuthology.orchestra.run.smithi188.stdout:PLATFORM_ID="platform:el9" 2024-02-18T11:20:36.545 INFO:teuthology.orchestra.run.smithi188.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-18T11:20:36.545 INFO:teuthology.orchestra.run.smithi188.stdout:ANSI_COLOR="0;31" 2024-02-18T11:20:36.545 INFO:teuthology.orchestra.run.smithi188.stdout:LOGO="fedora-logo-icon" 2024-02-18T11:20:36.545 INFO:teuthology.orchestra.run.smithi188.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-18T11:20:36.545 INFO:teuthology.orchestra.run.smithi188.stdout:HOME_URL="https://centos.org/" 2024-02-18T11:20:36.545 INFO:teuthology.orchestra.run.smithi188.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-18T11:20:36.545 INFO:teuthology.orchestra.run.smithi188.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-18T11:20:36.546 INFO:teuthology.orchestra.run.smithi188.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-18T11:20:36.546 INFO:teuthology.lock.ops:Updating smithi188.front.sepia.ceph.com on lock server 2024-02-18T11:20:36.569 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-02-18T11:20:36.580 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-02-18T11:20:36.653 INFO:teuthology.task.internal:Checking for old test directory... 2024-02-18T11:20:36.653 DEBUG:teuthology.orchestra.run.smithi012:> test '!' -e /home/ubuntu/cephtest 2024-02-18T11:20:36.657 DEBUG:teuthology.orchestra.run.smithi158:> test '!' -e /home/ubuntu/cephtest 2024-02-18T11:20:36.660 DEBUG:teuthology.orchestra.run.smithi188:> test '!' -e /home/ubuntu/cephtest 2024-02-18T11:20:36.676 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-02-18T11:20:36.682 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-02-18T11:20:36.683 DEBUG:teuthology.orchestra.run.smithi012:> test -z $(ls -A /var/lib/ceph) 2024-02-18T11:20:36.715 DEBUG:teuthology.orchestra.run.smithi158:> test -z $(ls -A /var/lib/ceph) 2024-02-18T11:20:36.720 DEBUG:teuthology.orchestra.run.smithi188:> test -z $(ls -A /var/lib/ceph) 2024-02-18T11:20:36.817 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-02-18T11:20:36.901 INFO:teuthology.run_tasks:Running task kernel... 2024-02-18T11:20:36.915 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '8d59ab0bd93c806231018a6729893bf309624262'} 2024-02-18T11:20:36.916 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-02-18T11:20:36.916 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-02-18T11:20:36.916 DEBUG:teuthology.orchestra.run.smithi012:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-18T11:20:36.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-18T11:20:36.933 DEBUG:teuthology.orchestra.run.smithi012:> uname -r 2024-02-18T11:20:36.991 INFO:teuthology.orchestra.run.smithi012.stdout:5.14.0-361.el9.x86_64 2024-02-18T11:20:36.992 INFO:teuthology.task.kernel:Running kernel on smithi012: 5.14.0-361.el9.x86_64 2024-02-18T11:20:36.992 DEBUG:teuthology.orchestra.run.smithi012:> sudo yum install -y kernel 2024-02-18T11:20:39.079 INFO:teuthology.orchestra.run.smithi012.stdout:CentOS-9 - AppStream 17 MB/s | 6.7 MB 00:00 2024-02-18T11:20:40.529 INFO:teuthology.orchestra.run.smithi012.stdout:CentOS-9 - BaseOS 7.6 MB/s | 2.0 MB 00:00 2024-02-18T11:20:45.311 INFO:teuthology.orchestra.run.smithi012.stdout:CentOS Stream 9 - BaseOS 1.8 MB/s | 8.0 MB 00:04 2024-02-18T11:20:48.636 INFO:teuthology.orchestra.run.smithi012.stdout:CentOS Stream 9 - AppStream 8.9 MB/s | 19 MB 00:02 2024-02-18T11:20:53.587 INFO:teuthology.orchestra.run.smithi012.stdout:CentOS Stream 9 - CRB 4.9 MB/s | 6.1 MB 00:01 2024-02-18T11:20:55.354 INFO:teuthology.orchestra.run.smithi012.stdout:CentOS Stream 9 - Extras packages 25 kB/s | 15 kB 00:00 2024-02-18T11:20:56.347 INFO:teuthology.orchestra.run.smithi012.stdout:Extra Packages for Enterprise Linux 24 MB/s | 20 MB 00:00 2024-02-18T11:21:01.471 INFO:teuthology.orchestra.run.smithi012.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-02-18T11:21:03.803 INFO:teuthology.orchestra.run.smithi012.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-02-18T11:21:03.803 INFO:teuthology.orchestra.run.smithi012.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-02-18T11:21:03.892 INFO:teuthology.orchestra.run.smithi012.stdout:Dependencies resolved. 2024-02-18T11:21:03.901 INFO:teuthology.orchestra.run.smithi012.stdout:================================================================================ 2024-02-18T11:21:03.901 INFO:teuthology.orchestra.run.smithi012.stdout: Package Arch Version Repository Size 2024-02-18T11:21:03.902 INFO:teuthology.orchestra.run.smithi012.stdout:================================================================================ 2024-02-18T11:21:03.902 INFO:teuthology.orchestra.run.smithi012.stdout:Installing: 2024-02-18T11:21:03.902 INFO:teuthology.orchestra.run.smithi012.stdout: kernel x86_64 5.14.0-421.el9 CentOS-BaseOS 6.1 M 2024-02-18T11:21:03.902 INFO:teuthology.orchestra.run.smithi012.stdout:Installing dependencies: 2024-02-18T11:21:03.902 INFO:teuthology.orchestra.run.smithi012.stdout: kernel-core x86_64 5.14.0-421.el9 CentOS-BaseOS 21 M 2024-02-18T11:21:03.902 INFO:teuthology.orchestra.run.smithi012.stdout: kernel-modules x86_64 5.14.0-421.el9 CentOS-BaseOS 39 M 2024-02-18T11:21:03.902 INFO:teuthology.orchestra.run.smithi012.stdout: kernel-modules-core x86_64 5.14.0-421.el9 CentOS-BaseOS 33 M 2024-02-18T11:21:03.902 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:21:03.902 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction Summary 2024-02-18T11:21:03.902 INFO:teuthology.orchestra.run.smithi012.stdout:================================================================================ 2024-02-18T11:21:03.902 INFO:teuthology.orchestra.run.smithi012.stdout:Install 4 Packages 2024-02-18T11:21:03.902 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:21:03.903 INFO:teuthology.orchestra.run.smithi012.stdout:Total download size: 100 M 2024-02-18T11:21:03.903 INFO:teuthology.orchestra.run.smithi012.stdout:Installed size: 123 M 2024-02-18T11:21:03.903 INFO:teuthology.orchestra.run.smithi012.stdout:Downloading Packages: 2024-02-18T11:21:04.156 INFO:teuthology.orchestra.run.smithi012.stdout:(1/4): kernel-5.14.0-421.el9.x86_64.rpm 24 MB/s | 6.1 MB 00:00 2024-02-18T11:21:04.464 INFO:teuthology.orchestra.run.smithi012.stdout:(2/4): kernel-core-5.14.0-421.el9.x86_64.rpm 37 MB/s | 21 MB 00:00 2024-02-18T11:21:04.890 INFO:teuthology.orchestra.run.smithi012.stdout:(3/4): kernel-modules-core-5.14.0-421.el9.x86_6 46 MB/s | 33 MB 00:00 2024-02-18T11:21:05.265 INFO:teuthology.orchestra.run.smithi012.stdout:(4/4): kernel-modules-5.14.0-421.el9.x86_64.rpm 29 MB/s | 39 MB 00:01 2024-02-18T11:21:05.268 INFO:teuthology.orchestra.run.smithi012.stdout:-------------------------------------------------------------------------------- 2024-02-18T11:21:05.268 INFO:teuthology.orchestra.run.smithi012.stdout:Total 73 MB/s | 100 MB 00:01 2024-02-18T11:21:05.268 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction check 2024-02-18T11:21:05.720 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction check succeeded. 2024-02-18T11:21:05.720 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction test 2024-02-18T11:21:06.786 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction test succeeded. 2024-02-18T11:21:06.787 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction 2024-02-18T11:21:09.328 INFO:teuthology.orchestra.run.smithi012.stdout: Preparing : 1/1 2024-02-18T11:21:10.007 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : kernel-modules-core-5.14.0-421.el9.x86_64 1/4 2024-02-18T11:21:10.156 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : kernel-core-5.14.0-421.el9.x86_64 2/4 2024-02-18T11:21:13.092 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: kernel-core-5.14.0-421.el9.x86_64 2/4 2024-02-18T11:21:13.277 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : kernel-modules-5.14.0-421.el9.x86_64 3/4 2024-02-18T11:21:18.353 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: kernel-modules-5.14.0-421.el9.x86_64 3/4 2024-02-18T11:21:18.457 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : kernel-5.14.0-421.el9.x86_64 4/4 2024-02-18T11:21:23.557 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: kernel-modules-core-5.14.0-421.el9.x86_64 4/4 2024-02-18T11:21:54.275 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: kernel-core-5.14.0-421.el9.x86_64 4/4 2024-02-18T11:21:54.275 INFO:teuthology.orchestra.run.smithi012.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-02-18T11:21:54.275 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:21:54.349 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: kernel-modules-5.14.0-421.el9.x86_64 4/4 2024-02-18T11:21:54.752 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: kernel-5.14.0-421.el9.x86_64 4/4 2024-02-18T11:21:54.752 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : kernel-5.14.0-421.el9.x86_64 1/4 2024-02-18T11:21:54.752 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : kernel-core-5.14.0-421.el9.x86_64 2/4 2024-02-18T11:21:54.752 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : kernel-modules-5.14.0-421.el9.x86_64 3/4 2024-02-18T11:21:55.200 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : kernel-modules-core-5.14.0-421.el9.x86_64 4/4 2024-02-18T11:21:55.201 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:21:55.201 INFO:teuthology.orchestra.run.smithi012.stdout:Installed: 2024-02-18T11:21:55.201 INFO:teuthology.orchestra.run.smithi012.stdout: kernel-5.14.0-421.el9.x86_64 2024-02-18T11:21:55.201 INFO:teuthology.orchestra.run.smithi012.stdout: kernel-core-5.14.0-421.el9.x86_64 2024-02-18T11:21:55.201 INFO:teuthology.orchestra.run.smithi012.stdout: kernel-modules-5.14.0-421.el9.x86_64 2024-02-18T11:21:55.201 INFO:teuthology.orchestra.run.smithi012.stdout: kernel-modules-core-5.14.0-421.el9.x86_64 2024-02-18T11:21:55.201 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:21:55.201 INFO:teuthology.orchestra.run.smithi012.stdout:Complete! 2024-02-18T11:21:55.708 DEBUG:teuthology.orchestra.run.smithi012:> rpm -q kernel | sort -rV | head -n 1 2024-02-18T11:21:55.848 INFO:teuthology.orchestra.run.smithi012.stdout:kernel-5.14.0-421.el9.x86_64 2024-02-18T11:21:55.849 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-421.el9.x86_64 2024-02-18T11:21:55.849 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-421.el9.x86_64 2024-02-18T11:21:55.849 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-18T11:21:55.849 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-18T11:21:55.849 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-02-18T11:21:55.849 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi012.front.sepia.ceph.com, path=None, version=distro) 2024-02-18T11:21:55.849 DEBUG:teuthology.orchestra.run.smithi012:> rpm -q kernel | sort -rV | head -n 1 2024-02-18T11:21:55.962 INFO:teuthology.orchestra.run.smithi012.stdout:kernel-5.14.0-421.el9.x86_64 2024-02-18T11:21:55.963 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-421.el9.x86_64 2024-02-18T11:21:55.963 DEBUG:teuthology.orchestra.run.smithi012:> sudo rpm -qi grub2-tools 2024-02-18T11:21:56.001 INFO:teuthology.orchestra.run.smithi012.stdout:Name : grub2-tools 2024-02-18T11:21:56.001 INFO:teuthology.orchestra.run.smithi012.stdout:Epoch : 1 2024-02-18T11:21:56.001 INFO:teuthology.orchestra.run.smithi012.stdout:Version : 2.06 2024-02-18T11:21:56.001 INFO:teuthology.orchestra.run.smithi012.stdout:Release : 68.el9 2024-02-18T11:21:56.001 INFO:teuthology.orchestra.run.smithi012.stdout:Architecture: x86_64 2024-02-18T11:21:56.001 INFO:teuthology.orchestra.run.smithi012.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-02-18T11:21:56.001 INFO:teuthology.orchestra.run.smithi012.stdout:Group : Unspecified 2024-02-18T11:21:56.001 INFO:teuthology.orchestra.run.smithi012.stdout:Size : 8264254 2024-02-18T11:21:56.001 INFO:teuthology.orchestra.run.smithi012.stdout:License : GPLv3+ 2024-02-18T11:21:56.001 INFO:teuthology.orchestra.run.smithi012.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-02-18T11:21:56.001 INFO:teuthology.orchestra.run.smithi012.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-02-18T11:21:56.002 INFO:teuthology.orchestra.run.smithi012.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-02-18T11:21:56.002 INFO:teuthology.orchestra.run.smithi012.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-02-18T11:21:56.002 INFO:teuthology.orchestra.run.smithi012.stdout:Packager : builder@centos.org 2024-02-18T11:21:56.002 INFO:teuthology.orchestra.run.smithi012.stdout:Vendor : CentOS 2024-02-18T11:21:56.002 INFO:teuthology.orchestra.run.smithi012.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-18T11:21:56.002 INFO:teuthology.orchestra.run.smithi012.stdout:Summary : Support tools for GRUB. 2024-02-18T11:21:56.002 INFO:teuthology.orchestra.run.smithi012.stdout:Description : 2024-02-18T11:21:56.002 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:21:56.002 INFO:teuthology.orchestra.run.smithi012.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-18T11:21:56.002 INFO:teuthology.orchestra.run.smithi012.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-18T11:21:56.002 INFO:teuthology.orchestra.run.smithi012.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-18T11:21:56.002 INFO:teuthology.orchestra.run.smithi012.stdout:hardware devices. 2024-02-18T11:21:56.002 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:21:56.002 INFO:teuthology.orchestra.run.smithi012.stdout:This subpackage provides tools for support of all platforms. 2024-02-18T11:21:56.003 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-18T11:21:56.003 INFO:teuthology.task.kernel:Updating grub on smithi012 to boot 5.14.0-421.el9.x86_64 2024-02-18T11:21:56.004 DEBUG:teuthology.orchestra.run.smithi012:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-18T11:21:56.624 INFO:teuthology.orchestra.run.smithi012.stderr:Generating grub configuration file ... 2024-02-18T11:21:57.962 INFO:teuthology.orchestra.run.smithi012.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-02-18T11:21:57.990 INFO:teuthology.orchestra.run.smithi012.stderr:done 2024-02-18T11:21:57.992 DEBUG:teuthology.orchestra.run.smithi012:> mktemp 2024-02-18T11:21:58.008 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/tmp.lfCuWuh4nC 2024-02-18T11:21:58.009 DEBUG:teuthology.orchestra.run.smithi012:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.lfCuWuh4nC 2024-02-18T11:21:58.083 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 0666 /tmp/tmp.lfCuWuh4nC 2024-02-18T11:21:58.237 DEBUG:teuthology.orchestra.remote:smithi012:/tmp/tmp.lfCuWuh4nC is 6KB 2024-02-18T11:21:58.287 DEBUG:teuthology.orchestra.run.smithi012:> rm -fr /tmp/tmp.lfCuWuh4nC 2024-02-18T11:21:58.302 DEBUG:teuthology.orchestra.run.smithi012:> sudo /bin/ls /boot/loader/entries || true 2024-02-18T11:21:58.371 INFO:teuthology.orchestra.run.smithi012.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-02-18T11:21:58.371 INFO:teuthology.orchestra.run.smithi012.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-02-18T11:21:58.371 INFO:teuthology.orchestra.run.smithi012.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-02-18T11:21:58.371 INFO:teuthology.orchestra.run.smithi012.stdout:15949e3c88704300882482eddfd7baae-5.14.0-421.el9.x86_64.conf 2024-02-18T11:21:58.372 DEBUG:teuthology.orchestra.run.smithi012:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-421.el9.x86_64 2024-02-18T11:21:58.529 DEBUG:teuthology.orchestra.run.smithi012:> sudo shutdown -r now 2024-02-18T11:22:28.533 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-02-18T11:22:28.534 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2024-02-18T11:22:28.534 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-18T11:23:28.537 DEBUG:teuthology.orchestra.remote:timed out 2024-02-18T11:23:37.539 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2024-02-18T11:23:37.540 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-18T11:23:37.946 DEBUG:teuthology.orchestra.run.smithi012:> true 2024-02-18T11:23:38.334 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi012.front.sepia.ceph.com' 2024-02-18T11:23:38.334 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-421.el9.x86_64"... 2024-02-18T11:23:38.334 DEBUG:teuthology.orchestra.run.smithi012:> uname -r 2024-02-18T11:23:38.348 INFO:teuthology.orchestra.run.smithi012.stdout:5.14.0-421.el9.x86_64 2024-02-18T11:23:38.348 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-421.el9.x86_64 vs 5.14.0-421.el9.x86_64 2024-02-18T11:23:38.348 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-18T11:23:38.349 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-18T11:23:39.350 DEBUG:teuthology.parallel:result is None 2024-02-18T11:23:39.350 INFO:teuthology.run_tasks:Running task internal.base... 2024-02-18T11:23:39.357 INFO:teuthology.task.internal:Creating test directory... 2024-02-18T11:23:39.358 DEBUG:teuthology.orchestra.run.smithi012:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-18T11:23:39.360 DEBUG:teuthology.orchestra.run.smithi158:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-18T11:23:39.362 DEBUG:teuthology.orchestra.run.smithi188:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-18T11:23:39.380 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-02-18T11:23:39.392 INFO:teuthology.run_tasks:Running task internal.archive... 2024-02-18T11:23:39.443 INFO:teuthology.task.internal:Creating archive directory... 2024-02-18T11:23:39.443 DEBUG:teuthology.orchestra.run.smithi012:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-18T11:23:39.446 DEBUG:teuthology.orchestra.run.smithi158:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-18T11:23:39.449 DEBUG:teuthology.orchestra.run.smithi188:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-18T11:23:39.823 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-02-18T11:23:39.829 INFO:teuthology.task.internal:Enabling coredump saving... 2024-02-18T11:23:39.829 DEBUG:teuthology.orchestra.run.smithi158:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-18T11:23:39.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-18T11:23:39.846 DEBUG:teuthology.orchestra.run.smithi188:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-18T11:23:39.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-18T11:23:39.863 DEBUG:teuthology.orchestra.run.smithi012:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-18T11:23:39.865 DEBUG:teuthology.orchestra.run.smithi158:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-18T11:23:39.889 DEBUG:teuthology.orchestra.run.smithi188:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-18T11:23:39.927 INFO:teuthology.orchestra.run.smithi012.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-18T11:23:39.940 INFO:teuthology.orchestra.run.smithi188.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-18T11:23:39.949 INFO:teuthology.orchestra.run.smithi158.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-18T11:23:39.962 INFO:teuthology.orchestra.run.smithi188.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-18T11:23:39.974 INFO:teuthology.orchestra.run.smithi158.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-18T11:23:39.983 INFO:teuthology.orchestra.run.smithi012.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-18T11:23:39.985 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-02-18T11:23:39.994 INFO:teuthology.task.internal:Configuring sudo... 2024-02-18T11:23:39.994 DEBUG:teuthology.orchestra.run.smithi012:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-18T11:23:39.996 DEBUG:teuthology.orchestra.run.smithi158:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-18T11:23:39.998 DEBUG:teuthology.orchestra.run.smithi188:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-18T11:23:40.034 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-02-18T11:23:40.043 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-02-18T11:23:40.043 DEBUG:teuthology.orchestra.run.smithi012:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-18T11:23:40.066 DEBUG:teuthology.orchestra.run.smithi158:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-18T11:23:40.069 DEBUG:teuthology.orchestra.run.smithi188:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-18T11:23:40.089 DEBUG:teuthology.orchestra.run.smithi012:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-18T11:23:40.162 DEBUG:teuthology.orchestra.run.smithi012:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-18T11:23:40.220 DEBUG:teuthology.orchestra.run.smithi012:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-18T11:23:40.302 DEBUG:teuthology.orchestra.run.smithi012:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-18T11:23:40.346 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-02-18T11:23:40.346 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-18T11:23:40.414 DEBUG:teuthology.orchestra.run.smithi158:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-18T11:23:40.455 DEBUG:teuthology.orchestra.run.smithi158:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-18T11:23:40.522 DEBUG:teuthology.orchestra.run.smithi158:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-18T11:23:40.564 DEBUG:teuthology.orchestra.run.smithi158:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-18T11:23:40.610 DEBUG:teuthology.orchestra.run.smithi158:> set -ex 2024-02-18T11:23:40.610 DEBUG:teuthology.orchestra.run.smithi158:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-18T11:23:40.680 DEBUG:teuthology.orchestra.run.smithi188:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-18T11:23:40.725 DEBUG:teuthology.orchestra.run.smithi188:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-18T11:23:40.779 DEBUG:teuthology.orchestra.run.smithi188:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-18T11:23:40.864 DEBUG:teuthology.orchestra.run.smithi188:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-18T11:23:40.909 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-02-18T11:23:40.909 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-18T11:23:40.978 DEBUG:teuthology.orchestra.run.smithi012:> sudo service rsyslog restart 2024-02-18T11:23:40.981 DEBUG:teuthology.orchestra.run.smithi158:> sudo service rsyslog restart 2024-02-18T11:23:40.983 DEBUG:teuthology.orchestra.run.smithi188:> sudo service rsyslog restart 2024-02-18T11:23:41.027 INFO:teuthology.orchestra.run.smithi158.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-18T11:23:41.033 INFO:teuthology.orchestra.run.smithi012.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-18T11:23:41.060 INFO:teuthology.orchestra.run.smithi188.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-18T11:23:41.549 INFO:teuthology.run_tasks:Running task internal.timer... 2024-02-18T11:23:41.557 INFO:teuthology.task.internal:Starting timer... 2024-02-18T11:23:41.557 INFO:teuthology.run_tasks:Running task pcp... 2024-02-18T11:23:41.596 INFO:teuthology.run_tasks:Running task selinux... 2024-02-18T11:23:41.648 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-02-18T11:23:41.649 DEBUG:teuthology.orchestra.run.smithi012:> sudo service auditd rotate 2024-02-18T11:23:41.715 INFO:teuthology.orchestra.run.smithi012.stdout:Rotating logs: 2024-02-18T11:23:41.717 DEBUG:teuthology.orchestra.run.smithi158:> sudo service auditd rotate 2024-02-18T11:23:41.773 INFO:teuthology.orchestra.run.smithi158.stdout:Rotating logs: 2024-02-18T11:23:41.775 DEBUG:teuthology.orchestra.run.smithi188:> sudo service auditd rotate 2024-02-18T11:23:41.839 INFO:teuthology.orchestra.run.smithi188.stdout:Rotating logs: 2024-02-18T11:23:41.841 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-02-18T11:23:41.841 DEBUG:teuthology.orchestra.run.smithi012:> /usr/sbin/getenforce 2024-02-18T11:23:41.862 INFO:teuthology.orchestra.run.smithi012.stdout:Permissive 2024-02-18T11:23:41.863 DEBUG:teuthology.orchestra.run.smithi158:> /usr/sbin/getenforce 2024-02-18T11:23:41.888 INFO:teuthology.orchestra.run.smithi158.stdout:Permissive 2024-02-18T11:23:41.888 DEBUG:teuthology.orchestra.run.smithi188:> /usr/sbin/getenforce 2024-02-18T11:23:41.912 INFO:teuthology.orchestra.run.smithi188.stdout:Permissive 2024-02-18T11:23:41.912 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi012.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi158.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi188.front.sepia.ceph.com': 'permissive'} 2024-02-18T11:23:41.912 DEBUG:teuthology.orchestra.run.smithi012:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-18T11:23:41.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-18T11:23:41.941 DEBUG:teuthology.orchestra.run.smithi158:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-18T11:23:41.970 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-18T11:23:41.970 DEBUG:teuthology.orchestra.run.smithi188:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-18T11:23:42.000 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-18T11:23:42.001 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-02-18T11:23:42.001 DEBUG:teuthology.orchestra.run.smithi012:> sudo /usr/sbin/setenforce permissive 2024-02-18T11:23:42.031 DEBUG:teuthology.orchestra.run.smithi158:> sudo /usr/sbin/setenforce permissive 2024-02-18T11:23:42.065 DEBUG:teuthology.orchestra.run.smithi188:> sudo /usr/sbin/setenforce permissive 2024-02-18T11:23:42.096 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-02-18T11:23:42.107 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-02-18T11:23:42.125 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-02-18T11:23:42.127 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi012.front.sepia.ceph.com,smithi158.front.sepia.ceph.com,smithi188.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-02-18T11:28:12.606 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi012.front.sepia.ceph.com'), Remote(name='ubuntu@smithi158.front.sepia.ceph.com'), Remote(name='ubuntu@smithi188.front.sepia.ceph.com')] 2024-02-18T11:28:12.607 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2024-02-18T11:28:12.609 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-18T11:28:12.699 DEBUG:teuthology.orchestra.run.smithi012:> true 2024-02-18T11:28:12.784 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi012.front.sepia.ceph.com' 2024-02-18T11:28:12.784 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi158.front.sepia.ceph.com' 2024-02-18T11:28:12.785 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-18T11:28:12.872 DEBUG:teuthology.orchestra.run.smithi158:> true 2024-02-18T11:28:12.938 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi158.front.sepia.ceph.com' 2024-02-18T11:28:12.939 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi188.front.sepia.ceph.com' 2024-02-18T11:28:12.939 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-18T11:28:13.026 DEBUG:teuthology.orchestra.run.smithi188:> true 2024-02-18T11:28:13.096 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi188.front.sepia.ceph.com' 2024-02-18T11:28:13.096 INFO:teuthology.run_tasks:Running task clock... 2024-02-18T11:28:13.107 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-02-18T11:28:13.107 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-18T11:28:13.108 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-18T11:28:13.110 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-18T11:28:13.111 DEBUG:teuthology.orchestra.run.smithi158:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-18T11:28:13.113 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-18T11:28:13.113 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-18T11:28:13.142 INFO:teuthology.orchestra.run.smithi158.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-18T11:28:13.143 INFO:teuthology.orchestra.run.smithi012.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-18T11:28:13.159 INFO:teuthology.orchestra.run.smithi158.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-18T11:28:13.161 INFO:teuthology.orchestra.run.smithi012.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-18T11:28:13.168 INFO:teuthology.orchestra.run.smithi188.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-18T11:28:13.185 INFO:teuthology.orchestra.run.smithi188.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-18T11:28:13.188 INFO:teuthology.orchestra.run.smithi158.stderr:sudo: ntpd: command not found 2024-02-18T11:28:13.193 INFO:teuthology.orchestra.run.smithi012.stderr:sudo: ntpd: command not found 2024-02-18T11:28:13.201 INFO:teuthology.orchestra.run.smithi158.stdout:506 Cannot talk to daemon 2024-02-18T11:28:13.207 INFO:teuthology.orchestra.run.smithi012.stdout:506 Cannot talk to daemon 2024-02-18T11:28:13.216 INFO:teuthology.orchestra.run.smithi188.stderr:sudo: ntpd: command not found 2024-02-18T11:28:13.217 INFO:teuthology.orchestra.run.smithi158.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-18T11:28:13.224 INFO:teuthology.orchestra.run.smithi012.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-18T11:28:13.229 INFO:teuthology.orchestra.run.smithi188.stdout:506 Cannot talk to daemon 2024-02-18T11:28:13.234 INFO:teuthology.orchestra.run.smithi158.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-18T11:28:13.241 INFO:teuthology.orchestra.run.smithi012.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-18T11:28:13.246 INFO:teuthology.orchestra.run.smithi188.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-18T11:28:13.262 INFO:teuthology.orchestra.run.smithi188.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-18T11:28:13.278 INFO:teuthology.orchestra.run.smithi158.stderr:bash: line 1: ntpq: command not found 2024-02-18T11:28:13.282 INFO:teuthology.orchestra.run.smithi158.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-18T11:28:13.282 INFO:teuthology.orchestra.run.smithi158.stdout:=============================================================================== 2024-02-18T11:28:13.282 INFO:teuthology.orchestra.run.smithi158.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-18T11:28:13.282 INFO:teuthology.orchestra.run.smithi158.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-18T11:28:13.282 INFO:teuthology.orchestra.run.smithi158.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-18T11:28:13.282 INFO:teuthology.orchestra.run.smithi158.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-18T11:28:13.295 INFO:teuthology.orchestra.run.smithi012.stderr:bash: line 1: ntpq: command not found 2024-02-18T11:28:13.298 INFO:teuthology.orchestra.run.smithi012.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-18T11:28:13.298 INFO:teuthology.orchestra.run.smithi012.stdout:=============================================================================== 2024-02-18T11:28:13.298 INFO:teuthology.orchestra.run.smithi012.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-18T11:28:13.298 INFO:teuthology.orchestra.run.smithi012.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-18T11:28:13.298 INFO:teuthology.orchestra.run.smithi012.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-18T11:28:13.298 INFO:teuthology.orchestra.run.smithi012.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-18T11:28:13.306 INFO:teuthology.orchestra.run.smithi188.stderr:bash: line 1: ntpq: command not found 2024-02-18T11:28:13.309 INFO:teuthology.orchestra.run.smithi188.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-18T11:28:13.309 INFO:teuthology.orchestra.run.smithi188.stdout:=============================================================================== 2024-02-18T11:28:13.309 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-18T11:28:13.309 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-18T11:28:13.309 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-18T11:28:13.309 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-18T11:28:13.310 INFO:teuthology.run_tasks:Running task install... 2024-02-18T11:28:13.319 DEBUG:teuthology.task.install:project ceph 2024-02-18T11:28:13.319 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '1c920afa8316ea21877f3186850254947531ea22'}} 2024-02-18T11:28:13.319 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1c920afa8316ea21877f3186850254947531ea22'} 2024-02-18T11:28:13.319 INFO:teuthology.task.install:Using flavor: default 2024-02-18T11:28:13.327 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-02-18T11:28:13.327 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-02-18T11:28:13.327 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1c920afa8316ea21877f3186850254947531ea22', 'tag': None, 'wait_for_package': False} 2024-02-18T11:28:13.328 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1c920afa8316ea21877f3186850254947531ea22 2024-02-18T11:28:13.330 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1c920afa8316ea21877f3186850254947531ea22', 'tag': None, 'wait_for_package': False} 2024-02-18T11:28:13.330 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1c920afa8316ea21877f3186850254947531ea22 2024-02-18T11:28:13.331 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1c920afa8316ea21877f3186850254947531ea22', 'tag': None, 'wait_for_package': False} 2024-02-18T11:28:13.331 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1c920afa8316ea21877f3186850254947531ea22 2024-02-18T11:28:13.473 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240217.015652/1c920afa8316ea21877f3186850254947531ea22/centos/9/flavors/default/ 2024-02-18T11:28:13.473 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1555.g1c920afa 2024-02-18T11:28:13.478 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240217.015652/1c920afa8316ea21877f3186850254947531ea22/centos/9/flavors/default/ 2024-02-18T11:28:13.478 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1555.g1c920afa 2024-02-18T11:28:13.487 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240217.015652/1c920afa8316ea21877f3186850254947531ea22/centos/9/flavors/default/ 2024-02-18T11:28:13.488 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1555.g1c920afa 2024-02-18T11:28:13.595 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240217.015652/1c920afa8316ea21877f3186850254947531ea22/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240217.015652/1c920afa8316ea21877f3186850254947531ea22/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240217.015652/1c920afa8316ea21877f3186850254947531ea22/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-18T11:28:13.595 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-02-18T11:28:13.595 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-18T11:28:13.624 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-02-18T11:28:13.624 DEBUG:teuthology.orchestra.run.smithi188:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1c920afa8316ea21877f3186850254947531ea22/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-18T11:28:13.645 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240217.015652/1c920afa8316ea21877f3186850254947531ea22/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240217.015652/1c920afa8316ea21877f3186850254947531ea22/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240217.015652/1c920afa8316ea21877f3186850254947531ea22/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-18T11:28:13.645 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-02-18T11:28:13.646 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-18T11:28:13.661 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240217.015652/1c920afa8316ea21877f3186850254947531ea22/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240217.015652/1c920afa8316ea21877f3186850254947531ea22/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240217.015652/1c920afa8316ea21877f3186850254947531ea22/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-18T11:28:13.661 DEBUG:teuthology.orchestra.run.smithi158:> set -ex 2024-02-18T11:28:13.661 DEBUG:teuthology.orchestra.run.smithi158:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-18T11:28:13.674 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-02-18T11:28:13.674 DEBUG:teuthology.orchestra.run.smithi012:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1c920afa8316ea21877f3186850254947531ea22/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-18T11:28:13.689 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-02-18T11:28:13.689 DEBUG:teuthology.orchestra.run.smithi158:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1c920afa8316ea21877f3186850254947531ea22/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-18T11:28:13.691 DEBUG:teuthology.orchestra.run.smithi188:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-18T11:28:13.743 DEBUG:teuthology.orchestra.run.smithi012:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-18T11:28:13.757 DEBUG:teuthology.orchestra.run.smithi158:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-18T11:28:13.769 DEBUG:teuthology.orchestra.run.smithi188:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-18T11:28:13.822 DEBUG:teuthology.orchestra.run.smithi012:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-18T11:28:13.836 INFO:teuthology.orchestra.run.smithi188.stdout:check_obsoletes = 1 2024-02-18T11:28:13.837 DEBUG:teuthology.orchestra.run.smithi158:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-18T11:28:13.838 DEBUG:teuthology.orchestra.run.smithi188:> sudo yum clean all 2024-02-18T11:28:13.889 INFO:teuthology.orchestra.run.smithi012.stdout:check_obsoletes = 1 2024-02-18T11:28:13.890 DEBUG:teuthology.orchestra.run.smithi012:> sudo yum clean all 2024-02-18T11:28:13.903 INFO:teuthology.orchestra.run.smithi158.stdout:check_obsoletes = 1 2024-02-18T11:28:13.904 DEBUG:teuthology.orchestra.run.smithi158:> sudo yum clean all 2024-02-18T11:28:14.070 INFO:teuthology.orchestra.run.smithi188.stdout:74 files removed 2024-02-18T11:28:14.097 DEBUG:teuthology.orchestra.run.smithi188:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-02-18T11:28:14.140 INFO:teuthology.orchestra.run.smithi158.stdout:74 files removed 2024-02-18T11:28:14.151 INFO:teuthology.orchestra.run.smithi012.stdout:74 files removed 2024-02-18T11:28:14.167 DEBUG:teuthology.orchestra.run.smithi158:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-02-18T11:28:14.178 DEBUG:teuthology.orchestra.run.smithi012:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-02-18T11:28:14.723 INFO:teuthology.orchestra.run.smithi188.stdout:ceph packages for x86_64 246 kB/s | 82 kB 00:00 2024-02-18T11:28:14.856 INFO:teuthology.orchestra.run.smithi158.stdout:ceph packages for x86_64 205 kB/s | 82 kB 00:00 2024-02-18T11:28:14.902 INFO:teuthology.orchestra.run.smithi012.stdout:ceph packages for x86_64 204 kB/s | 82 kB 00:00 2024-02-18T11:28:15.040 INFO:teuthology.orchestra.run.smithi188.stdout:ceph noarch packages 36 kB/s | 10 kB 00:00 2024-02-18T11:28:15.232 INFO:teuthology.orchestra.run.smithi158.stdout:ceph noarch packages 30 kB/s | 10 kB 00:00 2024-02-18T11:28:15.277 INFO:teuthology.orchestra.run.smithi012.stdout:ceph noarch packages 31 kB/s | 10 kB 00:00 2024-02-18T11:28:15.432 INFO:teuthology.orchestra.run.smithi188.stdout:ceph source packages 5.2 kB/s | 1.9 kB 00:00 2024-02-18T11:28:15.575 INFO:teuthology.orchestra.run.smithi158.stdout:ceph source packages 6.1 kB/s | 1.9 kB 00:00 2024-02-18T11:28:15.686 INFO:teuthology.orchestra.run.smithi012.stdout:ceph source packages 5.2 kB/s | 1.9 kB 00:00 2024-02-18T11:28:15.801 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS-9 - AppStream 20 MB/s | 6.7 MB 00:00 2024-02-18T11:28:15.960 INFO:teuthology.orchestra.run.smithi158.stdout:CentOS-9 - AppStream 19 MB/s | 6.7 MB 00:00 2024-02-18T11:28:16.247 INFO:teuthology.orchestra.run.smithi012.stdout:CentOS-9 - AppStream 13 MB/s | 6.7 MB 00:00 2024-02-18T11:28:17.293 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS-9 - BaseOS 6.8 MB/s | 2.0 MB 00:00 2024-02-18T11:28:17.344 INFO:teuthology.orchestra.run.smithi158.stdout:CentOS-9 - BaseOS 6.9 MB/s | 2.0 MB 00:00 2024-02-18T11:28:17.647 INFO:teuthology.orchestra.run.smithi012.stdout:CentOS-9 - BaseOS 8.0 MB/s | 2.0 MB 00:00 2024-02-18T11:28:18.059 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - BaseOS 18 MB/s | 8.0 MB 00:00 2024-02-18T11:28:18.510 INFO:teuthology.orchestra.run.smithi158.stdout:CentOS Stream 9 - BaseOS 9.5 MB/s | 8.0 MB 00:00 2024-02-18T11:28:20.054 INFO:teuthology.orchestra.run.smithi012.stdout:CentOS Stream 9 - BaseOS 3.9 MB/s | 8.0 MB 00:02 2024-02-18T11:28:20.260 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - AppStream 17 MB/s | 19 MB 00:01 2024-02-18T11:28:20.420 INFO:teuthology.orchestra.run.smithi158.stdout:CentOS Stream 9 - AppStream 23 MB/s | 19 MB 00:00 2024-02-18T11:28:21.847 INFO:teuthology.orchestra.run.smithi012.stdout:CentOS Stream 9 - AppStream 28 MB/s | 19 MB 00:00 2024-02-18T11:28:24.412 INFO:teuthology.orchestra.run.smithi158.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.1 MB 00:00 2024-02-18T11:28:24.652 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - CRB 7.3 MB/s | 6.1 MB 00:00 2024-02-18T11:28:25.896 INFO:teuthology.orchestra.run.smithi158.stdout:CentOS Stream 9 - Extras packages 43 kB/s | 15 kB 00:00 2024-02-18T11:28:26.269 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - Extras packages 30 kB/s | 15 kB 00:00 2024-02-18T11:28:26.814 INFO:teuthology.orchestra.run.smithi012.stdout:CentOS Stream 9 - CRB 4.7 MB/s | 6.1 MB 00:01 2024-02-18T11:28:27.965 INFO:teuthology.orchestra.run.smithi158.stdout:Extra Packages for Enterprise Linux 10 MB/s | 20 MB 00:01 2024-02-18T11:28:28.915 INFO:teuthology.orchestra.run.smithi012.stdout:CentOS Stream 9 - Extras packages 16 kB/s | 15 kB 00:00 2024-02-18T11:28:30.768 INFO:teuthology.orchestra.run.smithi012.stdout:Extra Packages for Enterprise Linux 12 MB/s | 20 MB 00:01 2024-02-18T11:28:31.687 INFO:teuthology.orchestra.run.smithi188.stdout:Extra Packages for Enterprise Linux 3.8 MB/s | 20 MB 00:05 2024-02-18T11:28:33.106 INFO:teuthology.orchestra.run.smithi158.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-02-18T11:28:35.620 INFO:teuthology.orchestra.run.smithi158.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-02-18T11:28:35.620 INFO:teuthology.orchestra.run.smithi158.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-02-18T11:28:35.633 INFO:teuthology.orchestra.run.smithi158.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-02-18T11:28:35.634 INFO:teuthology.orchestra.run.smithi158.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-02-18T11:28:35.635 INFO:teuthology.orchestra.run.smithi158.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-02-18T11:28:35.636 INFO:teuthology.orchestra.run.smithi158.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-02-18T11:28:35.638 INFO:teuthology.orchestra.run.smithi158.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-02-18T11:28:35.641 INFO:teuthology.orchestra.run.smithi158.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-02-18T11:28:35.642 INFO:teuthology.orchestra.run.smithi158.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-02-18T11:28:35.724 INFO:teuthology.orchestra.run.smithi158.stdout:Dependencies resolved. 2024-02-18T11:28:35.747 INFO:teuthology.orchestra.run.smithi158.stdout:============================================================================================ 2024-02-18T11:28:35.748 INFO:teuthology.orchestra.run.smithi158.stdout: Package Arch Version Repository Size 2024-02-18T11:28:35.748 INFO:teuthology.orchestra.run.smithi158.stdout:============================================================================================ 2024-02-18T11:28:35.748 INFO:teuthology.orchestra.run.smithi158.stdout:Installing: 2024-02-18T11:28:35.748 INFO:teuthology.orchestra.run.smithi158.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-02-18T11:28:35.748 INFO:teuthology.orchestra.run.smithi158.stdout: ceph x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 6.6 k 2024-02-18T11:28:35.748 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-base x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 5.3 M 2024-02-18T11:28:35.748 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-fuse x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 886 k 2024-02-18T11:28:35.748 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 145 k 2024-02-18T11:28:35.748 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-mgr x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 1.5 M 2024-02-18T11:28:35.748 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1555.g1c920afa.el9 ceph-noarch 138 k 2024-02-18T11:28:35.749 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1555.g1c920afa.el9 ceph-noarch 1.8 M 2024-02-18T11:28:35.749 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1555.g1c920afa.el9 ceph-noarch 7.4 M 2024-02-18T11:28:35.749 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-mgr-rook noarch 2:19.0.0-1555.g1c920afa.el9 ceph-noarch 52 k 2024-02-18T11:28:35.749 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-radosgw x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 9.8 M 2024-02-18T11:28:35.749 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-test x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 47 M 2024-02-18T11:28:35.749 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-volume noarch 2:19.0.0-1555.g1c920afa.el9 ceph-noarch 264 k 2024-02-18T11:28:35.749 INFO:teuthology.orchestra.run.smithi158.stdout: cephadm noarch 2:19.0.0-1555.g1c920afa.el9 ceph-noarch 748 k 2024-02-18T11:28:35.749 INFO:teuthology.orchestra.run.smithi158.stdout: cephfs-mirror x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 220 k 2024-02-18T11:28:35.749 INFO:teuthology.orchestra.run.smithi158.stdout: cephfs-shell x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 27 k 2024-02-18T11:28:35.749 INFO:teuthology.orchestra.run.smithi158.stdout: cephfs-top noarch 2:19.0.0-1555.g1c920afa.el9 ceph-noarch 18 k 2024-02-18T11:28:35.749 INFO:teuthology.orchestra.run.smithi158.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 CentOS-AppStream 20 k 2024-02-18T11:28:35.749 INFO:teuthology.orchestra.run.smithi158.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-02-18T11:28:35.749 INFO:teuthology.orchestra.run.smithi158.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-02-18T11:28:35.749 INFO:teuthology.orchestra.run.smithi158.stdout: libcephfs-devel x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 31 k 2024-02-18T11:28:35.750 INFO:teuthology.orchestra.run.smithi158.stdout: libcephfs2 x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 742 k 2024-02-18T11:28:35.750 INFO:teuthology.orchestra.run.smithi158.stdout: librados-devel x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 127 k 2024-02-18T11:28:35.750 INFO:teuthology.orchestra.run.smithi158.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-02-18T11:28:35.750 INFO:teuthology.orchestra.run.smithi158.stdout: libuuid-devel x86_64 2.37.4-18.el9 CentOS-AppStream 24 k 2024-02-18T11:28:35.750 INFO:teuthology.orchestra.run.smithi158.stdout: openssl-devel x86_64 1:3.0.7-25.el9 CentOS-AppStream 3.0 M 2024-02-18T11:28:35.750 INFO:teuthology.orchestra.run.smithi158.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-02-18T11:28:35.750 INFO:teuthology.orchestra.run.smithi158.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-02-18T11:28:35.750 INFO:teuthology.orchestra.run.smithi158.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-02-18T11:28:35.750 INFO:teuthology.orchestra.run.smithi158.stdout: python3-cephfs x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 161 k 2024-02-18T11:28:35.750 INFO:teuthology.orchestra.run.smithi158.stdout: python3-rados x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 320 k 2024-02-18T11:28:35.750 INFO:teuthology.orchestra.run.smithi158.stdout: python3-rbd x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 298 k 2024-02-18T11:28:35.750 INFO:teuthology.orchestra.run.smithi158.stdout: python3-rgw x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 99 k 2024-02-18T11:28:35.750 INFO:teuthology.orchestra.run.smithi158.stdout: rbd-fuse x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 85 k 2024-02-18T11:28:35.751 INFO:teuthology.orchestra.run.smithi158.stdout: rbd-mirror x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 3.1 M 2024-02-18T11:28:35.751 INFO:teuthology.orchestra.run.smithi158.stdout: rbd-nbd x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 171 k 2024-02-18T11:28:35.751 INFO:teuthology.orchestra.run.smithi158.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 CentOS-AppStream 87 k 2024-02-18T11:28:35.751 INFO:teuthology.orchestra.run.smithi158.stdout:Upgrading: 2024-02-18T11:28:35.751 INFO:teuthology.orchestra.run.smithi158.stdout: NetworkManager x86_64 1:1.45.91-1.el9 CentOS-BaseOS 2.3 M 2024-02-18T11:28:35.751 INFO:teuthology.orchestra.run.smithi158.stdout: NetworkManager-libnm x86_64 1:1.45.91-1.el9 CentOS-BaseOS 1.8 M 2024-02-18T11:28:35.751 INFO:teuthology.orchestra.run.smithi158.stdout: NetworkManager-team x86_64 1:1.45.91-1.el9 CentOS-BaseOS 41 k 2024-02-18T11:28:35.751 INFO:teuthology.orchestra.run.smithi158.stdout: NetworkManager-tui x86_64 1:1.45.91-1.el9 CentOS-BaseOS 246 k 2024-02-18T11:28:35.751 INFO:teuthology.orchestra.run.smithi158.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2024-02-18T11:28:35.751 INFO:teuthology.orchestra.run.smithi158.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 CentOS-BaseOS 35 k 2024-02-18T11:28:35.751 INFO:teuthology.orchestra.run.smithi158.stdout: elfutils-libelf x86_64 0.190-2.el9 CentOS-BaseOS 191 k 2024-02-18T11:28:35.751 INFO:teuthology.orchestra.run.smithi158.stdout: elfutils-libs x86_64 0.190-2.el9 CentOS-BaseOS 254 k 2024-02-18T11:28:35.751 INFO:teuthology.orchestra.run.smithi158.stdout: iproute x86_64 6.7.0-2.el9 CentOS-BaseOS 802 k 2024-02-18T11:28:35.751 INFO:teuthology.orchestra.run.smithi158.stdout: iproute-tc x86_64 6.7.0-2.el9 CentOS-BaseOS 433 k 2024-02-18T11:28:35.752 INFO:teuthology.orchestra.run.smithi158.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2024-02-18T11:28:35.752 INFO:teuthology.orchestra.run.smithi158.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2024-02-18T11:28:35.752 INFO:teuthology.orchestra.run.smithi158.stdout: libblkid x86_64 2.37.4-18.el9 CentOS-BaseOS 107 k 2024-02-18T11:28:35.752 INFO:teuthology.orchestra.run.smithi158.stdout: libfdisk x86_64 2.37.4-18.el9 CentOS-BaseOS 154 k 2024-02-18T11:28:35.752 INFO:teuthology.orchestra.run.smithi158.stdout: libmount x86_64 2.37.4-18.el9 CentOS-BaseOS 135 k 2024-02-18T11:28:35.752 INFO:teuthology.orchestra.run.smithi158.stdout: librados2 x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 3.4 M 2024-02-18T11:28:35.752 INFO:teuthology.orchestra.run.smithi158.stdout: librbd1 x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 3.1 M 2024-02-18T11:28:35.752 INFO:teuthology.orchestra.run.smithi158.stdout: libsmartcols x86_64 2.37.4-18.el9 CentOS-BaseOS 63 k 2024-02-18T11:28:35.752 INFO:teuthology.orchestra.run.smithi158.stdout: libuuid x86_64 2.37.4-18.el9 CentOS-BaseOS 28 k 2024-02-18T11:28:35.752 INFO:teuthology.orchestra.run.smithi158.stdout: openssl x86_64 1:3.0.7-25.el9 CentOS-BaseOS 1.2 M 2024-02-18T11:28:35.752 INFO:teuthology.orchestra.run.smithi158.stdout: openssl-libs x86_64 1:3.0.7-25.el9 CentOS-BaseOS 2.1 M 2024-02-18T11:28:35.752 INFO:teuthology.orchestra.run.smithi158.stdout: util-linux x86_64 2.37.4-18.el9 CentOS-BaseOS 2.2 M 2024-02-18T11:28:35.752 INFO:teuthology.orchestra.run.smithi158.stdout: util-linux-core x86_64 2.37.4-18.el9 CentOS-BaseOS 438 k 2024-02-18T11:28:35.752 INFO:teuthology.orchestra.run.smithi158.stdout: util-linux-user x86_64 2.37.4-18.el9 CentOS-BaseOS 30 k 2024-02-18T11:28:35.753 INFO:teuthology.orchestra.run.smithi158.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-02-18T11:28:35.753 INFO:teuthology.orchestra.run.smithi158.stdout: xfsprogs x86_64 6.3.0-1.el9 CentOS-BaseOS 1.1 M 2024-02-18T11:28:35.753 INFO:teuthology.orchestra.run.smithi158.stdout:Installing dependencies: 2024-02-18T11:28:35.753 INFO:teuthology.orchestra.run.smithi158.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-02-18T11:28:35.753 INFO:teuthology.orchestra.run.smithi158.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-02-18T11:28:35.753 INFO:teuthology.orchestra.run.smithi158.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-02-18T11:28:35.753 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-common x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 20 M 2024-02-18T11:28:35.753 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1555.g1c920afa.el9 ceph-noarch 31 k 2024-02-18T11:28:35.753 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-mds x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 2.1 M 2024-02-18T11:28:35.753 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1555.g1c920afa.el9 ceph-noarch 248 k 2024-02-18T11:28:35.753 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-mon x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 4.7 M 2024-02-18T11:28:35.753 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-osd x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 17 M 2024-02-18T11:28:35.753 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1555.g1c920afa.el9 ceph-noarch 15 k 2024-02-18T11:28:35.753 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-selinux x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 25 k 2024-02-18T11:28:35.753 INFO:teuthology.orchestra.run.smithi158.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-02-18T11:28:35.754 INFO:teuthology.orchestra.run.smithi158.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-02-18T11:28:35.754 INFO:teuthology.orchestra.run.smithi158.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-02-18T11:28:35.754 INFO:teuthology.orchestra.run.smithi158.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-02-18T11:28:35.754 INFO:teuthology.orchestra.run.smithi158.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-02-18T11:28:35.754 INFO:teuthology.orchestra.run.smithi158.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-02-18T11:28:35.754 INFO:teuthology.orchestra.run.smithi158.stdout: jq x86_64 1.6-16.el9 CentOS-BaseOS 186 k 2024-02-18T11:28:35.754 INFO:teuthology.orchestra.run.smithi158.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-02-18T11:28:35.754 INFO:teuthology.orchestra.run.smithi158.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-02-18T11:28:35.754 INFO:teuthology.orchestra.run.smithi158.stdout: libcephsqlite x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 163 k 2024-02-18T11:28:35.754 INFO:teuthology.orchestra.run.smithi158.stdout: libgfortran x86_64 11.4.1-3.el9 CentOS-BaseOS 802 k 2024-02-18T11:28:35.754 INFO:teuthology.orchestra.run.smithi158.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-02-18T11:28:35.754 INFO:teuthology.orchestra.run.smithi158.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-02-18T11:28:35.754 INFO:teuthology.orchestra.run.smithi158.stdout: libquadmath x86_64 11.4.1-3.el9 CentOS-BaseOS 194 k 2024-02-18T11:28:35.755 INFO:teuthology.orchestra.run.smithi158.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-02-18T11:28:35.755 INFO:teuthology.orchestra.run.smithi158.stdout: libradosstriper1 x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 499 k 2024-02-18T11:28:35.755 INFO:teuthology.orchestra.run.smithi158.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-02-18T11:28:35.755 INFO:teuthology.orchestra.run.smithi158.stdout: librgw2 x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 5.1 M 2024-02-18T11:28:35.755 INFO:teuthology.orchestra.run.smithi158.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-02-18T11:28:35.755 INFO:teuthology.orchestra.run.smithi158.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-02-18T11:28:35.755 INFO:teuthology.orchestra.run.smithi158.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-02-18T11:28:35.755 INFO:teuthology.orchestra.run.smithi158.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-02-18T11:28:35.755 INFO:teuthology.orchestra.run.smithi158.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-02-18T11:28:35.755 INFO:teuthology.orchestra.run.smithi158.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-02-18T11:28:35.755 INFO:teuthology.orchestra.run.smithi158.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-BaseOS 217 k 2024-02-18T11:28:35.755 INFO:teuthology.orchestra.run.smithi158.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-02-18T11:28:35.755 INFO:teuthology.orchestra.run.smithi158.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-02-18T11:28:35.755 INFO:teuthology.orchestra.run.smithi158.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-02-18T11:28:35.755 INFO:teuthology.orchestra.run.smithi158.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-02-18T11:28:35.756 INFO:teuthology.orchestra.run.smithi158.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-02-18T11:28:35.756 INFO:teuthology.orchestra.run.smithi158.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-02-18T11:28:35.756 INFO:teuthology.orchestra.run.smithi158.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-02-18T11:28:35.756 INFO:teuthology.orchestra.run.smithi158.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-02-18T11:28:35.756 INFO:teuthology.orchestra.run.smithi158.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-02-18T11:28:35.756 INFO:teuthology.orchestra.run.smithi158.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-02-18T11:28:35.756 INFO:teuthology.orchestra.run.smithi158.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-02-18T11:28:35.756 INFO:teuthology.orchestra.run.smithi158.stdout: python3-ceph-argparse x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 45 k 2024-02-18T11:28:35.756 INFO:teuthology.orchestra.run.smithi158.stdout: python3-ceph-common x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 131 k 2024-02-18T11:28:35.756 INFO:teuthology.orchestra.run.smithi158.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-02-18T11:28:35.756 INFO:teuthology.orchestra.run.smithi158.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-02-18T11:28:35.756 INFO:teuthology.orchestra.run.smithi158.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-02-18T11:28:35.756 INFO:teuthology.orchestra.run.smithi158.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-02-18T11:28:35.756 INFO:teuthology.orchestra.run.smithi158.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-02-18T11:28:35.757 INFO:teuthology.orchestra.run.smithi158.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-02-18T11:28:35.757 INFO:teuthology.orchestra.run.smithi158.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-02-18T11:28:35.757 INFO:teuthology.orchestra.run.smithi158.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-02-18T11:28:35.757 INFO:teuthology.orchestra.run.smithi158.stdout: python3-google-auth noarch 1:2.27.0-1.el9 epel 203 k 2024-02-18T11:28:35.757 INFO:teuthology.orchestra.run.smithi158.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-02-18T11:28:35.757 INFO:teuthology.orchestra.run.smithi158.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-02-18T11:28:35.757 INFO:teuthology.orchestra.run.smithi158.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-02-18T11:28:35.757 INFO:teuthology.orchestra.run.smithi158.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-02-18T11:28:35.757 INFO:teuthology.orchestra.run.smithi158.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-02-18T11:28:35.757 INFO:teuthology.orchestra.run.smithi158.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-02-18T11:28:35.757 INFO:teuthology.orchestra.run.smithi158.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-02-18T11:28:35.757 INFO:teuthology.orchestra.run.smithi158.stdout: python3-jinja2 noarch 2.11.3-5.el9 CentOS-AppStream 227 k 2024-02-18T11:28:35.757 INFO:teuthology.orchestra.run.smithi158.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-02-18T11:28:35.758 INFO:teuthology.orchestra.run.smithi158.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-02-18T11:28:35.758 INFO:teuthology.orchestra.run.smithi158.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-02-18T11:28:35.758 INFO:teuthology.orchestra.run.smithi158.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-02-18T11:28:35.758 INFO:teuthology.orchestra.run.smithi158.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-02-18T11:28:35.758 INFO:teuthology.orchestra.run.smithi158.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-02-18T11:28:35.758 INFO:teuthology.orchestra.run.smithi158.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-02-18T11:28:35.758 INFO:teuthology.orchestra.run.smithi158.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-02-18T11:28:35.758 INFO:teuthology.orchestra.run.smithi158.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-02-18T11:28:35.758 INFO:teuthology.orchestra.run.smithi158.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-02-18T11:28:35.758 INFO:teuthology.orchestra.run.smithi158.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-02-18T11:28:35.758 INFO:teuthology.orchestra.run.smithi158.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-02-18T11:28:35.758 INFO:teuthology.orchestra.run.smithi158.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-02-18T11:28:35.758 INFO:teuthology.orchestra.run.smithi158.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-02-18T11:28:35.758 INFO:teuthology.orchestra.run.smithi158.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-02-18T11:28:35.758 INFO:teuthology.orchestra.run.smithi158.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-02-18T11:28:35.759 INFO:teuthology.orchestra.run.smithi158.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-02-18T11:28:35.759 INFO:teuthology.orchestra.run.smithi158.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-02-18T11:28:35.759 INFO:teuthology.orchestra.run.smithi158.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-02-18T11:28:35.759 INFO:teuthology.orchestra.run.smithi158.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-02-18T11:28:35.759 INFO:teuthology.orchestra.run.smithi158.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-02-18T11:28:35.759 INFO:teuthology.orchestra.run.smithi158.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-02-18T11:28:35.759 INFO:teuthology.orchestra.run.smithi158.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-02-18T11:28:35.759 INFO:teuthology.orchestra.run.smithi158.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-02-18T11:28:35.759 INFO:teuthology.orchestra.run.smithi158.stdout: python3-requests noarch 2.25.1-8.el9 CentOS-BaseOS 113 k 2024-02-18T11:28:35.759 INFO:teuthology.orchestra.run.smithi158.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-02-18T11:28:35.759 INFO:teuthology.orchestra.run.smithi158.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-02-18T11:28:35.759 INFO:teuthology.orchestra.run.smithi158.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-02-18T11:28:35.759 INFO:teuthology.orchestra.run.smithi158.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-02-18T11:28:35.759 INFO:teuthology.orchestra.run.smithi158.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-02-18T11:28:35.759 INFO:teuthology.orchestra.run.smithi158.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-02-18T11:28:35.760 INFO:teuthology.orchestra.run.smithi158.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-02-18T11:28:35.760 INFO:teuthology.orchestra.run.smithi158.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-02-18T11:28:35.760 INFO:teuthology.orchestra.run.smithi158.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-02-18T11:28:35.760 INFO:teuthology.orchestra.run.smithi158.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-02-18T11:28:35.760 INFO:teuthology.orchestra.run.smithi158.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-02-18T11:28:35.760 INFO:teuthology.orchestra.run.smithi158.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-02-18T11:28:35.760 INFO:teuthology.orchestra.run.smithi158.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-02-18T11:28:35.760 INFO:teuthology.orchestra.run.smithi158.stdout: qatlib x86_64 23.11.0-1.el9 CentOS-AppStream 215 k 2024-02-18T11:28:35.760 INFO:teuthology.orchestra.run.smithi158.stdout: qatzip-libs x86_64 1.1.2-2.el9 CentOS-AppStream 43 k 2024-02-18T11:28:35.760 INFO:teuthology.orchestra.run.smithi158.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-02-18T11:28:35.760 INFO:teuthology.orchestra.run.smithi158.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-02-18T11:28:35.760 INFO:teuthology.orchestra.run.smithi158.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-02-18T11:28:35.760 INFO:teuthology.orchestra.run.smithi158.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-02-18T11:28:35.761 INFO:teuthology.orchestra.run.smithi158.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-02-18T11:28:35.761 INFO:teuthology.orchestra.run.smithi158.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-02-18T11:28:35.761 INFO:teuthology.orchestra.run.smithi158.stdout:Installing weak dependencies: 2024-02-18T11:28:35.761 INFO:teuthology.orchestra.run.smithi158.stdout: qatlib-service x86_64 23.11.0-1.el9 CentOS-AppStream 34 k 2024-02-18T11:28:35.761 INFO:teuthology.orchestra.run.smithi158.stdout: 2024-02-18T11:28:35.761 INFO:teuthology.orchestra.run.smithi158.stdout:Transaction Summary 2024-02-18T11:28:35.761 INFO:teuthology.orchestra.run.smithi158.stdout:============================================================================================ 2024-02-18T11:28:35.761 INFO:teuthology.orchestra.run.smithi158.stdout:Install 150 Packages 2024-02-18T11:28:35.761 INFO:teuthology.orchestra.run.smithi158.stdout:Upgrade 26 Packages 2024-02-18T11:28:35.761 INFO:teuthology.orchestra.run.smithi158.stdout: 2024-02-18T11:28:35.762 INFO:teuthology.orchestra.run.smithi158.stdout:Total download size: 223 M 2024-02-18T11:28:35.762 INFO:teuthology.orchestra.run.smithi158.stdout:Downloading Packages: 2024-02-18T11:28:35.840 INFO:teuthology.orchestra.run.smithi012.stdout:lab-extras 16 kB/s | 1.7 kB 00:00 2024-02-18T11:28:36.249 INFO:teuthology.orchestra.run.smithi158.stdout:(1/176): ceph-19.0.0-1555.g1c920afa.el9.x86_64. 37 kB/s | 6.6 kB 00:00 2024-02-18T11:28:36.466 INFO:teuthology.orchestra.run.smithi158.stdout:(2/176): ceph-fuse-19.0.0-1555.g1c920afa.el9.x8 4.0 MB/s | 886 kB 00:00 2024-02-18T11:28:36.525 INFO:teuthology.orchestra.run.smithi158.stdout:(3/176): ceph-immutable-object-cache-19.0.0-155 2.4 MB/s | 145 kB 00:00 2024-02-18T11:28:36.686 INFO:teuthology.orchestra.run.smithi188.stdout:lab-extras 11 kB/s | 1.7 kB 00:00 2024-02-18T11:28:36.884 INFO:teuthology.orchestra.run.smithi158.stdout:(4/176): ceph-base-19.0.0-1555.g1c920afa.el9.x8 6.6 MB/s | 5.3 MB 00:00 2024-02-18T11:28:36.934 INFO:teuthology.orchestra.run.smithi158.stdout:(5/176): ceph-mds-19.0.0-1555.g1c920afa.el9.x86 5.2 MB/s | 2.1 MB 00:00 2024-02-18T11:28:37.075 INFO:teuthology.orchestra.run.smithi158.stdout:(6/176): ceph-mgr-19.0.0-1555.g1c920afa.el9.x86 7.8 MB/s | 1.5 MB 00:00 2024-02-18T11:28:37.709 INFO:teuthology.orchestra.run.smithi158.stdout:(7/176): ceph-mon-19.0.0-1555.g1c920afa.el9.x86 6.0 MB/s | 4.7 MB 00:00 2024-02-18T11:28:38.486 INFO:teuthology.orchestra.run.smithi012.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-02-18T11:28:38.487 INFO:teuthology.orchestra.run.smithi012.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-02-18T11:28:38.502 INFO:teuthology.orchestra.run.smithi012.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-02-18T11:28:38.503 INFO:teuthology.orchestra.run.smithi012.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-02-18T11:28:38.504 INFO:teuthology.orchestra.run.smithi012.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-02-18T11:28:38.505 INFO:teuthology.orchestra.run.smithi012.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-02-18T11:28:38.507 INFO:teuthology.orchestra.run.smithi012.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-02-18T11:28:38.510 INFO:teuthology.orchestra.run.smithi012.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-02-18T11:28:38.511 INFO:teuthology.orchestra.run.smithi012.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-02-18T11:28:38.598 INFO:teuthology.orchestra.run.smithi012.stdout:Dependencies resolved. 2024-02-18T11:28:38.616 INFO:teuthology.orchestra.run.smithi012.stdout:============================================================================================ 2024-02-18T11:28:38.616 INFO:teuthology.orchestra.run.smithi012.stdout: Package Arch Version Repository Size 2024-02-18T11:28:38.616 INFO:teuthology.orchestra.run.smithi012.stdout:============================================================================================ 2024-02-18T11:28:38.616 INFO:teuthology.orchestra.run.smithi012.stdout:Installing: 2024-02-18T11:28:38.616 INFO:teuthology.orchestra.run.smithi012.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-02-18T11:28:38.616 INFO:teuthology.orchestra.run.smithi012.stdout: ceph x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 6.6 k 2024-02-18T11:28:38.617 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-base x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 5.3 M 2024-02-18T11:28:38.617 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-fuse x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 886 k 2024-02-18T11:28:38.617 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 145 k 2024-02-18T11:28:38.617 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 1.5 M 2024-02-18T11:28:38.617 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1555.g1c920afa.el9 ceph-noarch 138 k 2024-02-18T11:28:38.617 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1555.g1c920afa.el9 ceph-noarch 1.8 M 2024-02-18T11:28:38.617 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1555.g1c920afa.el9 ceph-noarch 7.4 M 2024-02-18T11:28:38.617 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-rook noarch 2:19.0.0-1555.g1c920afa.el9 ceph-noarch 52 k 2024-02-18T11:28:38.617 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-radosgw x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 9.8 M 2024-02-18T11:28:38.617 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-test x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 47 M 2024-02-18T11:28:38.617 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-volume noarch 2:19.0.0-1555.g1c920afa.el9 ceph-noarch 264 k 2024-02-18T11:28:38.617 INFO:teuthology.orchestra.run.smithi012.stdout: cephadm noarch 2:19.0.0-1555.g1c920afa.el9 ceph-noarch 748 k 2024-02-18T11:28:38.617 INFO:teuthology.orchestra.run.smithi012.stdout: cephfs-mirror x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 220 k 2024-02-18T11:28:38.617 INFO:teuthology.orchestra.run.smithi012.stdout: cephfs-shell x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 27 k 2024-02-18T11:28:38.617 INFO:teuthology.orchestra.run.smithi012.stdout: cephfs-top noarch 2:19.0.0-1555.g1c920afa.el9 ceph-noarch 18 k 2024-02-18T11:28:38.618 INFO:teuthology.orchestra.run.smithi012.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 CentOS-AppStream 20 k 2024-02-18T11:28:38.618 INFO:teuthology.orchestra.run.smithi012.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-02-18T11:28:38.618 INFO:teuthology.orchestra.run.smithi012.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-02-18T11:28:38.618 INFO:teuthology.orchestra.run.smithi012.stdout: libcephfs-devel x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 31 k 2024-02-18T11:28:38.618 INFO:teuthology.orchestra.run.smithi012.stdout: libcephfs2 x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 742 k 2024-02-18T11:28:38.618 INFO:teuthology.orchestra.run.smithi012.stdout: librados-devel x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 127 k 2024-02-18T11:28:38.618 INFO:teuthology.orchestra.run.smithi012.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-02-18T11:28:38.618 INFO:teuthology.orchestra.run.smithi012.stdout: libuuid-devel x86_64 2.37.4-18.el9 CentOS-AppStream 24 k 2024-02-18T11:28:38.618 INFO:teuthology.orchestra.run.smithi012.stdout: openssl-devel x86_64 1:3.0.7-25.el9 CentOS-AppStream 3.0 M 2024-02-18T11:28:38.618 INFO:teuthology.orchestra.run.smithi012.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-02-18T11:28:38.618 INFO:teuthology.orchestra.run.smithi012.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-02-18T11:28:38.618 INFO:teuthology.orchestra.run.smithi012.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-02-18T11:28:38.618 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cephfs x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 161 k 2024-02-18T11:28:38.619 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rados x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 320 k 2024-02-18T11:28:38.619 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rbd x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 298 k 2024-02-18T11:28:38.619 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rgw x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 99 k 2024-02-18T11:28:38.619 INFO:teuthology.orchestra.run.smithi012.stdout: rbd-fuse x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 85 k 2024-02-18T11:28:38.619 INFO:teuthology.orchestra.run.smithi012.stdout: rbd-mirror x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 3.1 M 2024-02-18T11:28:38.619 INFO:teuthology.orchestra.run.smithi012.stdout: rbd-nbd x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 171 k 2024-02-18T11:28:38.619 INFO:teuthology.orchestra.run.smithi012.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 CentOS-AppStream 87 k 2024-02-18T11:28:38.619 INFO:teuthology.orchestra.run.smithi012.stdout:Upgrading: 2024-02-18T11:28:38.619 INFO:teuthology.orchestra.run.smithi012.stdout: NetworkManager x86_64 1:1.45.91-1.el9 CentOS-BaseOS 2.3 M 2024-02-18T11:28:38.619 INFO:teuthology.orchestra.run.smithi012.stdout: NetworkManager-libnm x86_64 1:1.45.91-1.el9 CentOS-BaseOS 1.8 M 2024-02-18T11:28:38.619 INFO:teuthology.orchestra.run.smithi012.stdout: NetworkManager-team x86_64 1:1.45.91-1.el9 CentOS-BaseOS 41 k 2024-02-18T11:28:38.619 INFO:teuthology.orchestra.run.smithi012.stdout: NetworkManager-tui x86_64 1:1.45.91-1.el9 CentOS-BaseOS 246 k 2024-02-18T11:28:38.619 INFO:teuthology.orchestra.run.smithi012.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2024-02-18T11:28:38.619 INFO:teuthology.orchestra.run.smithi012.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 CentOS-BaseOS 35 k 2024-02-18T11:28:38.619 INFO:teuthology.orchestra.run.smithi012.stdout: elfutils-libelf x86_64 0.190-2.el9 CentOS-BaseOS 191 k 2024-02-18T11:28:38.619 INFO:teuthology.orchestra.run.smithi012.stdout: elfutils-libs x86_64 0.190-2.el9 CentOS-BaseOS 254 k 2024-02-18T11:28:38.620 INFO:teuthology.orchestra.run.smithi012.stdout: iproute x86_64 6.7.0-2.el9 CentOS-BaseOS 802 k 2024-02-18T11:28:38.620 INFO:teuthology.orchestra.run.smithi012.stdout: iproute-tc x86_64 6.7.0-2.el9 CentOS-BaseOS 433 k 2024-02-18T11:28:38.620 INFO:teuthology.orchestra.run.smithi012.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2024-02-18T11:28:38.620 INFO:teuthology.orchestra.run.smithi012.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2024-02-18T11:28:38.620 INFO:teuthology.orchestra.run.smithi012.stdout: libblkid x86_64 2.37.4-18.el9 CentOS-BaseOS 107 k 2024-02-18T11:28:38.620 INFO:teuthology.orchestra.run.smithi012.stdout: libfdisk x86_64 2.37.4-18.el9 CentOS-BaseOS 154 k 2024-02-18T11:28:38.620 INFO:teuthology.orchestra.run.smithi012.stdout: libmount x86_64 2.37.4-18.el9 CentOS-BaseOS 135 k 2024-02-18T11:28:38.620 INFO:teuthology.orchestra.run.smithi012.stdout: librados2 x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 3.4 M 2024-02-18T11:28:38.620 INFO:teuthology.orchestra.run.smithi012.stdout: librbd1 x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 3.1 M 2024-02-18T11:28:38.620 INFO:teuthology.orchestra.run.smithi012.stdout: libsmartcols x86_64 2.37.4-18.el9 CentOS-BaseOS 63 k 2024-02-18T11:28:38.620 INFO:teuthology.orchestra.run.smithi012.stdout: libuuid x86_64 2.37.4-18.el9 CentOS-BaseOS 28 k 2024-02-18T11:28:38.620 INFO:teuthology.orchestra.run.smithi012.stdout: openssl x86_64 1:3.0.7-25.el9 CentOS-BaseOS 1.2 M 2024-02-18T11:28:38.620 INFO:teuthology.orchestra.run.smithi012.stdout: openssl-libs x86_64 1:3.0.7-25.el9 CentOS-BaseOS 2.1 M 2024-02-18T11:28:38.621 INFO:teuthology.orchestra.run.smithi012.stdout: util-linux x86_64 2.37.4-18.el9 CentOS-BaseOS 2.2 M 2024-02-18T11:28:38.621 INFO:teuthology.orchestra.run.smithi012.stdout: util-linux-core x86_64 2.37.4-18.el9 CentOS-BaseOS 438 k 2024-02-18T11:28:38.621 INFO:teuthology.orchestra.run.smithi012.stdout: util-linux-user x86_64 2.37.4-18.el9 CentOS-BaseOS 30 k 2024-02-18T11:28:38.621 INFO:teuthology.orchestra.run.smithi012.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-02-18T11:28:38.621 INFO:teuthology.orchestra.run.smithi012.stdout: xfsprogs x86_64 6.3.0-1.el9 CentOS-BaseOS 1.1 M 2024-02-18T11:28:38.621 INFO:teuthology.orchestra.run.smithi012.stdout:Installing dependencies: 2024-02-18T11:28:38.621 INFO:teuthology.orchestra.run.smithi012.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-02-18T11:28:38.621 INFO:teuthology.orchestra.run.smithi012.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-02-18T11:28:38.621 INFO:teuthology.orchestra.run.smithi012.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-02-18T11:28:38.621 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-common x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 20 M 2024-02-18T11:28:38.621 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1555.g1c920afa.el9 ceph-noarch 31 k 2024-02-18T11:28:38.621 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mds x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 2.1 M 2024-02-18T11:28:38.621 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1555.g1c920afa.el9 ceph-noarch 248 k 2024-02-18T11:28:38.621 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mon x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 4.7 M 2024-02-18T11:28:38.621 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-osd x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 17 M 2024-02-18T11:28:38.621 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1555.g1c920afa.el9 ceph-noarch 15 k 2024-02-18T11:28:38.622 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-selinux x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 25 k 2024-02-18T11:28:38.622 INFO:teuthology.orchestra.run.smithi012.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-02-18T11:28:38.622 INFO:teuthology.orchestra.run.smithi012.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-02-18T11:28:38.622 INFO:teuthology.orchestra.run.smithi012.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-02-18T11:28:38.622 INFO:teuthology.orchestra.run.smithi012.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-02-18T11:28:38.622 INFO:teuthology.orchestra.run.smithi012.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-02-18T11:28:38.622 INFO:teuthology.orchestra.run.smithi012.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-02-18T11:28:38.622 INFO:teuthology.orchestra.run.smithi012.stdout: jq x86_64 1.6-16.el9 CentOS-BaseOS 186 k 2024-02-18T11:28:38.622 INFO:teuthology.orchestra.run.smithi012.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-02-18T11:28:38.622 INFO:teuthology.orchestra.run.smithi012.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-02-18T11:28:38.622 INFO:teuthology.orchestra.run.smithi012.stdout: libcephsqlite x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 163 k 2024-02-18T11:28:38.622 INFO:teuthology.orchestra.run.smithi012.stdout: libgfortran x86_64 11.4.1-3.el9 CentOS-BaseOS 802 k 2024-02-18T11:28:38.622 INFO:teuthology.orchestra.run.smithi012.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-02-18T11:28:38.622 INFO:teuthology.orchestra.run.smithi012.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-02-18T11:28:38.623 INFO:teuthology.orchestra.run.smithi012.stdout: libquadmath x86_64 11.4.1-3.el9 CentOS-BaseOS 194 k 2024-02-18T11:28:38.623 INFO:teuthology.orchestra.run.smithi012.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-02-18T11:28:38.623 INFO:teuthology.orchestra.run.smithi012.stdout: libradosstriper1 x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 499 k 2024-02-18T11:28:38.623 INFO:teuthology.orchestra.run.smithi012.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-02-18T11:28:38.623 INFO:teuthology.orchestra.run.smithi012.stdout: librgw2 x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 5.1 M 2024-02-18T11:28:38.623 INFO:teuthology.orchestra.run.smithi012.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-02-18T11:28:38.623 INFO:teuthology.orchestra.run.smithi012.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-02-18T11:28:38.623 INFO:teuthology.orchestra.run.smithi012.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-02-18T11:28:38.623 INFO:teuthology.orchestra.run.smithi012.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-02-18T11:28:38.623 INFO:teuthology.orchestra.run.smithi012.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-02-18T11:28:38.623 INFO:teuthology.orchestra.run.smithi012.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-02-18T11:28:38.623 INFO:teuthology.orchestra.run.smithi012.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-BaseOS 217 k 2024-02-18T11:28:38.623 INFO:teuthology.orchestra.run.smithi012.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-02-18T11:28:38.623 INFO:teuthology.orchestra.run.smithi012.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-02-18T11:28:38.623 INFO:teuthology.orchestra.run.smithi012.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-02-18T11:28:38.624 INFO:teuthology.orchestra.run.smithi012.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-02-18T11:28:38.624 INFO:teuthology.orchestra.run.smithi012.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-02-18T11:28:38.624 INFO:teuthology.orchestra.run.smithi012.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-02-18T11:28:38.624 INFO:teuthology.orchestra.run.smithi012.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-02-18T11:28:38.624 INFO:teuthology.orchestra.run.smithi012.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-02-18T11:28:38.624 INFO:teuthology.orchestra.run.smithi012.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-02-18T11:28:38.624 INFO:teuthology.orchestra.run.smithi012.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-02-18T11:28:38.624 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-02-18T11:28:38.624 INFO:teuthology.orchestra.run.smithi012.stdout: python3-ceph-argparse x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 45 k 2024-02-18T11:28:38.624 INFO:teuthology.orchestra.run.smithi012.stdout: python3-ceph-common x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 131 k 2024-02-18T11:28:38.624 INFO:teuthology.orchestra.run.smithi012.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-02-18T11:28:38.624 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-02-18T11:28:38.624 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-02-18T11:28:38.624 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-02-18T11:28:38.625 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-02-18T11:28:38.625 INFO:teuthology.orchestra.run.smithi012.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-02-18T11:28:38.625 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-02-18T11:28:38.625 INFO:teuthology.orchestra.run.smithi012.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-02-18T11:28:38.625 INFO:teuthology.orchestra.run.smithi012.stdout: python3-google-auth noarch 1:2.27.0-1.el9 epel 203 k 2024-02-18T11:28:38.625 INFO:teuthology.orchestra.run.smithi012.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-02-18T11:28:38.625 INFO:teuthology.orchestra.run.smithi012.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-02-18T11:28:38.625 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-02-18T11:28:38.625 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-02-18T11:28:38.625 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-02-18T11:28:38.625 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-02-18T11:28:38.625 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-02-18T11:28:38.625 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jinja2 noarch 2.11.3-5.el9 CentOS-AppStream 227 k 2024-02-18T11:28:38.625 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-02-18T11:28:38.626 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-02-18T11:28:38.626 INFO:teuthology.orchestra.run.smithi012.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-02-18T11:28:38.626 INFO:teuthology.orchestra.run.smithi012.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-02-18T11:28:38.626 INFO:teuthology.orchestra.run.smithi012.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-02-18T11:28:38.626 INFO:teuthology.orchestra.run.smithi012.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-02-18T11:28:38.626 INFO:teuthology.orchestra.run.smithi012.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-02-18T11:28:38.626 INFO:teuthology.orchestra.run.smithi012.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-02-18T11:28:38.626 INFO:teuthology.orchestra.run.smithi012.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-02-18T11:28:38.626 INFO:teuthology.orchestra.run.smithi012.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-02-18T11:28:38.626 INFO:teuthology.orchestra.run.smithi012.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-02-18T11:28:38.626 INFO:teuthology.orchestra.run.smithi012.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-02-18T11:28:38.626 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-02-18T11:28:38.626 INFO:teuthology.orchestra.run.smithi012.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-02-18T11:28:38.626 INFO:teuthology.orchestra.run.smithi012.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-02-18T11:28:38.627 INFO:teuthology.orchestra.run.smithi012.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-02-18T11:28:38.627 INFO:teuthology.orchestra.run.smithi012.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-02-18T11:28:38.627 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-02-18T11:28:38.627 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-02-18T11:28:38.627 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-02-18T11:28:38.627 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-02-18T11:28:38.627 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-02-18T11:28:38.627 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-02-18T11:28:38.627 INFO:teuthology.orchestra.run.smithi012.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-02-18T11:28:38.627 INFO:teuthology.orchestra.run.smithi012.stdout: python3-requests noarch 2.25.1-8.el9 CentOS-BaseOS 113 k 2024-02-18T11:28:38.627 INFO:teuthology.orchestra.run.smithi012.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-02-18T11:28:38.627 INFO:teuthology.orchestra.run.smithi012.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-02-18T11:28:38.627 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-02-18T11:28:38.627 INFO:teuthology.orchestra.run.smithi012.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-02-18T11:28:38.627 INFO:teuthology.orchestra.run.smithi012.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-02-18T11:28:38.628 INFO:teuthology.orchestra.run.smithi012.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-02-18T11:28:38.628 INFO:teuthology.orchestra.run.smithi012.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-02-18T11:28:38.628 INFO:teuthology.orchestra.run.smithi012.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-02-18T11:28:38.628 INFO:teuthology.orchestra.run.smithi012.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-02-18T11:28:38.628 INFO:teuthology.orchestra.run.smithi012.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-02-18T11:28:38.628 INFO:teuthology.orchestra.run.smithi012.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-02-18T11:28:38.628 INFO:teuthology.orchestra.run.smithi012.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-02-18T11:28:38.628 INFO:teuthology.orchestra.run.smithi012.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-02-18T11:28:38.628 INFO:teuthology.orchestra.run.smithi012.stdout: qatlib x86_64 23.11.0-1.el9 CentOS-AppStream 215 k 2024-02-18T11:28:38.628 INFO:teuthology.orchestra.run.smithi012.stdout: qatzip-libs x86_64 1.1.2-2.el9 CentOS-AppStream 43 k 2024-02-18T11:28:38.628 INFO:teuthology.orchestra.run.smithi012.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-02-18T11:28:38.628 INFO:teuthology.orchestra.run.smithi012.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-02-18T11:28:38.628 INFO:teuthology.orchestra.run.smithi012.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-02-18T11:28:38.628 INFO:teuthology.orchestra.run.smithi012.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-02-18T11:28:38.629 INFO:teuthology.orchestra.run.smithi012.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-02-18T11:28:38.629 INFO:teuthology.orchestra.run.smithi012.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-02-18T11:28:38.629 INFO:teuthology.orchestra.run.smithi012.stdout:Installing weak dependencies: 2024-02-18T11:28:38.629 INFO:teuthology.orchestra.run.smithi012.stdout: qatlib-service x86_64 23.11.0-1.el9 CentOS-AppStream 34 k 2024-02-18T11:28:38.629 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:28:38.629 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction Summary 2024-02-18T11:28:38.629 INFO:teuthology.orchestra.run.smithi012.stdout:============================================================================================ 2024-02-18T11:28:38.629 INFO:teuthology.orchestra.run.smithi012.stdout:Install 150 Packages 2024-02-18T11:28:38.629 INFO:teuthology.orchestra.run.smithi012.stdout:Upgrade 26 Packages 2024-02-18T11:28:38.629 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:28:38.630 INFO:teuthology.orchestra.run.smithi012.stdout:Total download size: 223 M 2024-02-18T11:28:38.630 INFO:teuthology.orchestra.run.smithi012.stdout:Downloading Packages: 2024-02-18T11:28:38.651 INFO:teuthology.orchestra.run.smithi158.stdout:(8/176): ceph-common-19.0.0-1555.g1c920afa.el9. 7.7 MB/s | 20 MB 00:02 2024-02-18T11:28:38.734 INFO:teuthology.orchestra.run.smithi158.stdout:(9/176): ceph-selinux-19.0.0-1555.g1c920afa.el9 304 kB/s | 25 kB 00:00 2024-02-18T11:28:38.885 INFO:teuthology.orchestra.run.smithi158.stdout:(10/176): ceph-radosgw-19.0.0-1555.g1c920afa.el 8.3 MB/s | 9.8 MB 00:01 2024-02-18T11:28:38.985 INFO:teuthology.orchestra.run.smithi158.stdout:(11/176): cephfs-mirror-19.0.0-1555.g1c920afa.e 2.2 MB/s | 220 kB 00:00 2024-02-18T11:28:39.202 INFO:teuthology.orchestra.run.smithi158.stdout:(12/176): ceph-osd-19.0.0-1555.g1c920afa.el9.x8 7.9 MB/s | 17 MB 00:02 2024-02-18T11:28:39.208 INFO:teuthology.orchestra.run.smithi188.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-02-18T11:28:39.209 INFO:teuthology.orchestra.run.smithi188.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-02-18T11:28:39.223 INFO:teuthology.orchestra.run.smithi188.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-02-18T11:28:39.224 INFO:teuthology.orchestra.run.smithi188.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-02-18T11:28:39.225 INFO:teuthology.orchestra.run.smithi188.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-02-18T11:28:39.226 INFO:teuthology.orchestra.run.smithi188.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-02-18T11:28:39.227 INFO:teuthology.orchestra.run.smithi158.stdout:(13/176): cephfs-shell-19.0.0-1555.g1c920afa.el 112 kB/s | 27 kB 00:00 2024-02-18T11:28:39.228 INFO:teuthology.orchestra.run.smithi188.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-02-18T11:28:39.231 INFO:teuthology.orchestra.run.smithi188.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-02-18T11:28:39.232 INFO:teuthology.orchestra.run.smithi188.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-02-18T11:28:39.270 INFO:teuthology.orchestra.run.smithi158.stdout:(14/176): libcephfs-devel-19.0.0-1555.g1c920afa 465 kB/s | 31 kB 00:00 2024-02-18T11:28:39.312 INFO:teuthology.orchestra.run.smithi188.stdout:Dependencies resolved. 2024-02-18T11:28:39.324 INFO:teuthology.orchestra.run.smithi188.stdout:============================================================================================ 2024-02-18T11:28:39.325 INFO:teuthology.orchestra.run.smithi188.stdout: Package Arch Version Repository Size 2024-02-18T11:28:39.325 INFO:teuthology.orchestra.run.smithi188.stdout:============================================================================================ 2024-02-18T11:28:39.325 INFO:teuthology.orchestra.run.smithi188.stdout:Installing: 2024-02-18T11:28:39.325 INFO:teuthology.orchestra.run.smithi188.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-02-18T11:28:39.325 INFO:teuthology.orchestra.run.smithi188.stdout: ceph x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 6.6 k 2024-02-18T11:28:39.325 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-base x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 5.3 M 2024-02-18T11:28:39.325 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-fuse x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 886 k 2024-02-18T11:28:39.325 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 145 k 2024-02-18T11:28:39.325 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 1.5 M 2024-02-18T11:28:39.325 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1555.g1c920afa.el9 ceph-noarch 138 k 2024-02-18T11:28:39.325 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1555.g1c920afa.el9 ceph-noarch 1.8 M 2024-02-18T11:28:39.326 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1555.g1c920afa.el9 ceph-noarch 7.4 M 2024-02-18T11:28:39.326 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-rook noarch 2:19.0.0-1555.g1c920afa.el9 ceph-noarch 52 k 2024-02-18T11:28:39.326 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-radosgw x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 9.8 M 2024-02-18T11:28:39.326 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-test x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 47 M 2024-02-18T11:28:39.326 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-volume noarch 2:19.0.0-1555.g1c920afa.el9 ceph-noarch 264 k 2024-02-18T11:28:39.326 INFO:teuthology.orchestra.run.smithi188.stdout: cephadm noarch 2:19.0.0-1555.g1c920afa.el9 ceph-noarch 748 k 2024-02-18T11:28:39.326 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-mirror x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 220 k 2024-02-18T11:28:39.326 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-shell x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 27 k 2024-02-18T11:28:39.326 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-top noarch 2:19.0.0-1555.g1c920afa.el9 ceph-noarch 18 k 2024-02-18T11:28:39.326 INFO:teuthology.orchestra.run.smithi188.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 CentOS-AppStream 20 k 2024-02-18T11:28:39.326 INFO:teuthology.orchestra.run.smithi188.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-02-18T11:28:39.326 INFO:teuthology.orchestra.run.smithi188.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-02-18T11:28:39.326 INFO:teuthology.orchestra.run.smithi188.stdout: libcephfs-devel x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 31 k 2024-02-18T11:28:39.327 INFO:teuthology.orchestra.run.smithi188.stdout: libcephfs2 x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 742 k 2024-02-18T11:28:39.327 INFO:teuthology.orchestra.run.smithi188.stdout: librados-devel x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 127 k 2024-02-18T11:28:39.327 INFO:teuthology.orchestra.run.smithi188.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-02-18T11:28:39.327 INFO:teuthology.orchestra.run.smithi188.stdout: libuuid-devel x86_64 2.37.4-18.el9 CentOS-AppStream 24 k 2024-02-18T11:28:39.327 INFO:teuthology.orchestra.run.smithi188.stdout: openssl-devel x86_64 1:3.0.7-25.el9 CentOS-AppStream 3.0 M 2024-02-18T11:28:39.327 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-02-18T11:28:39.327 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-02-18T11:28:39.327 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-02-18T11:28:39.327 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cephfs x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 161 k 2024-02-18T11:28:39.327 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rados x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 320 k 2024-02-18T11:28:39.327 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rbd x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 298 k 2024-02-18T11:28:39.327 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rgw x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 99 k 2024-02-18T11:28:39.327 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-fuse x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 85 k 2024-02-18T11:28:39.327 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-mirror x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 3.1 M 2024-02-18T11:28:39.328 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-nbd x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 171 k 2024-02-18T11:28:39.328 INFO:teuthology.orchestra.run.smithi188.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 CentOS-AppStream 87 k 2024-02-18T11:28:39.328 INFO:teuthology.orchestra.run.smithi188.stdout:Upgrading: 2024-02-18T11:28:39.328 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager x86_64 1:1.45.91-1.el9 CentOS-BaseOS 2.3 M 2024-02-18T11:28:39.328 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-libnm x86_64 1:1.45.91-1.el9 CentOS-BaseOS 1.8 M 2024-02-18T11:28:39.328 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-team x86_64 1:1.45.91-1.el9 CentOS-BaseOS 41 k 2024-02-18T11:28:39.328 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-tui x86_64 1:1.45.91-1.el9 CentOS-BaseOS 246 k 2024-02-18T11:28:39.328 INFO:teuthology.orchestra.run.smithi188.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2024-02-18T11:28:39.328 INFO:teuthology.orchestra.run.smithi188.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 CentOS-BaseOS 35 k 2024-02-18T11:28:39.328 INFO:teuthology.orchestra.run.smithi188.stdout: elfutils-libelf x86_64 0.190-2.el9 CentOS-BaseOS 191 k 2024-02-18T11:28:39.328 INFO:teuthology.orchestra.run.smithi188.stdout: elfutils-libs x86_64 0.190-2.el9 CentOS-BaseOS 254 k 2024-02-18T11:28:39.328 INFO:teuthology.orchestra.run.smithi188.stdout: iproute x86_64 6.7.0-2.el9 CentOS-BaseOS 802 k 2024-02-18T11:28:39.328 INFO:teuthology.orchestra.run.smithi188.stdout: iproute-tc x86_64 6.7.0-2.el9 CentOS-BaseOS 433 k 2024-02-18T11:28:39.329 INFO:teuthology.orchestra.run.smithi188.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2024-02-18T11:28:39.329 INFO:teuthology.orchestra.run.smithi188.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2024-02-18T11:28:39.329 INFO:teuthology.orchestra.run.smithi188.stdout: libblkid x86_64 2.37.4-18.el9 CentOS-BaseOS 107 k 2024-02-18T11:28:39.329 INFO:teuthology.orchestra.run.smithi188.stdout: libfdisk x86_64 2.37.4-18.el9 CentOS-BaseOS 154 k 2024-02-18T11:28:39.329 INFO:teuthology.orchestra.run.smithi188.stdout: libmount x86_64 2.37.4-18.el9 CentOS-BaseOS 135 k 2024-02-18T11:28:39.329 INFO:teuthology.orchestra.run.smithi188.stdout: librados2 x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 3.4 M 2024-02-18T11:28:39.329 INFO:teuthology.orchestra.run.smithi188.stdout: librbd1 x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 3.1 M 2024-02-18T11:28:39.329 INFO:teuthology.orchestra.run.smithi188.stdout: libsmartcols x86_64 2.37.4-18.el9 CentOS-BaseOS 63 k 2024-02-18T11:28:39.329 INFO:teuthology.orchestra.run.smithi188.stdout: libuuid x86_64 2.37.4-18.el9 CentOS-BaseOS 28 k 2024-02-18T11:28:39.329 INFO:teuthology.orchestra.run.smithi188.stdout: openssl x86_64 1:3.0.7-25.el9 CentOS-BaseOS 1.2 M 2024-02-18T11:28:39.329 INFO:teuthology.orchestra.run.smithi188.stdout: openssl-libs x86_64 1:3.0.7-25.el9 CentOS-BaseOS 2.1 M 2024-02-18T11:28:39.329 INFO:teuthology.orchestra.run.smithi188.stdout: util-linux x86_64 2.37.4-18.el9 CentOS-BaseOS 2.2 M 2024-02-18T11:28:39.329 INFO:teuthology.orchestra.run.smithi188.stdout: util-linux-core x86_64 2.37.4-18.el9 CentOS-BaseOS 438 k 2024-02-18T11:28:39.330 INFO:teuthology.orchestra.run.smithi188.stdout: util-linux-user x86_64 2.37.4-18.el9 CentOS-BaseOS 30 k 2024-02-18T11:28:39.330 INFO:teuthology.orchestra.run.smithi188.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-02-18T11:28:39.330 INFO:teuthology.orchestra.run.smithi188.stdout: xfsprogs x86_64 6.3.0-1.el9 CentOS-BaseOS 1.1 M 2024-02-18T11:28:39.330 INFO:teuthology.orchestra.run.smithi188.stdout:Installing dependencies: 2024-02-18T11:28:39.330 INFO:teuthology.orchestra.run.smithi188.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-02-18T11:28:39.330 INFO:teuthology.orchestra.run.smithi188.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-02-18T11:28:39.330 INFO:teuthology.orchestra.run.smithi188.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-02-18T11:28:39.330 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-common x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 20 M 2024-02-18T11:28:39.330 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1555.g1c920afa.el9 ceph-noarch 31 k 2024-02-18T11:28:39.330 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mds x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 2.1 M 2024-02-18T11:28:39.330 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1555.g1c920afa.el9 ceph-noarch 248 k 2024-02-18T11:28:39.330 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mon x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 4.7 M 2024-02-18T11:28:39.330 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-osd x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 17 M 2024-02-18T11:28:39.330 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1555.g1c920afa.el9 ceph-noarch 15 k 2024-02-18T11:28:39.330 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-selinux x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 25 k 2024-02-18T11:28:39.331 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-02-18T11:28:39.331 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-02-18T11:28:39.331 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-02-18T11:28:39.331 INFO:teuthology.orchestra.run.smithi188.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-02-18T11:28:39.331 INFO:teuthology.orchestra.run.smithi188.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-02-18T11:28:39.331 INFO:teuthology.orchestra.run.smithi188.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-02-18T11:28:39.331 INFO:teuthology.orchestra.run.smithi188.stdout: jq x86_64 1.6-16.el9 CentOS-BaseOS 186 k 2024-02-18T11:28:39.331 INFO:teuthology.orchestra.run.smithi188.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-02-18T11:28:39.331 INFO:teuthology.orchestra.run.smithi188.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-02-18T11:28:39.331 INFO:teuthology.orchestra.run.smithi188.stdout: libcephsqlite x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 163 k 2024-02-18T11:28:39.331 INFO:teuthology.orchestra.run.smithi188.stdout: libgfortran x86_64 11.4.1-3.el9 CentOS-BaseOS 802 k 2024-02-18T11:28:39.331 INFO:teuthology.orchestra.run.smithi188.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-02-18T11:28:39.331 INFO:teuthology.orchestra.run.smithi188.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-02-18T11:28:39.332 INFO:teuthology.orchestra.run.smithi188.stdout: libquadmath x86_64 11.4.1-3.el9 CentOS-BaseOS 194 k 2024-02-18T11:28:39.332 INFO:teuthology.orchestra.run.smithi188.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-02-18T11:28:39.332 INFO:teuthology.orchestra.run.smithi188.stdout: libradosstriper1 x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 499 k 2024-02-18T11:28:39.332 INFO:teuthology.orchestra.run.smithi188.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-02-18T11:28:39.332 INFO:teuthology.orchestra.run.smithi188.stdout: librgw2 x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 5.1 M 2024-02-18T11:28:39.332 INFO:teuthology.orchestra.run.smithi188.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-02-18T11:28:39.332 INFO:teuthology.orchestra.run.smithi188.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-02-18T11:28:39.332 INFO:teuthology.orchestra.run.smithi188.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-02-18T11:28:39.332 INFO:teuthology.orchestra.run.smithi188.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-02-18T11:28:39.332 INFO:teuthology.orchestra.run.smithi188.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-02-18T11:28:39.332 INFO:teuthology.orchestra.run.smithi188.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-02-18T11:28:39.332 INFO:teuthology.orchestra.run.smithi188.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-BaseOS 217 k 2024-02-18T11:28:39.332 INFO:teuthology.orchestra.run.smithi188.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-02-18T11:28:39.333 INFO:teuthology.orchestra.run.smithi188.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-02-18T11:28:39.333 INFO:teuthology.orchestra.run.smithi188.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-02-18T11:28:39.333 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-02-18T11:28:39.333 INFO:teuthology.orchestra.run.smithi188.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-02-18T11:28:39.333 INFO:teuthology.orchestra.run.smithi188.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-02-18T11:28:39.333 INFO:teuthology.orchestra.run.smithi188.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-02-18T11:28:39.333 INFO:teuthology.orchestra.run.smithi188.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-02-18T11:28:39.333 INFO:teuthology.orchestra.run.smithi188.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-02-18T11:28:39.333 INFO:teuthology.orchestra.run.smithi188.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-02-18T11:28:39.333 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-02-18T11:28:39.333 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ceph-argparse x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 45 k 2024-02-18T11:28:39.333 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ceph-common x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 131 k 2024-02-18T11:28:39.333 INFO:teuthology.orchestra.run.smithi188.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-02-18T11:28:39.333 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-02-18T11:28:39.333 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-02-18T11:28:39.334 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-02-18T11:28:39.334 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-02-18T11:28:39.334 INFO:teuthology.orchestra.run.smithi188.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-02-18T11:28:39.334 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-02-18T11:28:39.334 INFO:teuthology.orchestra.run.smithi188.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-02-18T11:28:39.334 INFO:teuthology.orchestra.run.smithi188.stdout: python3-google-auth noarch 1:2.27.0-1.el9 epel 203 k 2024-02-18T11:28:39.334 INFO:teuthology.orchestra.run.smithi188.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-02-18T11:28:39.334 INFO:teuthology.orchestra.run.smithi188.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-02-18T11:28:39.334 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-02-18T11:28:39.335 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-02-18T11:28:39.335 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-02-18T11:28:39.335 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-02-18T11:28:39.335 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-02-18T11:28:39.335 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jinja2 noarch 2.11.3-5.el9 CentOS-AppStream 227 k 2024-02-18T11:28:39.335 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-02-18T11:28:39.335 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-02-18T11:28:39.335 INFO:teuthology.orchestra.run.smithi188.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-02-18T11:28:39.335 INFO:teuthology.orchestra.run.smithi188.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-02-18T11:28:39.335 INFO:teuthology.orchestra.run.smithi188.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-02-18T11:28:39.335 INFO:teuthology.orchestra.run.smithi188.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-02-18T11:28:39.335 INFO:teuthology.orchestra.run.smithi188.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-02-18T11:28:39.335 INFO:teuthology.orchestra.run.smithi188.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-02-18T11:28:39.335 INFO:teuthology.orchestra.run.smithi188.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-02-18T11:28:39.335 INFO:teuthology.orchestra.run.smithi188.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-02-18T11:28:39.336 INFO:teuthology.orchestra.run.smithi188.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-02-18T11:28:39.336 INFO:teuthology.orchestra.run.smithi188.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-02-18T11:28:39.336 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-02-18T11:28:39.336 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-02-18T11:28:39.336 INFO:teuthology.orchestra.run.smithi188.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-02-18T11:28:39.336 INFO:teuthology.orchestra.run.smithi188.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-02-18T11:28:39.336 INFO:teuthology.orchestra.run.smithi188.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-02-18T11:28:39.336 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-02-18T11:28:39.336 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-02-18T11:28:39.336 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-02-18T11:28:39.336 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-02-18T11:28:39.336 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-02-18T11:28:39.336 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-02-18T11:28:39.336 INFO:teuthology.orchestra.run.smithi188.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-02-18T11:28:39.337 INFO:teuthology.orchestra.run.smithi188.stdout: python3-requests noarch 2.25.1-8.el9 CentOS-BaseOS 113 k 2024-02-18T11:28:39.337 INFO:teuthology.orchestra.run.smithi188.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-02-18T11:28:39.337 INFO:teuthology.orchestra.run.smithi188.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-02-18T11:28:39.337 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-02-18T11:28:39.337 INFO:teuthology.orchestra.run.smithi188.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-02-18T11:28:39.337 INFO:teuthology.orchestra.run.smithi188.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-02-18T11:28:39.337 INFO:teuthology.orchestra.run.smithi188.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-02-18T11:28:39.337 INFO:teuthology.orchestra.run.smithi188.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-02-18T11:28:39.337 INFO:teuthology.orchestra.run.smithi188.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-02-18T11:28:39.337 INFO:teuthology.orchestra.run.smithi188.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-02-18T11:28:39.337 INFO:teuthology.orchestra.run.smithi188.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-02-18T11:28:39.337 INFO:teuthology.orchestra.run.smithi188.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-02-18T11:28:39.337 INFO:teuthology.orchestra.run.smithi188.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-02-18T11:28:39.337 INFO:teuthology.orchestra.run.smithi188.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-02-18T11:28:39.337 INFO:teuthology.orchestra.run.smithi188.stdout: qatlib x86_64 23.11.0-1.el9 CentOS-AppStream 215 k 2024-02-18T11:28:39.338 INFO:teuthology.orchestra.run.smithi188.stdout: qatzip-libs x86_64 1.1.2-2.el9 CentOS-AppStream 43 k 2024-02-18T11:28:39.338 INFO:teuthology.orchestra.run.smithi188.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-02-18T11:28:39.338 INFO:teuthology.orchestra.run.smithi188.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-02-18T11:28:39.338 INFO:teuthology.orchestra.run.smithi188.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-02-18T11:28:39.338 INFO:teuthology.orchestra.run.smithi188.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-02-18T11:28:39.338 INFO:teuthology.orchestra.run.smithi188.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-02-18T11:28:39.338 INFO:teuthology.orchestra.run.smithi188.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-02-18T11:28:39.338 INFO:teuthology.orchestra.run.smithi188.stdout:Installing weak dependencies: 2024-02-18T11:28:39.338 INFO:teuthology.orchestra.run.smithi188.stdout: qatlib-service x86_64 23.11.0-1.el9 CentOS-AppStream 34 k 2024-02-18T11:28:39.338 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-02-18T11:28:39.338 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction Summary 2024-02-18T11:28:39.338 INFO:teuthology.orchestra.run.smithi188.stdout:============================================================================================ 2024-02-18T11:28:39.338 INFO:teuthology.orchestra.run.smithi188.stdout:Install 150 Packages 2024-02-18T11:28:39.338 INFO:teuthology.orchestra.run.smithi188.stdout:Upgrade 26 Packages 2024-02-18T11:28:39.339 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-02-18T11:28:39.339 INFO:teuthology.orchestra.run.smithi188.stdout:Total download size: 223 M 2024-02-18T11:28:39.339 INFO:teuthology.orchestra.run.smithi188.stdout:Downloading Packages: 2024-02-18T11:28:39.353 INFO:teuthology.orchestra.run.smithi158.stdout:(15/176): libcephfs2-19.0.0-1555.g1c920afa.el9. 5.8 MB/s | 742 kB 00:00 2024-02-18T11:28:39.378 INFO:teuthology.orchestra.run.smithi158.stdout:(16/176): libcephsqlite-19.0.0-1555.g1c920afa.e 1.5 MB/s | 163 kB 00:00 2024-02-18T11:28:39.429 INFO:teuthology.orchestra.run.smithi158.stdout:(17/176): librados-devel-19.0.0-1555.g1c920afa. 1.6 MB/s | 127 kB 00:00 2024-02-18T11:28:39.441 INFO:teuthology.orchestra.run.smithi012.stdout:(1/176): ceph-19.0.0-1555.g1c920afa.el9.x86_64. 29 kB/s | 6.6 kB 00:00 2024-02-18T11:28:39.471 INFO:teuthology.orchestra.run.smithi158.stdout:(18/176): libradosstriper1-19.0.0-1555.g1c920af 5.3 MB/s | 499 kB 00:00 2024-02-18T11:28:39.563 INFO:teuthology.orchestra.run.smithi158.stdout:(19/176): python3-ceph-argparse-19.0.0-1555.g1c 491 kB/s | 45 kB 00:00 2024-02-18T11:28:39.663 INFO:teuthology.orchestra.run.smithi158.stdout:(20/176): python3-ceph-common-19.0.0-1555.g1c92 1.3 MB/s | 131 kB 00:00 2024-02-18T11:28:39.781 INFO:teuthology.orchestra.run.smithi158.stdout:(21/176): python3-cephfs-19.0.0-1555.g1c920afa. 1.4 MB/s | 161 kB 00:00 2024-02-18T11:28:40.045 INFO:teuthology.orchestra.run.smithi188.stdout:(1/176): ceph-19.0.0-1555.g1c920afa.el9.x86_64. 18 kB/s | 6.6 kB 00:00 2024-02-18T11:28:40.279 INFO:teuthology.orchestra.run.smithi158.stdout:(22/176): librgw2-19.0.0-1555.g1c920afa.el9.x86 6.0 MB/s | 5.1 MB 00:00 2024-02-18T11:28:40.316 INFO:teuthology.orchestra.run.smithi012.stdout:(2/176): ceph-fuse-19.0.0-1555.g1c920afa.el9.x8 1.0 MB/s | 886 kB 00:00 2024-02-18T11:28:40.372 INFO:teuthology.orchestra.run.smithi158.stdout:(23/176): python3-rbd-19.0.0-1555.g1c920afa.el9 3.1 MB/s | 298 kB 00:00 2024-02-18T11:28:40.448 INFO:teuthology.orchestra.run.smithi158.stdout:(24/176): python3-rados-19.0.0-1555.g1c920afa.e 480 kB/s | 320 kB 00:00 2024-02-18T11:28:40.473 INFO:teuthology.orchestra.run.smithi158.stdout:(25/176): python3-rgw-19.0.0-1555.g1c920afa.el9 992 kB/s | 99 kB 00:00 2024-02-18T11:28:40.492 INFO:teuthology.orchestra.run.smithi012.stdout:(3/176): ceph-immutable-object-cache-19.0.0-155 830 kB/s | 145 kB 00:00 2024-02-18T11:28:40.548 INFO:teuthology.orchestra.run.smithi158.stdout:(26/176): rbd-fuse-19.0.0-1555.g1c920afa.el9.x8 852 kB/s | 85 kB 00:00 2024-02-18T11:28:40.937 INFO:teuthology.orchestra.run.smithi188.stdout:(2/176): ceph-fuse-19.0.0-1555.g1c920afa.el9.x8 994 kB/s | 886 kB 00:00 2024-02-18T11:28:41.112 INFO:teuthology.orchestra.run.smithi188.stdout:(3/176): ceph-immutable-object-cache-19.0.0-155 830 kB/s | 145 kB 00:00 2024-02-18T11:28:41.257 INFO:teuthology.orchestra.run.smithi158.stdout:(27/176): rbd-mirror-19.0.0-1555.g1c920afa.el9. 3.9 MB/s | 3.1 MB 00:00 2024-02-18T11:28:41.282 INFO:teuthology.orchestra.run.smithi158.stdout:(28/176): rbd-nbd-19.0.0-1555.g1c920afa.el9.x86 233 kB/s | 171 kB 00:00 2024-02-18T11:28:41.341 INFO:teuthology.orchestra.run.smithi158.stdout:(29/176): ceph-grafana-dashboards-19.0.0-1555.g 377 kB/s | 31 kB 00:00 2024-02-18T11:28:41.383 INFO:teuthology.orchestra.run.smithi158.stdout:(30/176): ceph-mgr-cephadm-19.0.0-1555.g1c920af 1.3 MB/s | 138 kB 00:00 2024-02-18T11:28:41.625 INFO:teuthology.orchestra.run.smithi012.stdout:(4/176): ceph-mds-19.0.0-1555.g1c920afa.el9.x86 1.9 MB/s | 2.1 MB 00:01 2024-02-18T11:28:41.987 INFO:teuthology.orchestra.run.smithi188.stdout:(4/176): ceph-base-19.0.0-1555.g1c920afa.el9.x8 2.3 MB/s | 5.3 MB 00:02 2024-02-18T11:28:42.033 INFO:teuthology.orchestra.run.smithi012.stdout:(5/176): ceph-mgr-19.0.0-1555.g1c920afa.el9.x86 3.6 MB/s | 1.5 MB 00:00 2024-02-18T11:28:42.292 INFO:teuthology.orchestra.run.smithi012.stdout:(6/176): ceph-base-19.0.0-1555.g1c920afa.el9.x8 1.7 MB/s | 5.3 MB 00:03 2024-02-18T11:28:42.325 INFO:teuthology.orchestra.run.smithi158.stdout:(31/176): ceph-test-19.0.0-1555.g1c920afa.el9.x 13 MB/s | 47 MB 00:03 2024-02-18T11:28:42.329 INFO:teuthology.orchestra.run.smithi188.stdout:(5/176): ceph-mds-19.0.0-1555.g1c920afa.el9.x86 1.7 MB/s | 2.1 MB 00:01 2024-02-18T11:28:42.366 INFO:teuthology.orchestra.run.smithi158.stdout:(32/176): ceph-mgr-dashboard-19.0.0-1555.g1c920 1.7 MB/s | 1.8 MB 00:01 2024-02-18T11:28:42.400 INFO:teuthology.orchestra.run.smithi158.stdout:(33/176): ceph-mgr-modules-core-19.0.0-1555.g1c 3.2 MB/s | 248 kB 00:00 2024-02-18T11:28:42.404 INFO:teuthology.orchestra.run.smithi188.stdout:(6/176): ceph-mgr-19.0.0-1555.g1c920afa.el9.x86 3.6 MB/s | 1.5 MB 00:00 2024-02-18T11:28:42.450 INFO:teuthology.orchestra.run.smithi158.stdout:(34/176): ceph-mgr-rook-19.0.0-1555.g1c920afa.e 625 kB/s | 52 kB 00:00 2024-02-18T11:28:42.476 INFO:teuthology.orchestra.run.smithi158.stdout:(35/176): ceph-prometheus-alerts-19.0.0-1555.g1 204 kB/s | 15 kB 00:00 2024-02-18T11:28:42.609 INFO:teuthology.orchestra.run.smithi158.stdout:(36/176): ceph-volume-19.0.0-1555.g1c920afa.el9 1.6 MB/s | 264 kB 00:00 2024-02-18T11:28:42.643 INFO:teuthology.orchestra.run.smithi158.stdout:(37/176): cephadm-19.0.0-1555.g1c920afa.el9.noa 4.4 MB/s | 748 kB 00:00 2024-02-18T11:28:42.718 INFO:teuthology.orchestra.run.smithi158.stdout:(38/176): cephfs-top-19.0.0-1555.g1c920afa.el9. 168 kB/s | 18 kB 00:00 2024-02-18T11:28:42.759 INFO:teuthology.orchestra.run.smithi012.stdout:(7/176): ceph-mon-19.0.0-1555.g1c920afa.el9.x86 6.5 MB/s | 4.7 MB 00:00 2024-02-18T11:28:42.919 INFO:teuthology.orchestra.run.smithi158.stdout:(39/176): ceph-mgr-diskprediction-local-19.0.0- 4.8 MB/s | 7.4 MB 00:01 2024-02-18T11:28:43.011 INFO:teuthology.orchestra.run.smithi158.stdout:(40/176): bison-3.7.4-5.el9.x86_64.rpm 3.1 MB/s | 920 kB 00:00 2024-02-18T11:28:43.036 INFO:teuthology.orchestra.run.smithi158.stdout:(41/176): automake-1.16.2-8.el9.noarch.rpm 1.6 MB/s | 662 kB 00:00 2024-02-18T11:28:43.061 INFO:teuthology.orchestra.run.smithi158.stdout:(42/176): boost-program-options-1.75.0-8.el9.x8 747 kB/s | 106 kB 00:00 2024-02-18T11:28:43.078 INFO:teuthology.orchestra.run.smithi158.stdout:(43/176): elfutils-libelf-devel-0.190-2.el9.x86 306 kB/s | 20 kB 00:00 2024-02-18T11:28:43.120 INFO:teuthology.orchestra.run.smithi158.stdout:(44/176): flexiblas-3.0.4-8.el9.x86_64.rpm 542 kB/s | 32 kB 00:00 2024-02-18T11:28:43.145 INFO:teuthology.orchestra.run.smithi158.stdout:(45/176): flex-2.6.4-9.el9.x86_64.rpm 3.6 MB/s | 307 kB 00:00 2024-02-18T11:28:43.246 INFO:teuthology.orchestra.run.smithi158.stdout:(46/176): flexiblas-netlib-3.0.4-8.el9.x86_64.r 18 MB/s | 3.0 MB 00:00 2024-02-18T11:28:43.263 INFO:teuthology.orchestra.run.smithi158.stdout:(47/176): flexiblas-openblas-openmp-3.0.4-8.el9 122 kB/s | 17 kB 00:00 2024-02-18T11:28:43.280 INFO:teuthology.orchestra.run.smithi158.stdout:(48/176): libaio-devel-0.3.111-13.el9.x86_64.rp 76 kB/s | 10 kB 00:00 2024-02-18T11:28:43.305 INFO:teuthology.orchestra.run.smithi158.stdout:(49/176): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 159 kB 00:00 2024-02-18T11:28:43.330 INFO:teuthology.orchestra.run.smithi158.stdout:(50/176): librabbitmq-0.11.0-7.el9.x86_64.rpm 883 kB/s | 44 kB 00:00 2024-02-18T11:28:43.364 INFO:teuthology.orchestra.run.smithi158.stdout:(51/176): librdkafka-1.6.1-102.el9.x86_64.rpm 7.7 MB/s | 661 kB 00:00 2024-02-18T11:28:43.389 INFO:teuthology.orchestra.run.smithi158.stdout:(52/176): libtool-2.4.6-45.el9.x86_64.rpm 6.7 MB/s | 578 kB 00:00 2024-02-18T11:28:43.406 INFO:teuthology.orchestra.run.smithi158.stdout:(53/176): libuuid-devel-2.37.4-18.el9.x86_64.rp 327 kB/s | 24 kB 00:00 2024-02-18T11:28:43.423 INFO:teuthology.orchestra.run.smithi158.stdout:(54/176): libzstd-devel-1.5.1-2.el9.x86_64.rpm 802 kB/s | 46 kB 00:00 2024-02-18T11:28:43.473 INFO:teuthology.orchestra.run.smithi158.stdout:(55/176): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.3 MB/s | 282 kB 00:00 2024-02-18T11:28:43.498 INFO:teuthology.orchestra.run.smithi158.stdout:(56/176): lua-5.4.4-4.el9.x86_64.rpm 2.0 MB/s | 187 kB 00:00 2024-02-18T11:28:43.515 INFO:teuthology.orchestra.run.smithi158.stdout:(57/176): openblas-0.3.21-2.el9.x86_64.rpm 373 kB/s | 34 kB 00:00 2024-02-18T11:28:43.576 INFO:teuthology.orchestra.run.smithi012.stdout:(8/176): ceph-radosgw-19.0.0-1555.g1c920afa.el9 12 MB/s | 9.8 MB 00:00 2024-02-18T11:28:43.643 INFO:teuthology.orchestra.run.smithi012.stdout:(9/176): ceph-selinux-19.0.0-1555.g1c920afa.el9 379 kB/s | 25 kB 00:00 2024-02-18T11:28:43.649 INFO:teuthology.orchestra.run.smithi158.stdout:(58/176): openblas-openmp-0.3.21-2.el9.x86_64.r 27 MB/s | 4.7 MB 00:00 2024-02-18T11:28:43.699 INFO:teuthology.orchestra.run.smithi158.stdout:(59/176): postgresql-13.11-1.el9.x86_64.rpm 8.4 MB/s | 1.5 MB 00:00 2024-02-18T11:28:43.750 INFO:teuthology.orchestra.run.smithi158.stdout:(60/176): openssl-devel-3.0.7-25.el9.x86_64.rpm 13 MB/s | 3.0 MB 00:00 2024-02-18T11:28:43.775 INFO:teuthology.orchestra.run.smithi158.stdout:(61/176): postgresql-contrib-13.11-1.el9.x86_64 6.4 MB/s | 811 kB 00:00 2024-02-18T11:28:43.800 INFO:teuthology.orchestra.run.smithi158.stdout:(62/176): postgresql-private-libs-13.11-1.el9.x 1.4 MB/s | 134 kB 00:00 2024-02-18T11:28:43.901 INFO:teuthology.orchestra.run.smithi158.stdout:(63/176): protobuf-3.14.0-13.el9.x86_64.rpm 8.1 MB/s | 1.0 MB 00:00 2024-02-18T11:28:43.976 INFO:teuthology.orchestra.run.smithi158.stdout:(64/176): postgresql-server-13.11-1.el9.x86_64. 28 MB/s | 5.7 MB 00:00 2024-02-18T11:28:44.001 INFO:teuthology.orchestra.run.smithi158.stdout:(65/176): python3-attrs-20.3.0-7.el9.noarch.rpm 387 kB/s | 78 kB 00:00 2024-02-18T11:28:44.134 INFO:teuthology.orchestra.run.smithi158.stdout:(66/176): python3-babel-2.9.1-2.el9.noarch.rpm 25 MB/s | 5.8 MB 00:00 2024-02-18T11:28:44.160 INFO:teuthology.orchestra.run.smithi158.stdout:(67/176): python3-jsonpatch-1.21-16.el9.noarch. 157 kB/s | 24 kB 00:00 2024-02-18T11:28:44.185 INFO:teuthology.orchestra.run.smithi158.stdout:(68/176): python3-jinja2-2.11.3-5.el9.noarch.rp 1.2 MB/s | 227 kB 00:00 2024-02-18T11:28:44.210 INFO:teuthology.orchestra.run.smithi158.stdout:(69/176): python3-jsonpointer-2.0-4.el9.noarch. 239 kB/s | 18 kB 00:00 2024-02-18T11:28:44.236 INFO:teuthology.orchestra.run.smithi158.stdout:(70/176): python3-mako-1.1.4-6.el9.noarch.rpm 1.9 MB/s | 144 kB 00:00 2024-02-18T11:28:44.261 INFO:teuthology.orchestra.run.smithi158.stdout:(71/176): python3-markupsafe-1.1.1-12.el9.x86_6 416 kB/s | 32 kB 00:00 2024-02-18T11:28:44.387 INFO:teuthology.orchestra.run.smithi158.stdout:(72/176): python3-numpy-1.20.1-5.el9.x86_64.rpm 27 MB/s | 4.8 MB 00:00 2024-02-18T11:28:44.412 INFO:teuthology.orchestra.run.smithi158.stdout:(73/176): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.2 MB/s | 217 kB 00:00 2024-02-18T11:28:44.428 INFO:teuthology.orchestra.run.smithi158.stdout:(74/176): python3-oauthlib-3.1.1-5.el9.noarch.r 1.0 MB/s | 174 kB 00:00 2024-02-18T11:28:44.454 INFO:teuthology.orchestra.run.smithi158.stdout:(75/176): python3-packaging-20.9-5.el9.noarch.r 1.0 MB/s | 69 kB 00:00 2024-02-18T11:28:44.487 INFO:teuthology.orchestra.run.smithi158.stdout:(76/176): python3-protobuf-3.14.0-13.el9.noarch 4.0 MB/s | 238 kB 00:00 2024-02-18T11:28:44.513 INFO:teuthology.orchestra.run.smithi158.stdout:(77/176): python3-prettytable-0.7.2-27.el9.noar 492 kB/s | 41 kB 00:00 2024-02-18T11:28:44.529 INFO:teuthology.orchestra.run.smithi188.stdout:(7/176): ceph-mon-19.0.0-1555.g1c920afa.el9.x86 2.1 MB/s | 4.7 MB 00:02 2024-02-18T11:28:44.548 INFO:teuthology.orchestra.run.smithi158.stdout:(78/176): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 132 kB 00:00 2024-02-18T11:28:44.574 INFO:teuthology.orchestra.run.smithi158.stdout:(79/176): python3-pyasn1-modules-0.4.8-6.el9.no 2.4 MB/s | 211 kB 00:00 2024-02-18T11:28:44.591 INFO:teuthology.orchestra.run.smithi158.stdout:(80/176): python3-pytz-2021.1-5.el9.noarch.rpm 610 kB/s | 47 kB 00:00 2024-02-18T11:28:44.608 INFO:teuthology.orchestra.run.smithi158.stdout:(81/176): python3-requests-oauthlib-1.3.0-12.el 732 kB/s | 43 kB 00:00 2024-02-18T11:28:44.642 INFO:teuthology.orchestra.run.smithi158.stdout:(82/176): python3-toml-0.10.2-6.el9.noarch.rpm 767 kB/s | 39 kB 00:00 2024-02-18T11:28:44.776 INFO:teuthology.orchestra.run.smithi158.stdout:(83/176): python3-wcwidth-0.2.5-8.el9.noarch.rp 244 kB/s | 41 kB 00:00 2024-02-18T11:28:44.851 INFO:teuthology.orchestra.run.smithi158.stdout:(84/176): qatlib-23.11.0-1.el9.x86_64.rpm 1.0 MB/s | 215 kB 00:00 2024-02-18T11:28:45.043 INFO:teuthology.orchestra.run.smithi158.stdout:(85/176): python3-scipy-1.6.2-8.el9.x86_64.rpm 35 MB/s | 16 MB 00:00 2024-02-18T11:28:45.059 INFO:teuthology.orchestra.run.smithi158.stdout:(86/176): qatlib-service-23.11.0-1.el9.x86_64.r 120 kB/s | 34 kB 00:00 2024-02-18T11:28:45.076 INFO:teuthology.orchestra.run.smithi158.stdout:(87/176): qatzip-libs-1.1.2-2.el9.x86_64.rpm 195 kB/s | 43 kB 00:00 2024-02-18T11:28:45.118 INFO:teuthology.orchestra.run.smithi158.stdout:(88/176): socat-1.7.4.1-5.el9.x86_64.rpm 5.0 MB/s | 300 kB 00:00 2024-02-18T11:28:45.135 INFO:teuthology.orchestra.run.smithi158.stdout:(89/176): uuid-1.6.2-55.el9.x86_64.rpm 752 kB/s | 56 kB 00:00 2024-02-18T11:28:45.160 INFO:teuthology.orchestra.run.smithi158.stdout:(90/176): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.0 MB/s | 87 kB 00:00 2024-02-18T11:28:45.186 INFO:teuthology.orchestra.run.smithi158.stdout:(91/176): xmlstarlet-1.6.1-20.el9.x86_64.rpm 938 kB/s | 63 kB 00:00 2024-02-18T11:28:45.220 INFO:teuthology.orchestra.run.smithi158.stdout:(92/176): fuse-2.9.9-15.el9.x86_64.rpm 1.3 MB/s | 79 kB 00:00 2024-02-18T11:28:45.245 INFO:teuthology.orchestra.run.smithi158.stdout:(93/176): zlib-devel-1.2.11-40.el9.x86_64.rpm 527 kB/s | 44 kB 00:00 2024-02-18T11:28:45.270 INFO:teuthology.orchestra.run.smithi158.stdout:(94/176): jq-1.6-16.el9.x86_64.rpm 2.2 MB/s | 186 kB 00:00 2024-02-18T11:28:45.303 INFO:teuthology.orchestra.run.smithi158.stdout:(95/176): libgfortran-11.4.1-3.el9.x86_64.rpm 9.4 MB/s | 802 kB 00:00 2024-02-18T11:28:45.329 INFO:teuthology.orchestra.run.smithi158.stdout:(96/176): libquadmath-11.4.1-3.el9.x86_64.rpm 3.3 MB/s | 194 kB 00:00 2024-02-18T11:28:45.354 INFO:teuthology.orchestra.run.smithi158.stdout:(97/176): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.6 MB/s | 217 kB 00:00 2024-02-18T11:28:45.379 INFO:teuthology.orchestra.run.smithi158.stdout:(98/176): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.1 MB/s | 241 kB 00:00 2024-02-18T11:28:45.413 INFO:teuthology.orchestra.run.smithi158.stdout:(99/176): python3-cryptography-36.0.1-4.el9.x86 14 MB/s | 1.1 MB 00:00 2024-02-18T11:28:45.438 INFO:teuthology.orchestra.run.smithi158.stdout:(100/176): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 103 kB 00:00 2024-02-18T11:28:45.464 INFO:teuthology.orchestra.run.smithi158.stdout:(101/176): python3-pycparser-2.20-6.el9.noarch. 1.5 MB/s | 124 kB 00:00 2024-02-18T11:28:45.489 INFO:teuthology.orchestra.run.smithi158.stdout:(102/176): python3-requests-2.25.1-8.el9.noarch 1.5 MB/s | 113 kB 00:00 2024-02-18T11:28:45.523 INFO:teuthology.orchestra.run.smithi158.stdout:(103/176): python3-urllib3-1.26.5-5.el9.noarch. 3.1 MB/s | 187 kB 00:00 2024-02-18T11:28:45.740 INFO:teuthology.orchestra.run.smithi158.stdout:(104/176): lua-devel-5.4.4-4.el9.x86_64.rpm 89 kB/s | 22 kB 00:00 2024-02-18T11:28:45.790 INFO:teuthology.orchestra.run.smithi158.stdout:(105/176): python3-devel-3.9.17-2.el9.x86_64.rp 754 kB/s | 245 kB 00:00 2024-02-18T11:28:46.007 INFO:teuthology.orchestra.run.smithi158.stdout:(106/176): protobuf-compiler-3.14.0-13.el9.x86_ 1.7 MB/s | 863 kB 00:00 2024-02-18T11:28:46.182 INFO:teuthology.orchestra.run.smithi158.stdout:(107/176): grpc-data-1.46.7-10.el9.noarch.rpm 111 kB/s | 19 kB 00:00 2024-02-18T11:28:46.441 INFO:teuthology.orchestra.run.smithi158.stdout:(108/176): abseil-cpp-20211102.0-3.el9.x86_64.r 782 kB/s | 548 kB 00:00 2024-02-18T11:28:46.616 INFO:teuthology.orchestra.run.smithi158.stdout:(109/176): libarrow-doc-9.0.0-10.el9.noarch.rpm 148 kB/s | 26 kB 00:00 2024-02-18T11:28:46.637 INFO:teuthology.orchestra.run.smithi188.stdout:(8/176): ceph-common-19.0.0-1555.g1c920afa.el9. 2.9 MB/s | 20 MB 00:06 2024-02-18T11:28:46.733 INFO:teuthology.orchestra.run.smithi158.stdout:(110/176): liboath-2.6.7-2.el9.x86_64.rpm 416 kB/s | 49 kB 00:00 2024-02-18T11:28:46.734 INFO:teuthology.orchestra.run.smithi012.stdout:(10/176): ceph-test-19.0.0-1555.g1c920afa.el9.x 15 MB/s | 47 MB 00:03 2024-02-18T11:28:46.737 INFO:teuthology.orchestra.run.smithi188.stdout:(9/176): ceph-selinux-19.0.0-1555.g1c920afa.el9 253 kB/s | 25 kB 00:00 2024-02-18T11:28:46.809 INFO:teuthology.orchestra.run.smithi012.stdout:(11/176): cephfs-mirror-19.0.0-1555.g1c920afa.e 2.9 MB/s | 220 kB 00:00 2024-02-18T11:28:46.817 INFO:teuthology.orchestra.run.smithi158.stdout:(111/176): gperftools-libs-2.9.1-2.el9.x86_64.r 301 kB/s | 309 kB 00:01 2024-02-18T11:28:46.885 INFO:teuthology.orchestra.run.smithi012.stdout:(12/176): cephfs-shell-19.0.0-1555.g1c920afa.el 361 kB/s | 27 kB 00:00 2024-02-18T11:28:46.934 INFO:teuthology.orchestra.run.smithi158.stdout:(112/176): libunwind-1.6.2-1.el9.x86_64.rpm 336 kB/s | 67 kB 00:00 2024-02-18T11:28:46.959 INFO:teuthology.orchestra.run.smithi158.stdout:(113/176): luarocks-3.9.2-1.el9.noarch.rpm 1.0 MB/s | 151 kB 00:00 2024-02-18T11:28:46.968 INFO:teuthology.orchestra.run.smithi012.stdout:(13/176): libcephfs-devel-19.0.0-1555.g1c920afa 375 kB/s | 31 kB 00:00 2024-02-18T11:28:47.086 INFO:teuthology.orchestra.run.smithi012.stdout:(14/176): libcephfs2-19.0.0-1555.g1c920afa.el9. 6.2 MB/s | 742 kB 00:00 2024-02-18T11:28:47.169 INFO:teuthology.orchestra.run.smithi012.stdout:(15/176): libcephsqlite-19.0.0-1555.g1c920afa.e 1.9 MB/s | 163 kB 00:00 2024-02-18T11:28:47.251 INFO:teuthology.orchestra.run.smithi158.stdout:(114/176): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.6 MB/s | 838 kB 00:00 2024-02-18T11:28:47.253 INFO:teuthology.orchestra.run.smithi012.stdout:(16/176): librados-devel-19.0.0-1555.g1c920afa. 1.5 MB/s | 127 kB 00:00 2024-02-18T11:28:47.276 INFO:teuthology.orchestra.run.smithi158.stdout:(115/176): python3-asyncssh-2.13.2-4.el9.noarch 1.7 MB/s | 548 kB 00:00 2024-02-18T11:28:47.345 INFO:teuthology.orchestra.run.smithi012.stdout:(17/176): libradosstriper1-19.0.0-1555.g1c920af 5.3 MB/s | 499 kB 00:00 2024-02-18T11:28:47.376 INFO:teuthology.orchestra.run.smithi158.stdout:(116/176): python3-cachetools-4.2.4-1.el9.noarc 322 kB/s | 32 kB 00:00 2024-02-18T11:28:47.402 INFO:teuthology.orchestra.run.smithi158.stdout:(117/176): python3-bcrypt-3.2.2-1.el9.x86_64.rp 289 kB/s | 43 kB 00:00 2024-02-18T11:28:47.435 INFO:teuthology.orchestra.run.smithi158.stdout:(118/176): python3-certifi-2021.10.8-2.el9.noar 256 kB/s | 15 kB 00:00 2024-02-18T11:28:47.544 INFO:teuthology.orchestra.run.smithi158.stdout:(119/176): python3-cheroot-8.6.0-4.el9.noarch.r 1.2 MB/s | 172 kB 00:00 2024-02-18T11:28:47.636 INFO:teuthology.orchestra.run.smithi158.stdout:(120/176): python3-cherrypy-18.6.1-2.el9.noarch 1.7 MB/s | 358 kB 00:00 2024-02-18T11:28:47.695 INFO:teuthology.orchestra.run.smithi158.stdout:(121/176): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.0 MB/s | 303 kB 00:00 2024-02-18T11:28:47.720 INFO:teuthology.orchestra.run.smithi158.stdout:(122/176): python3-colorama-0.4.4-12.el9.noarch 422 kB/s | 35 kB 00:00 2024-02-18T11:28:47.737 INFO:teuthology.orchestra.run.smithi012.stdout:(18/176): librgw2-19.0.0-1555.g1c920afa.el9.x86 13 MB/s | 5.1 MB 00:00 2024-02-18T11:28:47.828 INFO:teuthology.orchestra.run.smithi012.stdout:(19/176): python3-ceph-argparse-19.0.0-1555.g1c 494 kB/s | 45 kB 00:00 2024-02-18T11:28:47.904 INFO:teuthology.orchestra.run.smithi012.stdout:(20/176): python3-ceph-common-19.0.0-1555.g1c92 1.7 MB/s | 131 kB 00:00 2024-02-18T11:28:47.979 INFO:teuthology.orchestra.run.smithi012.stdout:(21/176): python3-cephfs-19.0.0-1555.g1c920afa. 2.1 MB/s | 161 kB 00:00 2024-02-18T11:28:48.063 INFO:teuthology.orchestra.run.smithi012.stdout:(22/176): python3-rados-19.0.0-1555.g1c920afa.e 3.8 MB/s | 320 kB 00:00 2024-02-18T11:28:48.078 INFO:teuthology.orchestra.run.smithi158.stdout:(123/176): python3-google-auth-2.27.0-1.el9.noa 528 kB/s | 203 kB 00:00 2024-02-18T11:28:48.138 INFO:teuthology.orchestra.run.smithi012.stdout:(23/176): python3-rbd-19.0.0-1555.g1c920afa.el9 3.9 MB/s | 298 kB 00:00 2024-02-18T11:28:48.196 INFO:teuthology.orchestra.run.smithi158.stdout:(124/176): python3-grpcio-tools-1.46.7-10.el9.x 1.2 MB/s | 144 kB 00:00 2024-02-18T11:28:48.205 INFO:teuthology.orchestra.run.smithi012.stdout:(24/176): python3-rgw-19.0.0-1555.g1c920afa.el9 1.5 MB/s | 99 kB 00:00 2024-02-18T11:28:48.254 INFO:teuthology.orchestra.run.smithi158.stdout:(125/176): libarrow-9.0.0-10.el9.x86_64.rpm 2.1 MB/s | 4.4 MB 00:02 2024-02-18T11:28:48.272 INFO:teuthology.orchestra.run.smithi012.stdout:(25/176): rbd-fuse-19.0.0-1555.g1c920afa.el9.x8 1.3 MB/s | 85 kB 00:00 2024-02-18T11:28:48.379 INFO:teuthology.orchestra.run.smithi188.stdout:(10/176): ceph-osd-19.0.0-1555.g1c920afa.el9.x8 2.8 MB/s | 17 MB 00:05 2024-02-18T11:28:48.506 INFO:teuthology.orchestra.run.smithi012.stdout:(26/176): rbd-mirror-19.0.0-1555.g1c920afa.el9. 13 MB/s | 3.1 MB 00:00 2024-02-18T11:28:48.554 INFO:teuthology.orchestra.run.smithi188.stdout:(11/176): cephfs-mirror-19.0.0-1555.g1c920afa.e 1.2 MB/s | 220 kB 00:00 2024-02-18T11:28:48.588 INFO:teuthology.orchestra.run.smithi158.stdout:(126/176): python3-jaraco-classes-3.2.1-5.el9.n 53 kB/s | 18 kB 00:00 2024-02-18T11:28:48.638 INFO:teuthology.orchestra.run.smithi188.stdout:(12/176): cephfs-shell-19.0.0-1555.g1c920afa.el 325 kB/s | 27 kB 00:00 2024-02-18T11:28:48.638 INFO:teuthology.orchestra.run.smithi158.stdout:(127/176): python3-jaraco-8.2.1-3.el9.noarch.rp 24 kB/s | 11 kB 00:00 2024-02-18T11:28:48.672 INFO:teuthology.orchestra.run.smithi158.stdout:(128/176): python3-jaraco-collections-3.0.0-8.e 277 kB/s | 23 kB 00:00 2024-02-18T11:28:48.706 INFO:teuthology.orchestra.run.smithi012.stdout:(27/176): ceph-osd-19.0.0-1555.g1c920afa.el9.x8 2.6 MB/s | 17 MB 00:06 2024-02-18T11:28:48.730 INFO:teuthology.orchestra.run.smithi188.stdout:(13/176): libcephfs-devel-19.0.0-1555.g1c920afa 340 kB/s | 31 kB 00:00 2024-02-18T11:28:48.731 INFO:teuthology.orchestra.run.smithi012.stdout:(28/176): rbd-nbd-19.0.0-1555.g1c920afa.el9.x86 758 kB/s | 171 kB 00:00 2024-02-18T11:28:48.747 INFO:teuthology.orchestra.run.smithi158.stdout:(129/176): python3-jaraco-text-3.2.0-6.el9.noar 262 kB/s | 20 kB 00:00 2024-02-18T11:28:48.772 INFO:teuthology.orchestra.run.smithi158.stdout:(130/176): python3-jaraco-functools-3.5.0-2.el9 145 kB/s | 19 kB 00:00 2024-02-18T11:28:48.773 INFO:teuthology.orchestra.run.smithi012.stdout:(29/176): ceph-grafana-dashboards-19.0.0-1555.g 472 kB/s | 31 kB 00:00 2024-02-18T11:28:48.815 INFO:teuthology.orchestra.run.smithi012.stdout:(30/176): ceph-mgr-cephadm-19.0.0-1555.g1c920af 1.6 MB/s | 138 kB 00:00 2024-02-18T11:28:48.823 INFO:teuthology.orchestra.run.smithi158.stdout:(131/176): python3-grpcio-1.46.7-10.el9.x86_64. 1.8 MB/s | 2.0 MB 00:01 2024-02-18T11:28:48.922 INFO:teuthology.orchestra.run.smithi188.stdout:(14/176): ceph-radosgw-19.0.0-1555.g1c920afa.el 2.2 MB/s | 9.8 MB 00:04 2024-02-18T11:28:49.022 INFO:teuthology.orchestra.run.smithi188.stdout:(15/176): libcephfs2-19.0.0-1555.g1c920afa.el9. 2.5 MB/s | 742 kB 00:00 2024-02-18T11:28:49.047 INFO:teuthology.orchestra.run.smithi188.stdout:(16/176): libcephsqlite-19.0.0-1555.g1c920afa.e 1.3 MB/s | 163 kB 00:00 2024-02-18T11:28:49.147 INFO:teuthology.orchestra.run.smithi188.stdout:(17/176): librados-devel-19.0.0-1555.g1c920afa. 1.0 MB/s | 127 kB 00:00 2024-02-18T11:28:49.165 INFO:teuthology.orchestra.run.smithi158.stdout:(132/176): python3-kubernetes-26.1.0-2.el9.noar 2.4 MB/s | 1.0 MB 00:00 2024-02-18T11:28:49.187 INFO:teuthology.orchestra.run.smithi158.stdout:(133/176): python3-logutils-0.3.5-21.el9.noarch 112 kB/s | 46 kB 00:00 2024-02-18T11:28:49.287 INFO:teuthology.orchestra.run.smithi158.stdout:(134/176): python3-natsort-7.1.1-5.el9.noarch.r 471 kB/s | 58 kB 00:00 2024-02-18T11:28:49.298 INFO:teuthology.orchestra.run.smithi188.stdout:(18/176): libradosstriper1-19.0.0-1555.g1c920af 1.9 MB/s | 499 kB 00:00 2024-02-18T11:28:49.299 INFO:teuthology.orchestra.run.smithi012.stdout:(31/176): ceph-mgr-diskprediction-local-19.0.0- 15 MB/s | 7.4 MB 00:00 2024-02-18T11:28:49.312 INFO:teuthology.orchestra.run.smithi158.stdout:(135/176): python3-pecan-1.4.2-3.el9.noarch.rpm 2.1 MB/s | 272 kB 00:00 2024-02-18T11:28:49.356 INFO:teuthology.orchestra.run.smithi188.stdout:(19/176): python3-ceph-argparse-19.0.0-1555.g1c 774 kB/s | 45 kB 00:00 2024-02-18T11:28:49.363 INFO:teuthology.orchestra.run.smithi158.stdout:(136/176): python3-portend-3.1.0-2.el9.noarch.r 218 kB/s | 16 kB 00:00 2024-02-18T11:28:49.374 INFO:teuthology.orchestra.run.smithi012.stdout:(32/176): ceph-mgr-modules-core-19.0.0-1555.g1c 3.2 MB/s | 248 kB 00:00 2024-02-18T11:28:49.423 INFO:teuthology.orchestra.run.smithi188.stdout:(20/176): python3-ceph-common-19.0.0-1555.g1c92 1.9 MB/s | 131 kB 00:00 2024-02-18T11:28:49.430 INFO:teuthology.orchestra.run.smithi158.stdout:(137/176): python3-pyperclip-1.8.2-2.el9.noarch 424 kB/s | 28 kB 00:00 2024-02-18T11:28:49.441 INFO:teuthology.orchestra.run.smithi012.stdout:(33/176): ceph-mgr-dashboard-19.0.0-1555.g1c920 2.7 MB/s | 1.8 MB 00:00 2024-02-18T11:28:49.455 INFO:teuthology.orchestra.run.smithi158.stdout:(138/176): python3-pyOpenSSL-21.0.0-1.el9.noarc 632 kB/s | 90 kB 00:00 2024-02-18T11:28:49.466 INFO:teuthology.orchestra.run.smithi012.stdout:(34/176): ceph-mgr-rook-19.0.0-1555.g1c920afa.e 570 kB/s | 52 kB 00:00 2024-02-18T11:28:49.508 INFO:teuthology.orchestra.run.smithi012.stdout:(35/176): ceph-prometheus-alerts-19.0.0-1555.g1 229 kB/s | 15 kB 00:00 2024-02-18T11:28:49.514 INFO:teuthology.orchestra.run.smithi158.stdout:(139/176): python3-repoze-lru-0.7-16.el9.noarch 368 kB/s | 31 kB 00:00 2024-02-18T11:28:49.536 INFO:teuthology.orchestra.run.smithi012.stdout:(36/176): ceph-volume-19.0.0-1555.g1c920afa.el9 3.8 MB/s | 264 kB 00:00 2024-02-18T11:28:49.600 INFO:teuthology.orchestra.run.smithi012.stdout:(37/176): cephfs-top-19.0.0-1555.g1c920afa.el9. 274 kB/s | 18 kB 00:00 2024-02-18T11:28:49.630 INFO:teuthology.orchestra.run.smithi158.stdout:(140/176): python3-routes-2.5.1-5.el9.noarch.rp 1.1 MB/s | 188 kB 00:00 2024-02-18T11:28:49.649 INFO:teuthology.orchestra.run.smithi188.stdout:(21/176): python3-cephfs-19.0.0-1555.g1c920afa. 717 kB/s | 161 kB 00:00 2024-02-18T11:28:49.726 INFO:teuthology.orchestra.run.smithi012.stdout:(38/176): cephadm-19.0.0-1555.g1c920afa.el9.noa 3.4 MB/s | 748 kB 00:00 2024-02-18T11:28:49.757 INFO:teuthology.orchestra.run.smithi188.stdout:(22/176): python3-rados-19.0.0-1555.g1c920afa.e 2.9 MB/s | 320 kB 00:00 2024-02-18T11:28:49.823 INFO:teuthology.orchestra.run.smithi158.stdout:(141/176): python3-tempora-5.0.0-2.el9.noarch.r 187 kB/s | 36 kB 00:00 2024-02-18T11:28:49.848 INFO:teuthology.orchestra.run.smithi158.stdout:(142/176): python3-rsa-4.9-2.el9.noarch.rpm 177 kB/s | 59 kB 00:00 2024-02-18T11:28:49.874 INFO:teuthology.orchestra.run.smithi188.stdout:(23/176): python3-rbd-19.0.0-1555.g1c920afa.el9 2.5 MB/s | 298 kB 00:00 2024-02-18T11:28:49.909 INFO:teuthology.orchestra.run.smithi012.stdout:(39/176): automake-1.16.2-8.el9.noarch.rpm 2.1 MB/s | 662 kB 00:00 2024-02-18T11:28:49.923 INFO:teuthology.orchestra.run.smithi158.stdout:(143/176): python3-typing-extensions-4.4.0-2.el 507 kB/s | 51 kB 00:00 2024-02-18T11:28:49.941 INFO:teuthology.orchestra.run.smithi188.stdout:(24/176): python3-rgw-19.0.0-1555.g1c920afa.el9 1.5 MB/s | 99 kB 00:00 2024-02-18T11:28:49.943 INFO:teuthology.orchestra.run.smithi012.stdout:(40/176): bison-3.7.4-5.el9.x86_64.rpm 4.1 MB/s | 920 kB 00:00 2024-02-18T11:28:49.968 INFO:teuthology.orchestra.run.smithi012.stdout:(41/176): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 106 kB 00:00 2024-02-18T11:28:50.002 INFO:teuthology.orchestra.run.smithi012.stdout:(42/176): elfutils-libelf-devel-0.190-2.el9.x86 349 kB/s | 20 kB 00:00 2024-02-18T11:28:50.008 INFO:teuthology.orchestra.run.smithi188.stdout:(25/176): rbd-fuse-19.0.0-1555.g1c920afa.el9.x8 1.3 MB/s | 85 kB 00:00 2024-02-18T11:28:50.027 INFO:teuthology.orchestra.run.smithi012.stdout:(43/176): flex-2.6.4-9.el9.x86_64.rpm 5.1 MB/s | 307 kB 00:00 2024-02-18T11:28:50.040 INFO:teuthology.orchestra.run.smithi158.stdout:(144/176): python3-websocket-client-1.2.3-2.el9 770 kB/s | 90 kB 00:00 2024-02-18T11:28:50.053 INFO:teuthology.orchestra.run.smithi012.stdout:(44/176): flexiblas-3.0.4-8.el9.x86_64.rpm 636 kB/s | 32 kB 00:00 2024-02-18T11:28:50.170 INFO:teuthology.orchestra.run.smithi012.stdout:(45/176): flexiblas-netlib-3.0.4-8.el9.x86_64.r 21 MB/s | 3.0 MB 00:00 2024-02-18T11:28:50.195 INFO:teuthology.orchestra.run.smithi012.stdout:(46/176): flexiblas-openblas-openmp-3.0.4-8.el9 121 kB/s | 17 kB 00:00 2024-02-18T11:28:50.215 INFO:teuthology.orchestra.run.smithi158.stdout:(145/176): python3-webob-1.8.7-6.el9.noarch.rpm 625 kB/s | 230 kB 00:00 2024-02-18T11:28:50.237 INFO:teuthology.orchestra.run.smithi012.stdout:(47/176): libaio-devel-0.3.111-13.el9.x86_64.rp 243 kB/s | 10 kB 00:00 2024-02-18T11:28:50.262 INFO:teuthology.orchestra.run.smithi012.stdout:(48/176): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.3 MB/s | 159 kB 00:00 2024-02-18T11:28:50.279 INFO:teuthology.orchestra.run.smithi012.stdout:(49/176): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.0 MB/s | 44 kB 00:00 2024-02-18T11:28:50.291 INFO:teuthology.orchestra.run.smithi158.stdout:(146/176): python3-werkzeug-2.0.3-3.el9.1.noarc 1.7 MB/s | 427 kB 00:00 2024-02-18T11:28:50.312 INFO:teuthology.orchestra.run.smithi012.stdout:(50/176): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 661 kB 00:00 2024-02-18T11:28:50.316 INFO:teuthology.orchestra.run.smithi158.stdout:(147/176): python3-zc-lockfile-2.0-10.el9.noarc 199 kB/s | 20 kB 00:00 2024-02-18T11:28:50.338 INFO:teuthology.orchestra.run.smithi012.stdout:(51/176): libtool-2.4.6-45.el9.x86_64.rpm 9.6 MB/s | 578 kB 00:00 2024-02-18T11:28:50.363 INFO:teuthology.orchestra.run.smithi012.stdout:(52/176): libuuid-devel-2.37.4-18.el9.x86_64.rp 497 kB/s | 24 kB 00:00 2024-02-18T11:28:50.388 INFO:teuthology.orchestra.run.smithi012.stdout:(53/176): libzstd-devel-1.5.1-2.el9.x86_64.rpm 929 kB/s | 46 kB 00:00 2024-02-18T11:28:50.414 INFO:teuthology.orchestra.run.smithi012.stdout:(54/176): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.5 MB/s | 282 kB 00:00 2024-02-18T11:28:50.417 INFO:teuthology.orchestra.run.smithi188.stdout:(26/176): librgw2-19.0.0-1555.g1c920afa.el9.x86 4.0 MB/s | 5.1 MB 00:01 2024-02-18T11:28:50.439 INFO:teuthology.orchestra.run.smithi012.stdout:(55/176): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2024-02-18T11:28:50.465 INFO:teuthology.orchestra.run.smithi012.stdout:(56/176): openblas-0.3.21-2.el9.x86_64.rpm 685 kB/s | 34 kB 00:00 2024-02-18T11:28:50.466 INFO:teuthology.orchestra.run.smithi158.stdout:(148/176): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-02-18T11:28:50.501 INFO:teuthology.orchestra.run.smithi188.stdout:(27/176): rbd-nbd-19.0.0-1555.g1c920afa.el9.x86 2.0 MB/s | 171 kB 00:00 2024-02-18T11:28:50.565 INFO:teuthology.orchestra.run.smithi012.stdout:(57/176): openblas-openmp-0.3.21-2.el9.x86_64.r 38 MB/s | 4.7 MB 00:00 2024-02-18T11:28:50.567 INFO:teuthology.orchestra.run.smithi188.stdout:(28/176): ceph-grafana-dashboards-19.0.0-1555.g 473 kB/s | 31 kB 00:00 2024-02-18T11:28:50.623 INFO:teuthology.orchestra.run.smithi012.stdout:(58/176): openssl-devel-3.0.7-25.el9.x86_64.rpm 19 MB/s | 3.0 MB 00:00 2024-02-18T11:28:50.634 INFO:teuthology.orchestra.run.smithi188.stdout:(29/176): ceph-mgr-cephadm-19.0.0-1555.g1c920af 2.0 MB/s | 138 kB 00:00 2024-02-18T11:28:50.665 INFO:teuthology.orchestra.run.smithi012.stdout:(59/176): postgresql-13.11-1.el9.x86_64.rpm 16 MB/s | 1.5 MB 00:00 2024-02-18T11:28:50.707 INFO:teuthology.orchestra.run.smithi012.stdout:(60/176): postgresql-contrib-13.11-1.el9.x86_64 9.8 MB/s | 811 kB 00:00 2024-02-18T11:28:50.718 INFO:teuthology.orchestra.run.smithi188.stdout:(30/176): rbd-mirror-19.0.0-1555.g1c920afa.el9. 4.4 MB/s | 3.1 MB 00:00 2024-02-18T11:28:50.725 INFO:teuthology.orchestra.run.smithi158.stdout:(149/176): python3-more-itertools-8.12.0-2.el9. 41 kB/s | 79 kB 00:01 2024-02-18T11:28:50.732 INFO:teuthology.orchestra.run.smithi012.stdout:(61/176): postgresql-private-libs-13.11-1.el9.x 2.0 MB/s | 134 kB 00:00 2024-02-18T11:28:50.825 INFO:teuthology.orchestra.run.smithi012.stdout:(62/176): postgresql-server-13.11-1.el9.x86_64. 49 MB/s | 5.7 MB 00:00 2024-02-18T11:28:50.833 INFO:teuthology.orchestra.run.smithi158.stdout:(150/176): re2-20211101-3.el9.x86_64.rpm 353 kB/s | 192 kB 00:00 2024-02-18T11:28:50.866 INFO:teuthology.orchestra.run.smithi012.stdout:(63/176): protobuf-3.14.0-13.el9.x86_64.rpm 7.7 MB/s | 1.0 MB 00:00 2024-02-18T11:28:50.867 INFO:teuthology.orchestra.run.smithi158.stdout:(151/176): libacl-devel-2.3.1-4.el9.x86_64.rpm 2.0 MB/s | 68 kB 00:00 2024-02-18T11:28:50.892 INFO:teuthology.orchestra.run.smithi012.stdout:(64/176): python3-attrs-20.3.0-7.el9.noarch.rpm 1.2 MB/s | 78 kB 00:00 2024-02-18T11:28:50.959 INFO:teuthology.orchestra.run.smithi158.stdout:(152/176): NetworkManager-1.45.91-1.el9.x86_64. 25 MB/s | 2.3 MB 00:00 2024-02-18T11:28:51.026 INFO:teuthology.orchestra.run.smithi158.stdout:(153/176): NetworkManager-libnm-1.45.91-1.el9.x 27 MB/s | 1.8 MB 00:00 2024-02-18T11:28:51.035 INFO:teuthology.orchestra.run.smithi188.stdout:(31/176): ceph-mgr-dashboard-19.0.0-1555.g1c920 4.5 MB/s | 1.8 MB 00:00 2024-02-18T11:28:51.059 INFO:teuthology.orchestra.run.smithi158.stdout:(154/176): NetworkManager-team-1.45.91-1.el9.x8 1.2 MB/s | 41 kB 00:00 2024-02-18T11:28:51.100 INFO:teuthology.orchestra.run.smithi012.stdout:(65/176): ceph-common-19.0.0-1555.g1c920afa.el9 1.7 MB/s | 20 MB 00:11 2024-02-18T11:28:51.110 INFO:teuthology.orchestra.run.smithi158.stdout:(155/176): librados2-19.0.0-1555.g1c920afa.el9. 5.2 MB/s | 3.4 MB 00:00 2024-02-18T11:28:51.119 INFO:teuthology.orchestra.run.smithi188.stdout:(32/176): ceph-mgr-modules-core-19.0.0-1555.g1c 2.9 MB/s | 248 kB 00:00 2024-02-18T11:28:51.143 INFO:teuthology.orchestra.run.smithi158.stdout:(156/176): NetworkManager-tui-1.45.91-1.el9.x86 2.9 MB/s | 246 kB 00:00 2024-02-18T11:28:51.169 INFO:teuthology.orchestra.run.smithi158.stdout:(157/176): acl-2.3.1-4.el9.x86_64.rpm 1.2 MB/s | 69 kB 00:00 2024-02-18T11:28:51.184 INFO:teuthology.orchestra.run.smithi012.stdout:(66/176): python3-babel-2.9.1-2.el9.noarch.rpm 18 MB/s | 5.8 MB 00:00 2024-02-18T11:28:51.186 INFO:teuthology.orchestra.run.smithi188.stdout:(33/176): ceph-mgr-rook-19.0.0-1555.g1c920afa.e 786 kB/s | 52 kB 00:00 2024-02-18T11:28:51.194 INFO:teuthology.orchestra.run.smithi158.stdout:(158/176): elfutils-debuginfod-client-0.190-2.e 708 kB/s | 35 kB 00:00 2024-02-18T11:28:51.209 INFO:teuthology.orchestra.run.smithi012.stdout:(67/176): python3-jinja2-2.11.3-5.el9.noarch.rp 723 kB/s | 227 kB 00:00 2024-02-18T11:28:51.219 INFO:teuthology.orchestra.run.smithi158.stdout:(159/176): elfutils-libelf-0.190-2.el9.x86_64.r 3.7 MB/s | 191 kB 00:00 2024-02-18T11:28:51.226 INFO:teuthology.orchestra.run.smithi012.stdout:(68/176): python3-jsonpatch-1.21-16.el9.noarch. 200 kB/s | 24 kB 00:00 2024-02-18T11:28:51.244 INFO:teuthology.orchestra.run.smithi188.stdout:(34/176): ceph-prometheus-alerts-19.0.0-1555.g1 264 kB/s | 15 kB 00:00 2024-02-18T11:28:51.245 INFO:teuthology.orchestra.run.smithi158.stdout:(160/176): elfutils-libs-0.190-2.el9.x86_64.rpm 5.0 MB/s | 254 kB 00:00 2024-02-18T11:28:51.260 INFO:teuthology.orchestra.run.smithi012.stdout:(69/176): python3-jsonpointer-2.0-4.el9.noarch. 359 kB/s | 18 kB 00:00 2024-02-18T11:28:51.278 INFO:teuthology.orchestra.run.smithi158.stdout:(161/176): iproute-6.7.0-2.el9.x86_64.rpm 14 MB/s | 802 kB 00:00 2024-02-18T11:28:51.285 INFO:teuthology.orchestra.run.smithi012.stdout:(70/176): python3-mako-1.1.4-6.el9.noarch.rpm 1.9 MB/s | 144 kB 00:00 2024-02-18T11:28:51.302 INFO:teuthology.orchestra.run.smithi012.stdout:(71/176): python3-markupsafe-1.1.1-12.el9.x86_6 419 kB/s | 32 kB 00:00 2024-02-18T11:28:51.303 INFO:teuthology.orchestra.run.smithi158.stdout:(162/176): iproute-tc-6.7.0-2.el9.x86_64.rpm 7.4 MB/s | 433 kB 00:00 2024-02-18T11:28:51.329 INFO:teuthology.orchestra.run.smithi158.stdout:(163/176): libacl-2.3.1-4.el9.x86_64.rpm 444 kB/s | 22 kB 00:00 2024-02-18T11:28:51.335 INFO:teuthology.orchestra.run.smithi012.stdout:(72/176): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.3 MB/s | 217 kB 00:00 2024-02-18T11:28:51.336 INFO:teuthology.orchestra.run.smithi188.stdout:(35/176): ceph-volume-19.0.0-1555.g1c920afa.el9 2.8 MB/s | 264 kB 00:00 2024-02-18T11:28:51.354 INFO:teuthology.orchestra.run.smithi158.stdout:(164/176): libblkid-2.37.4-18.el9.x86_64.rpm 2.1 MB/s | 107 kB 00:00 2024-02-18T11:28:51.379 INFO:teuthology.orchestra.run.smithi158.stdout:(165/176): libfdisk-2.37.4-18.el9.x86_64.rpm 3.0 MB/s | 154 kB 00:00 2024-02-18T11:28:51.396 INFO:teuthology.orchestra.run.smithi158.stdout:(166/176): libmount-2.37.4-18.el9.x86_64.rpm 3.2 MB/s | 135 kB 00:00 2024-02-18T11:28:51.403 INFO:teuthology.orchestra.run.smithi012.stdout:(73/176): python3-numpy-1.20.1-5.el9.x86_64.rpm 41 MB/s | 4.8 MB 00:00 2024-02-18T11:28:51.422 INFO:teuthology.orchestra.run.smithi158.stdout:(167/176): libsmartcols-2.37.4-18.el9.x86_64.rp 1.5 MB/s | 63 kB 00:00 2024-02-18T11:28:51.428 INFO:teuthology.orchestra.run.smithi012.stdout:(74/176): python3-oauthlib-3.1.1-5.el9.noarch.r 1.4 MB/s | 174 kB 00:00 2024-02-18T11:28:51.438 INFO:teuthology.orchestra.run.smithi158.stdout:(168/176): libuuid-2.37.4-18.el9.x86_64.rpm 662 kB/s | 28 kB 00:00 2024-02-18T11:28:51.453 INFO:teuthology.orchestra.run.smithi012.stdout:(75/176): python3-packaging-20.9-5.el9.noarch.r 602 kB/s | 69 kB 00:00 2024-02-18T11:28:51.470 INFO:teuthology.orchestra.run.smithi012.stdout:(76/176): python3-prettytable-0.7.2-27.el9.noar 993 kB/s | 41 kB 00:00 2024-02-18T11:28:51.472 INFO:teuthology.orchestra.run.smithi158.stdout:(169/176): openssl-3.0.7-25.el9.x86_64.rpm 23 MB/s | 1.2 MB 00:00 2024-02-18T11:28:51.494 INFO:teuthology.orchestra.run.smithi188.stdout:(36/176): cephadm-19.0.0-1555.g1c920afa.el9.noa 4.6 MB/s | 748 kB 00:00 2024-02-18T11:28:51.495 INFO:teuthology.orchestra.run.smithi012.stdout:(77/176): python3-protobuf-3.14.0-13.el9.noarch 3.5 MB/s | 238 kB 00:00 2024-02-18T11:28:51.514 INFO:teuthology.orchestra.run.smithi158.stdout:(170/176): openssl-libs-3.0.7-25.el9.x86_64.rpm 29 MB/s | 2.1 MB 00:00 2024-02-18T11:28:51.520 INFO:teuthology.orchestra.run.smithi012.stdout:(78/176): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 132 kB 00:00 2024-02-18T11:28:51.546 INFO:teuthology.orchestra.run.smithi012.stdout:(79/176): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 211 kB 00:00 2024-02-18T11:28:51.562 INFO:teuthology.orchestra.run.smithi188.stdout:(37/176): cephfs-top-19.0.0-1555.g1c920afa.el9. 274 kB/s | 18 kB 00:00 2024-02-18T11:28:51.562 INFO:teuthology.orchestra.run.smithi012.stdout:(80/176): python3-pytz-2021.1-5.el9.noarch.rpm 713 kB/s | 47 kB 00:00 2024-02-18T11:28:51.564 INFO:teuthology.orchestra.run.smithi158.stdout:(171/176): util-linux-2.37.4-18.el9.x86_64.rpm 25 MB/s | 2.2 MB 00:00 2024-02-18T11:28:51.579 INFO:teuthology.orchestra.run.smithi012.stdout:(81/176): python3-requests-oauthlib-1.3.0-12.el 736 kB/s | 43 kB 00:00 2024-02-18T11:28:51.589 INFO:teuthology.orchestra.run.smithi158.stdout:(172/176): util-linux-core-2.37.4-18.el9.x86_64 5.9 MB/s | 438 kB 00:00 2024-02-18T11:28:51.596 INFO:teuthology.orchestra.run.smithi012.stdout:(82/176): python3-toml-0.10.2-6.el9.noarch.rpm 1.1 MB/s | 39 kB 00:00 2024-02-18T11:28:51.615 INFO:teuthology.orchestra.run.smithi158.stdout:(173/176): util-linux-user-2.37.4-18.el9.x86_64 609 kB/s | 30 kB 00:00 2024-02-18T11:28:51.647 INFO:teuthology.orchestra.run.smithi012.stdout:(83/176): python3-wcwidth-0.2.5-8.el9.noarch.rp 646 kB/s | 41 kB 00:00 2024-02-18T11:28:51.648 INFO:teuthology.orchestra.run.smithi158.stdout:(174/176): xfsdump-3.1.12-4.el9.x86_64.rpm 5.4 MB/s | 321 kB 00:00 2024-02-18T11:28:51.682 INFO:teuthology.orchestra.run.smithi158.stdout:(175/176): xfsprogs-6.3.0-1.el9.x86_64.rpm 16 MB/s | 1.1 MB 00:00 2024-02-18T11:28:51.687 INFO:teuthology.orchestra.run.smithi188.stdout:(38/176): automake-1.16.2-8.el9.noarch.rpm 5.2 MB/s | 662 kB 00:00 2024-02-18T11:28:51.733 INFO:teuthology.orchestra.run.smithi158.stdout:(176/176): librbd1-19.0.0-1555.g1c920afa.el9.x8 3.1 MB/s | 3.1 MB 00:01 2024-02-18T11:28:51.736 INFO:teuthology.orchestra.run.smithi158.stdout:-------------------------------------------------------------------------------- 2024-02-18T11:28:51.737 INFO:teuthology.orchestra.run.smithi158.stdout:Total 14 MB/s | 223 MB 00:15 2024-02-18T11:28:51.745 INFO:teuthology.orchestra.run.smithi188.stdout:(39/176): bison-3.7.4-5.el9.x86_64.rpm 15 MB/s | 920 kB 00:00 2024-02-18T11:28:51.779 INFO:teuthology.orchestra.run.smithi188.stdout:(40/176): boost-program-options-1.75.0-8.el9.x8 3.1 MB/s | 106 kB 00:00 2024-02-18T11:28:51.788 INFO:teuthology.orchestra.run.smithi158.stdout:Running transaction check 2024-02-18T11:28:51.813 INFO:teuthology.orchestra.run.smithi188.stdout:(41/176): elfutils-libelf-devel-0.190-2.el9.x86 617 kB/s | 20 kB 00:00 2024-02-18T11:28:51.830 INFO:teuthology.orchestra.run.smithi012.stdout:(84/176): python3-scipy-1.6.2-8.el9.x86_64.rpm 59 MB/s | 16 MB 00:00 2024-02-18T11:28:51.846 INFO:teuthology.orchestra.run.smithi188.stdout:(42/176): flex-2.6.4-9.el9.x86_64.rpm 9.0 MB/s | 307 kB 00:00 2024-02-18T11:28:51.855 INFO:teuthology.orchestra.run.smithi012.stdout:(85/176): qatlib-23.11.0-1.el9.x86_64.rpm 843 kB/s | 215 kB 00:00 2024-02-18T11:28:51.872 INFO:teuthology.orchestra.run.smithi012.stdout:(86/176): qatlib-service-23.11.0-1.el9.x86_64.r 152 kB/s | 34 kB 00:00 2024-02-18T11:28:51.889 INFO:teuthology.orchestra.run.smithi012.stdout:(87/176): qatzip-libs-1.1.2-2.el9.x86_64.rpm 748 kB/s | 43 kB 00:00 2024-02-18T11:28:51.923 INFO:teuthology.orchestra.run.smithi012.stdout:(88/176): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 300 kB 00:00 2024-02-18T11:28:51.930 INFO:teuthology.orchestra.run.smithi188.stdout:(43/176): ceph-mgr-diskprediction-local-19.0.0- 6.1 MB/s | 7.4 MB 00:01 2024-02-18T11:28:51.940 INFO:teuthology.orchestra.run.smithi012.stdout:(89/176): uuid-1.6.2-55.el9.x86_64.rpm 845 kB/s | 56 kB 00:00 2024-02-18T11:28:51.955 INFO:teuthology.orchestra.run.smithi188.stdout:(44/176): flexiblas-3.0.4-8.el9.x86_64.rpm 291 kB/s | 32 kB 00:00 2024-02-18T11:28:51.965 INFO:teuthology.orchestra.run.smithi012.stdout:(90/176): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.1 MB/s | 87 kB 00:00 2024-02-18T11:28:51.982 INFO:teuthology.orchestra.run.smithi012.stdout:(91/176): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 63 kB 00:00 2024-02-18T11:28:51.998 INFO:teuthology.orchestra.run.smithi012.stdout:(92/176): zlib-devel-1.2.11-40.el9.x86_64.rpm 758 kB/s | 44 kB 00:00 2024-02-18T11:28:52.022 INFO:teuthology.orchestra.run.smithi188.stdout:(45/176): flexiblas-netlib-3.0.4-8.el9.x86_64.r 33 MB/s | 3.0 MB 00:00 2024-02-18T11:28:52.024 INFO:teuthology.orchestra.run.smithi012.stdout:(93/176): fuse-2.9.9-15.el9.x86_64.rpm 1.3 MB/s | 79 kB 00:00 2024-02-18T11:28:52.039 INFO:teuthology.orchestra.run.smithi188.stdout:(46/176): flexiblas-openblas-openmp-3.0.4-8.el9 212 kB/s | 17 kB 00:00 2024-02-18T11:28:52.049 INFO:teuthology.orchestra.run.smithi012.stdout:(94/176): jq-1.6-16.el9.x86_64.rpm 3.6 MB/s | 186 kB 00:00 2024-02-18T11:28:52.056 INFO:teuthology.orchestra.run.smithi188.stdout:(47/176): libaio-devel-0.3.111-13.el9.x86_64.rp 302 kB/s | 10 kB 00:00 2024-02-18T11:28:52.067 INFO:teuthology.orchestra.run.smithi158.stdout:Transaction check succeeded. 2024-02-18T11:28:52.067 INFO:teuthology.orchestra.run.smithi158.stdout:Running transaction test 2024-02-18T11:28:52.074 INFO:teuthology.orchestra.run.smithi012.stdout:(95/176): libgfortran-11.4.1-3.el9.x86_64.rpm 10 MB/s | 802 kB 00:00 2024-02-18T11:28:52.081 INFO:teuthology.orchestra.run.smithi188.stdout:(48/176): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.7 MB/s | 159 kB 00:00 2024-02-18T11:28:52.098 INFO:teuthology.orchestra.run.smithi188.stdout:(49/176): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.0 MB/s | 44 kB 00:00 2024-02-18T11:28:52.100 INFO:teuthology.orchestra.run.smithi012.stdout:(96/176): libquadmath-11.4.1-3.el9.x86_64.rpm 2.6 MB/s | 194 kB 00:00 2024-02-18T11:28:52.125 INFO:teuthology.orchestra.run.smithi012.stdout:(97/176): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.3 MB/s | 217 kB 00:00 2024-02-18T11:28:52.150 INFO:teuthology.orchestra.run.smithi012.stdout:(98/176): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.2 MB/s | 241 kB 00:00 2024-02-18T11:28:52.157 INFO:teuthology.orchestra.run.smithi188.stdout:(50/176): librdkafka-1.6.1-102.el9.x86_64.rpm 8.6 MB/s | 661 kB 00:00 2024-02-18T11:28:52.182 INFO:teuthology.orchestra.run.smithi188.stdout:(51/176): libtool-2.4.6-45.el9.x86_64.rpm 6.9 MB/s | 578 kB 00:00 2024-02-18T11:28:52.184 INFO:teuthology.orchestra.run.smithi012.stdout:(99/176): python3-cryptography-36.0.1-4.el9.x86 14 MB/s | 1.1 MB 00:00 2024-02-18T11:28:52.199 INFO:teuthology.orchestra.run.smithi188.stdout:(52/176): libuuid-devel-2.37.4-18.el9.x86_64.rp 599 kB/s | 24 kB 00:00 2024-02-18T11:28:52.209 INFO:teuthology.orchestra.run.smithi012.stdout:(100/176): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 103 kB 00:00 2024-02-18T11:28:52.216 INFO:teuthology.orchestra.run.smithi188.stdout:(53/176): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.4 MB/s | 46 kB 00:00 2024-02-18T11:28:52.226 INFO:teuthology.orchestra.run.smithi012.stdout:(101/176): python3-pycparser-2.20-6.el9.noarch. 1.7 MB/s | 124 kB 00:00 2024-02-18T11:28:52.242 INFO:teuthology.orchestra.run.smithi188.stdout:(54/176): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.5 MB/s | 282 kB 00:00 2024-02-18T11:28:52.252 INFO:teuthology.orchestra.run.smithi012.stdout:(102/176): python3-requests-2.25.1-8.el9.noarch 1.7 MB/s | 113 kB 00:00 2024-02-18T11:28:52.267 INFO:teuthology.orchestra.run.smithi188.stdout:(55/176): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2024-02-18T11:28:52.277 INFO:teuthology.orchestra.run.smithi012.stdout:(103/176): python3-urllib3-1.26.5-5.el9.noarch. 3.6 MB/s | 187 kB 00:00 2024-02-18T11:28:52.284 INFO:teuthology.orchestra.run.smithi188.stdout:(56/176): openblas-0.3.21-2.el9.x86_64.rpm 826 kB/s | 34 kB 00:00 2024-02-18T11:28:52.359 INFO:teuthology.orchestra.run.smithi188.stdout:(57/176): openblas-openmp-0.3.21-2.el9.x86_64.r 52 MB/s | 4.7 MB 00:00 2024-02-18T11:28:52.360 INFO:teuthology.orchestra.run.smithi012.stdout:(104/176): lua-devel-5.4.4-4.el9.x86_64.rpm 206 kB/s | 22 kB 00:00 2024-02-18T11:28:52.410 INFO:teuthology.orchestra.run.smithi188.stdout:(58/176): openssl-devel-3.0.7-25.el9.x86_64.rpm 24 MB/s | 3.0 MB 00:00 2024-02-18T11:28:52.451 INFO:teuthology.orchestra.run.smithi188.stdout:(59/176): postgresql-13.11-1.el9.x86_64.rpm 17 MB/s | 1.5 MB 00:00 2024-02-18T11:28:52.477 INFO:teuthology.orchestra.run.smithi188.stdout:(60/176): postgresql-contrib-13.11-1.el9.x86_64 12 MB/s | 811 kB 00:00 2024-02-18T11:28:52.502 INFO:teuthology.orchestra.run.smithi188.stdout:(61/176): postgresql-private-libs-13.11-1.el9.x 2.7 MB/s | 134 kB 00:00 2024-02-18T11:28:52.536 INFO:teuthology.orchestra.run.smithi012.stdout:(105/176): python3-devel-3.9.17-2.el9.x86_64.rp 794 kB/s | 245 kB 00:00 2024-02-18T11:28:52.569 INFO:teuthology.orchestra.run.smithi012.stdout:(106/176): abseil-cpp-20211102.0-3.el9.x86_64.r 2.6 MB/s | 548 kB 00:00 2024-02-18T11:28:52.594 INFO:teuthology.orchestra.run.smithi012.stdout:(107/176): protobuf-compiler-3.14.0-13.el9.x86_ 2.7 MB/s | 863 kB 00:00 2024-02-18T11:28:52.611 INFO:teuthology.orchestra.run.smithi188.stdout:(62/176): postgresql-server-13.11-1.el9.x86_64. 43 MB/s | 5.7 MB 00:00 2024-02-18T11:28:52.611 INFO:teuthology.orchestra.run.smithi012.stdout:(108/176): grpc-data-1.46.7-10.el9.noarch.rpm 467 kB/s | 19 kB 00:00 2024-02-18T11:28:52.645 INFO:teuthology.orchestra.run.smithi012.stdout:(109/176): libarrow-doc-9.0.0-10.el9.noarch.rpm 777 kB/s | 26 kB 00:00 2024-02-18T11:28:52.652 INFO:teuthology.orchestra.run.smithi188.stdout:(63/176): protobuf-3.14.0-13.el9.x86_64.rpm 6.8 MB/s | 1.0 MB 00:00 2024-02-18T11:28:52.678 INFO:teuthology.orchestra.run.smithi188.stdout:(64/176): python3-attrs-20.3.0-7.el9.noarch.rpm 1.2 MB/s | 78 kB 00:00 2024-02-18T11:28:52.678 INFO:teuthology.orchestra.run.smithi012.stdout:(110/176): liboath-2.6.7-2.el9.x86_64.rpm 1.4 MB/s | 49 kB 00:00 2024-02-18T11:28:52.704 INFO:teuthology.orchestra.run.smithi012.stdout:(111/176): gperftools-libs-2.9.1-2.el9.x86_64.r 1.8 MB/s | 309 kB 00:00 2024-02-18T11:28:52.729 INFO:teuthology.orchestra.run.smithi012.stdout:(112/176): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-02-18T11:28:52.754 INFO:teuthology.orchestra.run.smithi012.stdout:(113/176): luarocks-3.9.2-1.el9.noarch.rpm 2.9 MB/s | 151 kB 00:00 2024-02-18T11:28:52.778 INFO:teuthology.orchestra.run.smithi188.stdout:(65/176): python3-babel-2.9.1-2.el9.noarch.rpm 46 MB/s | 5.8 MB 00:00 2024-02-18T11:28:52.803 INFO:teuthology.orchestra.run.smithi188.stdout:(66/176): python3-jinja2-2.11.3-5.el9.noarch.rp 1.8 MB/s | 227 kB 00:00 2024-02-18T11:28:52.813 INFO:teuthology.orchestra.run.smithi012.stdout:(114/176): parquet-libs-9.0.0-10.el9.x86_64.rpm 9.8 MB/s | 838 kB 00:00 2024-02-18T11:28:52.829 INFO:teuthology.orchestra.run.smithi188.stdout:(67/176): python3-jsonpatch-1.21-16.el9.noarch. 494 kB/s | 24 kB 00:00 2024-02-18T11:28:52.838 INFO:teuthology.orchestra.run.smithi012.stdout:(115/176): python3-asyncssh-2.13.2-4.el9.noarch 6.4 MB/s | 548 kB 00:00 2024-02-18T11:28:52.848 INFO:teuthology.orchestra.run.smithi188.stdout:(68/176): python3-jsonpointer-2.0-4.el9.noarch. 408 kB/s | 18 kB 00:00 2024-02-18T11:28:52.855 INFO:teuthology.orchestra.run.smithi012.stdout:(116/176): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.0 MB/s | 43 kB 00:00 2024-02-18T11:28:52.873 INFO:teuthology.orchestra.run.smithi188.stdout:(69/176): python3-mako-1.1.4-6.el9.noarch.rpm 3.2 MB/s | 144 kB 00:00 2024-02-18T11:28:52.890 INFO:teuthology.orchestra.run.smithi188.stdout:(70/176): python3-markupsafe-1.1.1-12.el9.x86_6 759 kB/s | 32 kB 00:00 2024-02-18T11:28:52.964 INFO:teuthology.orchestra.run.smithi012.stdout:(117/176): libarrow-9.0.0-10.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-02-18T11:28:52.974 INFO:teuthology.orchestra.run.smithi188.stdout:(71/176): python3-numpy-1.20.1-5.el9.x86_64.rpm 48 MB/s | 4.8 MB 00:00 2024-02-18T11:28:53.000 INFO:teuthology.orchestra.run.smithi188.stdout:(72/176): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.0 MB/s | 217 kB 00:00 2024-02-18T11:28:53.025 INFO:teuthology.orchestra.run.smithi188.stdout:(73/176): python3-oauthlib-3.1.1-5.el9.noarch.r 3.4 MB/s | 174 kB 00:00 2024-02-18T11:28:53.050 INFO:teuthology.orchestra.run.smithi188.stdout:(74/176): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 69 kB 00:00 2024-02-18T11:28:53.067 INFO:teuthology.orchestra.run.smithi188.stdout:(75/176): python3-prettytable-0.7.2-27.el9.noar 985 kB/s | 41 kB 00:00 2024-02-18T11:28:53.092 INFO:teuthology.orchestra.run.smithi188.stdout:(76/176): python3-protobuf-3.14.0-13.el9.noarch 5.6 MB/s | 238 kB 00:00 2024-02-18T11:28:53.118 INFO:teuthology.orchestra.run.smithi188.stdout:(77/176): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 132 kB 00:00 2024-02-18T11:28:53.151 INFO:teuthology.orchestra.run.smithi188.stdout:(78/176): python3-pyasn1-modules-0.4.8-6.el9.no 3.5 MB/s | 211 kB 00:00 2024-02-18T11:28:53.169 INFO:teuthology.orchestra.run.smithi188.stdout:(79/176): python3-pytz-2021.1-5.el9.noarch.rpm 951 kB/s | 47 kB 00:00 2024-02-18T11:28:53.185 INFO:teuthology.orchestra.run.smithi188.stdout:(80/176): python3-requests-oauthlib-1.3.0-12.el 1.3 MB/s | 43 kB 00:00 2024-02-18T11:28:53.269 INFO:teuthology.orchestra.run.smithi188.stdout:(81/176): python3-toml-0.10.2-6.el9.noarch.rpm 479 kB/s | 39 kB 00:00 2024-02-18T11:28:53.339 INFO:teuthology.orchestra.run.smithi012.stdout:(118/176): python3-certifi-2021.10.8-2.el9.noar 31 kB/s | 15 kB 00:00 2024-02-18T11:28:53.356 INFO:teuthology.orchestra.run.smithi012.stdout:(119/176): python3-cachetools-4.2.4-1.el9.noarc 62 kB/s | 32 kB 00:00 2024-02-18T11:28:53.381 INFO:teuthology.orchestra.run.smithi012.stdout:(120/176): python3-cheroot-8.6.0-4.el9.noarch.r 412 kB/s | 172 kB 00:00 2024-02-18T11:28:53.407 INFO:teuthology.orchestra.run.smithi012.stdout:(121/176): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.9 MB/s | 303 kB 00:00 2024-02-18T11:28:53.419 INFO:teuthology.orchestra.run.smithi188.stdout:(82/176): python3-scipy-1.6.2-8.el9.x86_64.rpm 63 MB/s | 16 MB 00:00 2024-02-18T11:28:53.424 INFO:teuthology.orchestra.run.smithi012.stdout:(122/176): python3-colorama-0.4.4-12.el9.noarch 840 kB/s | 35 kB 00:00 2024-02-18T11:28:53.436 INFO:teuthology.orchestra.run.smithi188.stdout:(83/176): python3-wcwidth-0.2.5-8.el9.noarch.rp 248 kB/s | 41 kB 00:00 2024-02-18T11:28:53.449 INFO:teuthology.orchestra.run.smithi012.stdout:(123/176): python3-cherrypy-18.6.1-2.el9.noarch 3.2 MB/s | 358 kB 00:00 2024-02-18T11:28:53.462 INFO:teuthology.orchestra.run.smithi188.stdout:(84/176): qatlib-23.11.0-1.el9.x86_64.rpm 5.0 MB/s | 215 kB 00:00 2024-02-18T11:28:53.474 INFO:teuthology.orchestra.run.smithi012.stdout:(124/176): python3-google-auth-2.27.0-1.el9.noa 3.0 MB/s | 203 kB 00:00 2024-02-18T11:28:53.479 INFO:teuthology.orchestra.run.smithi188.stdout:(85/176): qatlib-service-23.11.0-1.el9.x86_64.r 808 kB/s | 34 kB 00:00 2024-02-18T11:28:53.496 INFO:teuthology.orchestra.run.smithi188.stdout:(86/176): qatzip-libs-1.1.2-2.el9.x86_64.rpm 1.3 MB/s | 43 kB 00:00 2024-02-18T11:28:53.499 INFO:teuthology.orchestra.run.smithi012.stdout:(125/176): python3-grpcio-tools-1.46.7-10.el9.x 2.8 MB/s | 144 kB 00:00 2024-02-18T11:28:53.516 INFO:teuthology.orchestra.run.smithi012.stdout:(126/176): python3-jaraco-8.2.1-3.el9.noarch.rp 255 kB/s | 11 kB 00:00 2024-02-18T11:28:53.529 INFO:teuthology.orchestra.run.smithi188.stdout:(87/176): socat-1.7.4.1-5.el9.x86_64.rpm 5.8 MB/s | 300 kB 00:00 2024-02-18T11:28:53.533 INFO:teuthology.orchestra.run.smithi012.stdout:(127/176): python3-jaraco-classes-3.2.1-5.el9.n 528 kB/s | 18 kB 00:00 2024-02-18T11:28:53.546 INFO:teuthology.orchestra.run.smithi188.stdout:(88/176): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 56 kB 00:00 2024-02-18T11:28:53.550 INFO:teuthology.orchestra.run.smithi012.stdout:(128/176): python3-jaraco-collections-3.0.0-8.e 688 kB/s | 23 kB 00:00 2024-02-18T11:28:53.572 INFO:teuthology.orchestra.run.smithi188.stdout:(89/176): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 2.0 MB/s | 87 kB 00:00 2024-02-18T11:28:53.575 INFO:teuthology.orchestra.run.smithi012.stdout:(129/176): python3-jaraco-functools-3.5.0-2.el9 466 kB/s | 19 kB 00:00 2024-02-18T11:28:53.597 INFO:teuthology.orchestra.run.smithi188.stdout:(90/176): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2024-02-18T11:28:53.617 INFO:teuthology.orchestra.run.smithi012.stdout:(130/176): python3-grpcio-1.46.7-10.el9.x86_64. 11 MB/s | 2.0 MB 00:00 2024-02-18T11:28:53.622 INFO:teuthology.orchestra.run.smithi188.stdout:(91/176): zlib-devel-1.2.11-40.el9.x86_64.rpm 882 kB/s | 44 kB 00:00 2024-02-18T11:28:53.643 INFO:teuthology.orchestra.run.smithi012.stdout:(131/176): python3-jaraco-text-3.2.0-6.el9.noar 213 kB/s | 20 kB 00:00 2024-02-18T11:28:53.647 INFO:teuthology.orchestra.run.smithi188.stdout:(92/176): fuse-2.9.9-15.el9.x86_64.rpm 1.5 MB/s | 79 kB 00:00 2024-02-18T11:28:53.673 INFO:teuthology.orchestra.run.smithi188.stdout:(93/176): jq-1.6-16.el9.x86_64.rpm 3.6 MB/s | 186 kB 00:00 2024-02-18T11:28:53.676 INFO:teuthology.orchestra.run.smithi012.stdout:(132/176): python3-logutils-0.3.5-21.el9.noarch 786 kB/s | 46 kB 00:00 2024-02-18T11:28:53.706 INFO:teuthology.orchestra.run.smithi188.stdout:(94/176): libgfortran-11.4.1-3.el9.x86_64.rpm 13 MB/s | 802 kB 00:00 2024-02-18T11:28:53.718 INFO:teuthology.orchestra.run.smithi012.stdout:(133/176): python3-kubernetes-26.1.0-2.el9.noar 7.2 MB/s | 1.0 MB 00:00 2024-02-18T11:28:53.725 INFO:teuthology.orchestra.run.smithi158.stdout:Transaction test succeeded. 2024-02-18T11:28:53.725 INFO:teuthology.orchestra.run.smithi158.stdout:Running transaction 2024-02-18T11:28:53.731 INFO:teuthology.orchestra.run.smithi188.stdout:(95/176): libquadmath-11.4.1-3.el9.x86_64.rpm 3.3 MB/s | 194 kB 00:00 2024-02-18T11:28:53.765 INFO:teuthology.orchestra.run.smithi188.stdout:(96/176): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.6 MB/s | 217 kB 00:00 2024-02-18T11:28:53.790 INFO:teuthology.orchestra.run.smithi188.stdout:(97/176): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.0 MB/s | 241 kB 00:00 2024-02-18T11:28:53.802 INFO:teuthology.orchestra.run.smithi012.stdout:(134/176): python3-natsort-7.1.1-5.el9.noarch.r 461 kB/s | 58 kB 00:00 2024-02-18T11:28:53.827 INFO:teuthology.orchestra.run.smithi012.stdout:(135/176): python3-more-itertools-8.12.0-2.el9. 428 kB/s | 79 kB 00:00 2024-02-18T11:28:53.853 INFO:teuthology.orchestra.run.smithi012.stdout:(136/176): python3-portend-3.1.0-2.el9.noarch.r 327 kB/s | 16 kB 00:00 2024-02-18T11:28:53.878 INFO:teuthology.orchestra.run.smithi012.stdout:(137/176): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-02-18T11:28:53.903 INFO:teuthology.orchestra.run.smithi012.stdout:(138/176): python3-pyperclip-1.8.2-2.el9.noarch 562 kB/s | 28 kB 00:00 2024-02-18T11:28:53.916 INFO:teuthology.orchestra.run.smithi188.stdout:(98/176): python3-cryptography-36.0.1-4.el9.x86 7.6 MB/s | 1.1 MB 00:00 2024-02-18T11:28:53.928 INFO:teuthology.orchestra.run.smithi012.stdout:(139/176): python3-repoze-lru-0.7-16.el9.noarch 616 kB/s | 31 kB 00:00 2024-02-18T11:28:53.941 INFO:teuthology.orchestra.run.smithi188.stdout:(99/176): python3-ply-3.11-14.el9.noarch.rpm 686 kB/s | 103 kB 00:00 2024-02-18T11:28:53.954 INFO:teuthology.orchestra.run.smithi012.stdout:(140/176): python3-pecan-1.4.2-3.el9.noarch.rpm 1.1 MB/s | 272 kB 00:00 2024-02-18T11:28:53.966 INFO:teuthology.orchestra.run.smithi188.stdout:(100/176): python3-pycparser-2.20-6.el9.noarch. 2.4 MB/s | 124 kB 00:00 2024-02-18T11:28:53.979 INFO:teuthology.orchestra.run.smithi012.stdout:(141/176): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-02-18T11:28:53.991 INFO:teuthology.orchestra.run.smithi188.stdout:(101/176): python3-requests-2.25.1-8.el9.noarch 2.2 MB/s | 113 kB 00:00 2024-02-18T11:28:54.004 INFO:teuthology.orchestra.run.smithi012.stdout:(142/176): python3-tempora-5.0.0-2.el9.noarch.r 712 kB/s | 36 kB 00:00 2024-02-18T11:28:54.020 INFO:teuthology.orchestra.run.smithi188.stdout:(102/176): python3-urllib3-1.26.5-5.el9.noarch. 3.7 MB/s | 187 kB 00:00 2024-02-18T11:28:54.030 INFO:teuthology.orchestra.run.smithi012.stdout:(143/176): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-02-18T11:28:54.055 INFO:teuthology.orchestra.run.smithi012.stdout:(144/176): python3-typing-extensions-4.4.0-2.el 675 kB/s | 51 kB 00:00 2024-02-18T11:28:54.072 INFO:teuthology.orchestra.run.smithi012.stdout:(145/176): python3-websocket-client-1.2.3-2.el9 2.1 MB/s | 90 kB 00:00 2024-02-18T11:28:54.092 INFO:teuthology.orchestra.run.smithi188.stdout:(103/176): lua-devel-5.4.4-4.el9.x86_64.rpm 298 kB/s | 22 kB 00:00 2024-02-18T11:28:54.097 INFO:teuthology.orchestra.run.smithi012.stdout:(146/176): python3-webob-1.8.7-6.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-02-18T11:28:54.122 INFO:teuthology.orchestra.run.smithi012.stdout:(147/176): python3-zc-lockfile-2.0-10.el9.noarc 397 kB/s | 20 kB 00:00 2024-02-18T11:28:54.147 INFO:teuthology.orchestra.run.smithi012.stdout:(148/176): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-02-18T11:28:54.173 INFO:teuthology.orchestra.run.smithi012.stdout:(149/176): python3-werkzeug-2.0.3-3.el9.1.noarc 3.6 MB/s | 427 kB 00:00 2024-02-18T11:28:54.184 INFO:teuthology.orchestra.run.smithi188.stdout:(104/176): python3-devel-3.9.17-2.el9.x86_64.rp 1.2 MB/s | 245 kB 00:00 2024-02-18T11:28:54.231 INFO:teuthology.orchestra.run.smithi012.stdout:(150/176): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-02-18T11:28:54.259 INFO:teuthology.orchestra.run.smithi188.stdout:(105/176): protobuf-compiler-3.14.0-13.el9.x86_ 5.1 MB/s | 863 kB 00:00 2024-02-18T11:28:54.265 INFO:teuthology.orchestra.run.smithi012.stdout:(151/176): libacl-devel-2.3.1-4.el9.x86_64.rpm 2.0 MB/s | 68 kB 00:00 2024-02-18T11:28:54.332 INFO:teuthology.orchestra.run.smithi012.stdout:(152/176): NetworkManager-1.45.91-1.el9.x86_64. 34 MB/s | 2.3 MB 00:00 2024-02-18T11:28:54.360 INFO:teuthology.orchestra.run.smithi188.stdout:(106/176): abseil-cpp-20211102.0-3.el9.x86_64.r 3.1 MB/s | 548 kB 00:00 2024-02-18T11:28:54.382 INFO:teuthology.orchestra.run.smithi012.stdout:(153/176): NetworkManager-libnm-1.45.91-1.el9.x 36 MB/s | 1.8 MB 00:00 2024-02-18T11:28:54.385 INFO:teuthology.orchestra.run.smithi188.stdout:(107/176): gperftools-libs-2.9.1-2.el9.x86_64.r 2.4 MB/s | 309 kB 00:00 2024-02-18T11:28:54.408 INFO:teuthology.orchestra.run.smithi012.stdout:(154/176): NetworkManager-team-1.45.91-1.el9.x8 1.6 MB/s | 41 kB 00:00 2024-02-18T11:28:54.410 INFO:teuthology.orchestra.run.smithi188.stdout:(108/176): grpc-data-1.46.7-10.el9.noarch.rpm 393 kB/s | 19 kB 00:00 2024-02-18T11:28:54.441 INFO:teuthology.orchestra.run.smithi012.stdout:(155/176): NetworkManager-tui-1.45.91-1.el9.x86 7.2 MB/s | 246 kB 00:00 2024-02-18T11:28:54.467 INFO:teuthology.orchestra.run.smithi012.stdout:(156/176): acl-2.3.1-4.el9.x86_64.rpm 2.7 MB/s | 69 kB 00:00 2024-02-18T11:28:54.485 INFO:teuthology.orchestra.run.smithi188.stdout:(109/176): libarrow-doc-9.0.0-10.el9.noarch.rpm 353 kB/s | 26 kB 00:00 2024-02-18T11:28:54.492 INFO:teuthology.orchestra.run.smithi012.stdout:(157/176): elfutils-debuginfod-client-0.190-2.e 1.4 MB/s | 35 kB 00:00 2024-02-18T11:28:54.544 INFO:teuthology.orchestra.run.smithi188.stdout:(110/176): libarrow-9.0.0-10.el9.x86_64.rpm 28 MB/s | 4.4 MB 00:00 2024-02-18T11:28:54.550 INFO:teuthology.orchestra.run.smithi012.stdout:(158/176): librados2-19.0.0-1555.g1c920afa.el9. 8.4 MB/s | 3.4 MB 00:00 2024-02-18T11:28:54.570 INFO:teuthology.orchestra.run.smithi188.stdout:(111/176): liboath-2.6.7-2.el9.x86_64.rpm 591 kB/s | 49 kB 00:00 2024-02-18T11:28:54.576 INFO:teuthology.orchestra.run.smithi012.stdout:(159/176): elfutils-libelf-0.190-2.el9.x86_64.r 2.2 MB/s | 191 kB 00:00 2024-02-18T11:28:54.595 INFO:teuthology.orchestra.run.smithi188.stdout:(112/176): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-02-18T11:28:54.601 INFO:teuthology.orchestra.run.smithi012.stdout:(160/176): elfutils-libs-0.190-2.el9.x86_64.rpm 5.0 MB/s | 254 kB 00:00 2024-02-18T11:28:54.636 INFO:teuthology.orchestra.run.smithi012.stdout:(161/176): iproute-6.7.0-2.el9.x86_64.rpm 13 MB/s | 802 kB 00:00 2024-02-18T11:28:54.668 INFO:teuthology.orchestra.run.smithi012.stdout:(162/176): iproute-tc-6.7.0-2.el9.x86_64.rpm 6.5 MB/s | 433 kB 00:00 2024-02-18T11:28:54.687 INFO:teuthology.orchestra.run.smithi012.stdout:(163/176): libacl-2.3.1-4.el9.x86_64.rpm 446 kB/s | 22 kB 00:00 2024-02-18T11:28:54.710 INFO:teuthology.orchestra.run.smithi012.stdout:(164/176): libblkid-2.37.4-18.el9.x86_64.rpm 2.5 MB/s | 107 kB 00:00 2024-02-18T11:28:54.736 INFO:teuthology.orchestra.run.smithi012.stdout:(165/176): libfdisk-2.37.4-18.el9.x86_64.rpm 3.0 MB/s | 154 kB 00:00 2024-02-18T11:28:54.761 INFO:teuthology.orchestra.run.smithi012.stdout:(166/176): libmount-2.37.4-18.el9.x86_64.rpm 2.6 MB/s | 135 kB 00:00 2024-02-18T11:28:54.762 INFO:teuthology.orchestra.run.smithi188.stdout:(113/176): luarocks-3.9.2-1.el9.noarch.rpm 787 kB/s | 151 kB 00:00 2024-02-18T11:28:54.786 INFO:teuthology.orchestra.run.smithi012.stdout:(167/176): libsmartcols-2.37.4-18.el9.x86_64.rp 1.2 MB/s | 63 kB 00:00 2024-02-18T11:28:54.804 INFO:teuthology.orchestra.run.smithi188.stdout:(114/176): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.9 MB/s | 838 kB 00:00 2024-02-18T11:28:54.812 INFO:teuthology.orchestra.run.smithi012.stdout:(168/176): libuuid-2.37.4-18.el9.x86_64.rpm 552 kB/s | 28 kB 00:00 2024-02-18T11:28:54.837 INFO:teuthology.orchestra.run.smithi188.stdout:(115/176): python3-asyncssh-2.13.2-4.el9.noarch 7.3 MB/s | 548 kB 00:00 2024-02-18T11:28:54.845 INFO:teuthology.orchestra.run.smithi012.stdout:(169/176): openssl-3.0.7-25.el9.x86_64.rpm 20 MB/s | 1.2 MB 00:00 2024-02-18T11:28:54.863 INFO:teuthology.orchestra.run.smithi188.stdout:(116/176): python3-bcrypt-3.2.2-1.el9.x86_64.rp 755 kB/s | 43 kB 00:00 2024-02-18T11:28:54.882 INFO:teuthology.orchestra.run.smithi188.stdout:(117/176): python3-cachetools-4.2.4-1.el9.noarc 726 kB/s | 32 kB 00:00 2024-02-18T11:28:54.899 INFO:teuthology.orchestra.run.smithi188.stdout:(118/176): python3-certifi-2021.10.8-2.el9.noar 419 kB/s | 15 kB 00:00 2024-02-18T11:28:54.924 INFO:teuthology.orchestra.run.smithi188.stdout:(119/176): python3-cheroot-8.6.0-4.el9.noarch.r 4.0 MB/s | 172 kB 00:00 2024-02-18T11:28:54.950 INFO:teuthology.orchestra.run.smithi188.stdout:(120/176): python3-cherrypy-18.6.1-2.el9.noarch 7.0 MB/s | 358 kB 00:00 2024-02-18T11:28:54.983 INFO:teuthology.orchestra.run.smithi188.stdout:(121/176): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-02-18T11:28:55.009 INFO:teuthology.orchestra.run.smithi188.stdout:(122/176): python3-colorama-0.4.4-12.el9.noarch 609 kB/s | 35 kB 00:00 2024-02-18T11:28:55.034 INFO:teuthology.orchestra.run.smithi188.stdout:(123/176): python3-google-auth-2.27.0-1.el9.noa 3.9 MB/s | 203 kB 00:00 2024-02-18T11:28:55.079 INFO:teuthology.orchestra.run.smithi012.stdout:(170/176): openssl-libs-3.0.7-25.el9.x86_64.rpm 8.1 MB/s | 2.1 MB 00:00 2024-02-18T11:28:55.129 INFO:teuthology.orchestra.run.smithi012.stdout:(171/176): util-linux-2.37.4-18.el9.x86_64.rpm 7.8 MB/s | 2.2 MB 00:00 2024-02-18T11:28:55.154 INFO:teuthology.orchestra.run.smithi012.stdout:(172/176): util-linux-core-2.37.4-18.el9.x86_64 5.9 MB/s | 438 kB 00:00 2024-02-18T11:28:55.180 INFO:teuthology.orchestra.run.smithi012.stdout:(173/176): util-linux-user-2.37.4-18.el9.x86_64 610 kB/s | 30 kB 00:00 2024-02-18T11:28:55.255 INFO:teuthology.orchestra.run.smithi012.stdout:(174/176): librbd1-19.0.0-1555.g1c920afa.el9.x8 2.9 MB/s | 3.1 MB 00:01 2024-02-18T11:28:55.280 INFO:teuthology.orchestra.run.smithi012.stdout:(175/176): xfsdump-3.1.12-4.el9.x86_64.rpm 2.5 MB/s | 321 kB 00:00 2024-02-18T11:28:55.372 INFO:teuthology.orchestra.run.smithi012.stdout:(176/176): xfsprogs-6.3.0-1.el9.x86_64.rpm 5.5 MB/s | 1.1 MB 00:00 2024-02-18T11:28:55.376 INFO:teuthology.orchestra.run.smithi012.stdout:-------------------------------------------------------------------------------- 2024-02-18T11:28:55.376 INFO:teuthology.orchestra.run.smithi012.stdout:Total 13 MB/s | 223 MB 00:16 2024-02-18T11:28:55.424 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction check 2024-02-18T11:28:55.468 INFO:teuthology.orchestra.run.smithi188.stdout:(124/176): ceph-test-19.0.0-1555.g1c920afa.el9. 5.4 MB/s | 47 MB 00:08 2024-02-18T11:28:55.518 INFO:teuthology.orchestra.run.smithi188.stdout:(125/176): python3-grpcio-1.46.7-10.el9.x86_64. 4.0 MB/s | 2.0 MB 00:00 2024-02-18T11:28:55.543 INFO:teuthology.orchestra.run.smithi188.stdout:(126/176): python3-grpcio-tools-1.46.7-10.el9.x 285 kB/s | 144 kB 00:00 2024-02-18T11:28:55.568 INFO:teuthology.orchestra.run.smithi188.stdout:(127/176): python3-jaraco-8.2.1-3.el9.noarch.rp 109 kB/s | 11 kB 00:00 2024-02-18T11:28:55.594 INFO:teuthology.orchestra.run.smithi188.stdout:(128/176): python3-jaraco-classes-3.2.1-5.el9.n 353 kB/s | 18 kB 00:00 2024-02-18T11:28:55.619 INFO:teuthology.orchestra.run.smithi188.stdout:(129/176): python3-jaraco-collections-3.0.0-8.e 309 kB/s | 23 kB 00:00 2024-02-18T11:28:55.644 INFO:teuthology.orchestra.run.smithi188.stdout:(130/176): python3-jaraco-functools-3.5.0-2.el9 259 kB/s | 19 kB 00:00 2024-02-18T11:28:55.669 INFO:teuthology.orchestra.run.smithi188.stdout:(131/176): python3-jaraco-text-3.2.0-6.el9.noar 393 kB/s | 20 kB 00:00 2024-02-18T11:28:55.711 INFO:teuthology.orchestra.run.smithi188.stdout:(132/176): python3-kubernetes-26.1.0-2.el9.noar 11 MB/s | 1.0 MB 00:00 2024-02-18T11:28:55.736 INFO:teuthology.orchestra.run.smithi188.stdout:(133/176): python3-logutils-0.3.5-21.el9.noarch 509 kB/s | 46 kB 00:00 2024-02-18T11:28:55.748 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction check succeeded. 2024-02-18T11:28:55.748 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction test 2024-02-18T11:28:55.762 INFO:teuthology.orchestra.run.smithi188.stdout:(134/176): python3-more-itertools-8.12.0-2.el9. 865 kB/s | 79 kB 00:00 2024-02-18T11:28:55.787 INFO:teuthology.orchestra.run.smithi188.stdout:(135/176): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-02-18T11:28:55.812 INFO:teuthology.orchestra.run.smithi188.stdout:(136/176): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-02-18T11:28:55.837 INFO:teuthology.orchestra.run.smithi188.stdout:(137/176): python3-portend-3.1.0-2.el9.noarch.r 220 kB/s | 16 kB 00:00 2024-02-18T11:28:55.863 INFO:teuthology.orchestra.run.smithi188.stdout:(138/176): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-02-18T11:28:55.906 INFO:teuthology.orchestra.run.smithi188.stdout:(139/176): python3-pyperclip-1.8.2-2.el9.noarch 304 kB/s | 28 kB 00:00 2024-02-18T11:28:55.930 INFO:teuthology.orchestra.run.smithi188.stdout:(140/176): python3-repoze-lru-0.7-16.el9.noarch 335 kB/s | 31 kB 00:00 2024-02-18T11:28:55.955 INFO:teuthology.orchestra.run.smithi188.stdout:(141/176): python3-routes-2.5.1-5.el9.noarch.rp 3.8 MB/s | 188 kB 00:00 2024-02-18T11:28:55.980 INFO:teuthology.orchestra.run.smithi188.stdout:(142/176): python3-rsa-4.9-2.el9.noarch.rpm 799 kB/s | 59 kB 00:00 2024-02-18T11:28:56.006 INFO:teuthology.orchestra.run.smithi188.stdout:(143/176): python3-tempora-5.0.0-2.el9.noarch.r 479 kB/s | 36 kB 00:00 2024-02-18T11:28:56.031 INFO:teuthology.orchestra.run.smithi188.stdout:(144/176): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-02-18T11:28:56.056 INFO:teuthology.orchestra.run.smithi188.stdout:(145/176): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-02-18T11:28:56.090 INFO:teuthology.orchestra.run.smithi188.stdout:(146/176): python3-websocket-client-1.2.3-2.el9 1.0 MB/s | 90 kB 00:00 2024-02-18T11:28:56.124 INFO:teuthology.orchestra.run.smithi188.stdout:(147/176): python3-werkzeug-2.0.3-3.el9.1.noarc 6.3 MB/s | 427 kB 00:00 2024-02-18T11:28:56.149 INFO:teuthology.orchestra.run.smithi188.stdout:(148/176): python3-zc-lockfile-2.0-10.el9.noarc 217 kB/s | 20 kB 00:00 2024-02-18T11:28:56.174 INFO:teuthology.orchestra.run.smithi188.stdout:(149/176): re2-20211101-3.el9.x86_64.rpm 2.3 MB/s | 192 kB 00:00 2024-02-18T11:28:56.233 INFO:teuthology.orchestra.run.smithi188.stdout:(150/176): thrift-0.15.0-2.el9.x86_64.rpm 19 MB/s | 1.6 MB 00:00 2024-02-18T11:28:56.267 INFO:teuthology.orchestra.run.smithi188.stdout:(151/176): libacl-devel-2.3.1-4.el9.x86_64.rpm 2.0 MB/s | 68 kB 00:00 2024-02-18T11:28:56.425 INFO:teuthology.orchestra.run.smithi188.stdout:(152/176): NetworkManager-1.45.91-1.el9.x86_64. 14 MB/s | 2.3 MB 00:00 2024-02-18T11:28:56.509 INFO:teuthology.orchestra.run.smithi188.stdout:(153/176): NetworkManager-libnm-1.45.91-1.el9.x 22 MB/s | 1.8 MB 00:00 2024-02-18T11:28:56.534 INFO:teuthology.orchestra.run.smithi188.stdout:(154/176): NetworkManager-team-1.45.91-1.el9.x8 1.6 MB/s | 41 kB 00:00 2024-02-18T11:28:56.567 INFO:teuthology.orchestra.run.smithi188.stdout:(155/176): NetworkManager-tui-1.45.91-1.el9.x86 7.2 MB/s | 246 kB 00:00 2024-02-18T11:28:56.601 INFO:teuthology.orchestra.run.smithi188.stdout:(156/176): acl-2.3.1-4.el9.x86_64.rpm 2.0 MB/s | 69 kB 00:00 2024-02-18T11:28:56.627 INFO:teuthology.orchestra.run.smithi188.stdout:(157/176): elfutils-debuginfod-client-0.190-2.e 1.4 MB/s | 35 kB 00:00 2024-02-18T11:28:56.730 INFO:teuthology.orchestra.run.smithi188.stdout:(158/176): elfutils-libelf-0.190-2.el9.x86_64.r 1.8 MB/s | 191 kB 00:00 2024-02-18T11:28:56.794 INFO:teuthology.orchestra.run.smithi188.stdout:(159/176): elfutils-libs-0.190-2.el9.x86_64.rpm 3.9 MB/s | 254 kB 00:00 2024-02-18T11:28:56.844 INFO:teuthology.orchestra.run.smithi188.stdout:(160/176): librados2-19.0.0-1555.g1c920afa.el9. 4.9 MB/s | 3.4 MB 00:00 2024-02-18T11:28:56.886 INFO:teuthology.orchestra.run.smithi188.stdout:(161/176): iproute-6.7.0-2.el9.x86_64.rpm 8.5 MB/s | 802 kB 00:00 2024-02-18T11:28:56.920 INFO:teuthology.orchestra.run.smithi188.stdout:(162/176): iproute-tc-6.7.0-2.el9.x86_64.rpm 5.7 MB/s | 433 kB 00:00 2024-02-18T11:28:56.945 INFO:teuthology.orchestra.run.smithi188.stdout:(163/176): libacl-2.3.1-4.el9.x86_64.rpm 381 kB/s | 22 kB 00:00 2024-02-18T11:28:56.970 INFO:teuthology.orchestra.run.smithi188.stdout:(164/176): libblkid-2.37.4-18.el9.x86_64.rpm 2.1 MB/s | 107 kB 00:00 2024-02-18T11:28:56.996 INFO:teuthology.orchestra.run.smithi188.stdout:(165/176): libfdisk-2.37.4-18.el9.x86_64.rpm 3.0 MB/s | 154 kB 00:00 2024-02-18T11:28:57.021 INFO:teuthology.orchestra.run.smithi188.stdout:(166/176): libmount-2.37.4-18.el9.x86_64.rpm 2.6 MB/s | 135 kB 00:00 2024-02-18T11:28:57.046 INFO:teuthology.orchestra.run.smithi188.stdout:(167/176): libsmartcols-2.37.4-18.el9.x86_64.rp 1.2 MB/s | 63 kB 00:00 2024-02-18T11:28:57.071 INFO:teuthology.orchestra.run.smithi188.stdout:(168/176): libuuid-2.37.4-18.el9.x86_64.rpm 551 kB/s | 28 kB 00:00 2024-02-18T11:28:57.105 INFO:teuthology.orchestra.run.smithi188.stdout:(169/176): openssl-3.0.7-25.el9.x86_64.rpm 20 MB/s | 1.2 MB 00:00 2024-02-18T11:28:57.164 INFO:teuthology.orchestra.run.smithi188.stdout:(170/176): librbd1-19.0.0-1555.g1c920afa.el9.x8 3.2 MB/s | 3.1 MB 00:00 2024-02-18T11:28:57.214 INFO:teuthology.orchestra.run.smithi188.stdout:(171/176): openssl-libs-3.0.7-25.el9.x86_64.rpm 15 MB/s | 2.1 MB 00:00 2024-02-18T11:28:57.265 INFO:teuthology.orchestra.run.smithi188.stdout:(172/176): util-linux-2.37.4-18.el9.x86_64.rpm 14 MB/s | 2.2 MB 00:00 2024-02-18T11:28:57.290 INFO:teuthology.orchestra.run.smithi188.stdout:(173/176): util-linux-core-2.37.4-18.el9.x86_64 3.5 MB/s | 438 kB 00:00 2024-02-18T11:28:57.315 INFO:teuthology.orchestra.run.smithi188.stdout:(174/176): util-linux-user-2.37.4-18.el9.x86_64 304 kB/s | 30 kB 00:00 2024-02-18T11:28:57.340 INFO:teuthology.orchestra.run.smithi188.stdout:(175/176): xfsdump-3.1.12-4.el9.x86_64.rpm 6.3 MB/s | 321 kB 00:00 2024-02-18T11:28:57.374 INFO:teuthology.orchestra.run.smithi188.stdout:(176/176): xfsprogs-6.3.0-1.el9.x86_64.rpm 13 MB/s | 1.1 MB 00:00 2024-02-18T11:28:57.380 INFO:teuthology.orchestra.run.smithi188.stdout:-------------------------------------------------------------------------------- 2024-02-18T11:28:57.380 INFO:teuthology.orchestra.run.smithi188.stdout:Total 12 MB/s | 223 MB 00:18 2024-02-18T11:28:57.422 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction check 2024-02-18T11:28:57.697 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction check succeeded. 2024-02-18T11:28:57.697 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction test 2024-02-18T11:28:57.785 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction test succeeded. 2024-02-18T11:28:57.786 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction 2024-02-18T11:28:59.358 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction test succeeded. 2024-02-18T11:28:59.359 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction 2024-02-18T11:29:00.826 INFO:teuthology.orchestra.run.smithi158.stdout: Preparing : 1/1 2024-02-18T11:29:00.928 INFO:teuthology.orchestra.run.smithi158.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 1/202 2024-02-18T11:29:00.962 INFO:teuthology.orchestra.run.smithi158.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/202 2024-02-18T11:29:00.993 INFO:teuthology.orchestra.run.smithi158.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/202 2024-02-18T11:29:01.019 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/202 2024-02-18T11:29:01.054 INFO:teuthology.orchestra.run.smithi158.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 4/202 2024-02-18T11:29:01.092 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 5/202 2024-02-18T11:29:01.147 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/202 2024-02-18T11:29:01.180 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : thrift-0.15.0-2.el9.x86_64 7/202 2024-02-18T11:29:01.206 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 8/202 2024-02-18T11:29:01.248 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 9/202 2024-02-18T11:29:01.567 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 10/202 2024-02-18T11:29:01.632 INFO:teuthology.orchestra.run.smithi158.stdout: Upgrading : librados2-2:19.0.0-1555.g1c920afa.el9.x86_64 11/202 2024-02-18T11:29:01.754 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: librados2-2:19.0.0-1555.g1c920afa.el9.x86_64 11/202 2024-02-18T11:29:01.791 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : libcephfs2-2:19.0.0-1555.g1c920afa.el9.x86_64 12/202 2024-02-18T11:29:01.882 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: libcephfs2-2:19.0.0-1555.g1c920afa.el9.x86_64 12/202 2024-02-18T11:29:01.937 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-rados-2:19.0.0-1555.g1c920afa.el9.x86_64 13/202 2024-02-18T11:29:02.089 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 14/202 2024-02-18T11:29:02.142 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/202 2024-02-18T11:29:02.162 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : libcephsqlite-2:19.0.0-1555.g1c920afa.el9.x86_64 16/202 2024-02-18T11:29:02.265 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1555.g1c920afa.el9.x86_64 16/202 2024-02-18T11:29:02.288 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : libradosstriper1-2:19.0.0-1555.g1c920afa.el9.x86 17/202 2024-02-18T11:29:02.352 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1555.g1c920afa.el9.x86 17/202 2024-02-18T11:29:02.391 INFO:teuthology.orchestra.run.smithi158.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/202 2024-02-18T11:29:02.423 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/202 2024-02-18T11:29:02.438 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/202 2024-02-18T11:29:02.615 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/202 2024-02-18T11:29:02.645 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/202 2024-02-18T11:29:02.677 INFO:teuthology.orchestra.run.smithi158.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/202 2024-02-18T11:29:02.774 INFO:teuthology.orchestra.run.smithi158.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/202 2024-02-18T11:29:02.879 INFO:teuthology.orchestra.run.smithi158.stdout: Upgrading : NetworkManager-libnm-1:1.45.91-1.el9.x86_64 23/202 2024-02-18T11:29:02.939 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.91-1.el9.x86_64 23/202 2024-02-18T11:29:03.013 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: NetworkManager-1:1.45.91-1.el9.x86_64 24/202 2024-02-18T11:29:03.066 INFO:teuthology.orchestra.run.smithi158.stdout: Upgrading : NetworkManager-1:1.45.91-1.el9.x86_64 24/202 2024-02-18T11:29:03.286 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: NetworkManager-1:1.45.91-1.el9.x86_64 24/202 2024-02-18T11:29:03.405 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/202 2024-02-18T11:29:03.510 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/202 2024-02-18T11:29:03.554 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/202 2024-02-18T11:29:03.641 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/202 2024-02-18T11:29:03.694 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/202 2024-02-18T11:29:03.724 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/202 2024-02-18T11:29:03.760 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/202 2024-02-18T11:29:03.800 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/202 2024-02-18T11:29:03.873 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/202 2024-02-18T11:29:03.907 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/202 2024-02-18T11:29:03.951 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/202 2024-02-18T11:29:03.983 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/202 2024-02-18T11:29:04.011 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/202 2024-02-18T11:29:04.082 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/202 2024-02-18T11:29:04.116 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-ceph-common-2:19.0.0-1555.g1c920afa.el9. 39/202 2024-02-18T11:29:04.152 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-ceph-argparse-2:19.0.0-1555.g1c920afa.el 40/202 2024-02-18T11:29:04.180 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-cephfs-2:19.0.0-1555.g1c920afa.el9.x86_6 41/202 2024-02-18T11:29:04.248 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/202 2024-02-18T11:29:04.294 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/202 2024-02-18T11:29:04.369 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/202 2024-02-18T11:29:04.501 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/202 2024-02-18T11:29:04.555 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/202 2024-02-18T11:29:05.013 INFO:teuthology.orchestra.run.smithi012.stdout: Preparing : 1/1 2024-02-18T11:29:05.089 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/202 2024-02-18T11:29:05.120 INFO:teuthology.orchestra.run.smithi012.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 1/202 2024-02-18T11:29:05.159 INFO:teuthology.orchestra.run.smithi012.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/202 2024-02-18T11:29:05.167 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/202 2024-02-18T11:29:05.207 INFO:teuthology.orchestra.run.smithi012.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/202 2024-02-18T11:29:05.228 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/202 2024-02-18T11:29:05.250 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/202 2024-02-18T11:29:05.272 INFO:teuthology.orchestra.run.smithi012.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 4/202 2024-02-18T11:29:05.295 INFO:teuthology.orchestra.run.smithi158.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/202 2024-02-18T11:29:05.303 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/202 2024-02-18T11:29:05.308 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 5/202 2024-02-18T11:29:05.342 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/202 2024-02-18T11:29:05.363 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/202 2024-02-18T11:29:05.380 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : librados-devel-2:19.0.0-1555.g1c920afa.el9.x86_6 52/202 2024-02-18T11:29:05.411 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 53/202 2024-02-18T11:29:05.412 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : thrift-0.15.0-2.el9.x86_64 7/202 2024-02-18T11:29:05.437 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 8/202 2024-02-18T11:29:05.490 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 9/202 2024-02-18T11:29:05.510 INFO:teuthology.orchestra.run.smithi158.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 54/202 2024-02-18T11:29:05.562 INFO:teuthology.orchestra.run.smithi158.stdout: Upgrading : iproute-6.7.0-2.el9.x86_64 55/202 2024-02-18T11:29:05.821 INFO:teuthology.orchestra.run.smithi158.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 56/202 2024-02-18T11:29:05.840 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 10/202 2024-02-18T11:29:05.930 INFO:teuthology.orchestra.run.smithi012.stdout: Upgrading : librados2-2:19.0.0-1555.g1c920afa.el9.x86_64 11/202 2024-02-18T11:29:05.991 INFO:teuthology.orchestra.run.smithi158.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 57/202 2024-02-18T11:29:06.037 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: librados2-2:19.0.0-1555.g1c920afa.el9.x86_64 11/202 2024-02-18T11:29:06.148 INFO:teuthology.orchestra.run.smithi158.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 58/202 2024-02-18T11:29:06.181 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libcephfs2-2:19.0.0-1555.g1c920afa.el9.x86_64 12/202 2024-02-18T11:29:06.257 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: libcephfs2-2:19.0.0-1555.g1c920afa.el9.x86_64 12/202 2024-02-18T11:29:06.289 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-rados-2:19.0.0-1555.g1c920afa.el9.x86_64 13/202 2024-02-18T11:29:06.329 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 59/202 2024-02-18T11:29:06.345 INFO:teuthology.orchestra.run.smithi188.stdout: Preparing : 1/1 2024-02-18T11:29:06.455 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 14/202 2024-02-18T11:29:06.460 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 1/202 2024-02-18T11:29:06.470 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 60/202 2024-02-18T11:29:06.496 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/202 2024-02-18T11:29:06.511 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/202 2024-02-18T11:29:06.520 INFO:teuthology.orchestra.run.smithi158.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 61/202 2024-02-18T11:29:06.534 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/202 2024-02-18T11:29:06.538 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libcephsqlite-2:19.0.0-1555.g1c920afa.el9.x86_64 16/202 2024-02-18T11:29:06.553 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/202 2024-02-18T11:29:06.576 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 62/202 2024-02-18T11:29:06.620 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 4/202 2024-02-18T11:29:06.643 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1555.g1c920afa.el9.x86_64 16/202 2024-02-18T11:29:06.649 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 63/202 2024-02-18T11:29:06.658 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 5/202 2024-02-18T11:29:06.672 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libradosstriper1-2:19.0.0-1555.g1c920afa.el9.x86 17/202 2024-02-18T11:29:06.679 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 64/202 2024-02-18T11:29:06.713 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/202 2024-02-18T11:29:06.716 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 65/202 2024-02-18T11:29:06.729 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1555.g1c920afa.el9.x86 17/202 2024-02-18T11:29:06.746 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : thrift-0.15.0-2.el9.x86_64 7/202 2024-02-18T11:29:06.763 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 66/202 2024-02-18T11:29:06.767 INFO:teuthology.orchestra.run.smithi012.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/202 2024-02-18T11:29:06.780 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 8/202 2024-02-18T11:29:06.790 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/202 2024-02-18T11:29:06.798 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 67/202 2024-02-18T11:29:06.805 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/202 2024-02-18T11:29:06.831 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 9/202 2024-02-18T11:29:06.842 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 68/202 2024-02-18T11:29:06.884 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 69/202 2024-02-18T11:29:07.019 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 70/202 2024-02-18T11:29:07.037 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/202 2024-02-18T11:29:07.064 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 71/202 2024-02-18T11:29:07.075 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/202 2024-02-18T11:29:07.094 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 72/202 2024-02-18T11:29:07.113 INFO:teuthology.orchestra.run.smithi012.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/202 2024-02-18T11:29:07.139 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 73/202 2024-02-18T11:29:07.157 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 10/202 2024-02-18T11:29:07.233 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 74/202 2024-02-18T11:29:07.238 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : librados2-2:19.0.0-1555.g1c920afa.el9.x86_64 11/202 2024-02-18T11:29:07.280 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-google-auth-1:2.27.0-1.el9.noarch 75/202 2024-02-18T11:29:07.320 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 76/202 2024-02-18T11:29:07.345 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 77/202 2024-02-18T11:29:07.353 INFO:teuthology.orchestra.run.smithi012.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/202 2024-02-18T11:29:07.360 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librados2-2:19.0.0-1555.g1c920afa.el9.x86_64 11/202 2024-02-18T11:29:07.371 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 78/202 2024-02-18T11:29:07.391 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libcephfs2-2:19.0.0-1555.g1c920afa.el9.x86_64 12/202 2024-02-18T11:29:07.443 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 79/202 2024-02-18T11:29:07.489 INFO:teuthology.orchestra.run.smithi012.stdout: Upgrading : NetworkManager-libnm-1:1.45.91-1.el9.x86_64 23/202 2024-02-18T11:29:07.548 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: libcephfs2-2:19.0.0-1555.g1c920afa.el9.x86_64 12/202 2024-02-18T11:29:07.549 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.91-1.el9.x86_64 23/202 2024-02-18T11:29:07.563 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 80/202 2024-02-18T11:29:07.578 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-rados-2:19.0.0-1555.g1c920afa.el9.x86_64 13/202 2024-02-18T11:29:07.670 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 81/202 2024-02-18T11:29:07.670 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: NetworkManager-1:1.45.91-1.el9.x86_64 24/202 2024-02-18T11:29:07.720 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 82/202 2024-02-18T11:29:07.729 INFO:teuthology.orchestra.run.smithi012.stdout: Upgrading : NetworkManager-1:1.45.91-1.el9.x86_64 24/202 2024-02-18T11:29:07.734 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 14/202 2024-02-18T11:29:07.769 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-ply-3.11-14.el9.noarch 83/202 2024-02-18T11:29:07.784 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/202 2024-02-18T11:29:07.812 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libcephsqlite-2:19.0.0-1555.g1c920afa.el9.x86_64 16/202 2024-02-18T11:29:07.825 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 84/202 2024-02-18T11:29:07.914 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1555.g1c920afa.el9.x86_64 16/202 2024-02-18T11:29:07.939 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libradosstriper1-2:19.0.0-1555.g1c920afa.el9.x86 17/202 2024-02-18T11:29:07.948 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: NetworkManager-1:1.45.91-1.el9.x86_64 24/202 2024-02-18T11:29:08.002 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1555.g1c920afa.el9.x86 17/202 2024-02-18T11:29:08.008 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 85/202 2024-02-18T11:29:08.042 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/202 2024-02-18T11:29:08.063 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/202 2024-02-18T11:29:08.065 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/202 2024-02-18T11:29:08.066 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 86/202 2024-02-18T11:29:08.080 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/202 2024-02-18T11:29:08.216 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 87/202 2024-02-18T11:29:08.256 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 88/202 2024-02-18T11:29:08.267 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/202 2024-02-18T11:29:08.273 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/202 2024-02-18T11:29:08.289 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 89/202 2024-02-18T11:29:08.301 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/202 2024-02-18T11:29:08.320 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 90/202 2024-02-18T11:29:08.321 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/202 2024-02-18T11:29:08.352 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/202 2024-02-18T11:29:08.356 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : jq-1.6-16.el9.x86_64 91/202 2024-02-18T11:29:08.372 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/202 2024-02-18T11:29:08.386 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : fuse-2.9.9-15.el9.x86_64 92/202 2024-02-18T11:29:08.415 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 93/202 2024-02-18T11:29:08.433 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/202 2024-02-18T11:29:08.449 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/202 2024-02-18T11:29:08.449 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 94/202 2024-02-18T11:29:08.462 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/202 2024-02-18T11:29:08.488 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : uuid-1.6.2-55.el9.x86_64 95/202 2024-02-18T11:29:08.498 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/202 2024-02-18T11:29:08.520 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 96/202 2024-02-18T11:29:08.547 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : NetworkManager-libnm-1:1.45.91-1.el9.x86_64 23/202 2024-02-18T11:29:08.548 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/202 2024-02-18T11:29:08.550 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 97/202 2024-02-18T11:29:08.590 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.91-1.el9.x86_64 23/202 2024-02-18T11:29:08.621 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/202 2024-02-18T11:29:08.633 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 98/202 2024-02-18T11:29:08.663 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: NetworkManager-1:1.45.91-1.el9.x86_64 24/202 2024-02-18T11:29:08.669 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 99/202 2024-02-18T11:29:08.671 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/202 2024-02-18T11:29:08.702 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 100/202 2024-02-18T11:29:08.708 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : NetworkManager-1:1.45.91-1.el9.x86_64 24/202 2024-02-18T11:29:08.714 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/202 2024-02-18T11:29:08.743 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 101/202 2024-02-18T11:29:08.747 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/202 2024-02-18T11:29:08.777 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/202 2024-02-18T11:29:08.778 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 102/202 2024-02-18T11:29:08.848 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/202 2024-02-18T11:29:08.885 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 103/202 2024-02-18T11:29:08.890 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-ceph-common-2:19.0.0-1555.g1c920afa.el9. 39/202 2024-02-18T11:29:08.920 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: NetworkManager-1:1.45.91-1.el9.x86_64 24/202 2024-02-18T11:29:08.933 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-ceph-argparse-2:19.0.0-1555.g1c920afa.el 40/202 2024-02-18T11:29:08.962 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-cephfs-2:19.0.0-1555.g1c920afa.el9.x86_6 41/202 2024-02-18T11:29:08.974 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 104/202 2024-02-18T11:29:09.026 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 105/202 2024-02-18T11:29:09.035 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/202 2024-02-18T11:29:09.037 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/202 2024-02-18T11:29:09.057 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 106/202 2024-02-18T11:29:09.076 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/202 2024-02-18T11:29:09.105 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 107/202 2024-02-18T11:29:09.134 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/202 2024-02-18T11:29:09.153 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/202 2024-02-18T11:29:09.172 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/202 2024-02-18T11:29:09.202 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-packaging-20.9-5.el9.noarch 108/202 2024-02-18T11:29:09.254 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/202 2024-02-18T11:29:09.259 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 109/202 2024-02-18T11:29:09.289 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/202 2024-02-18T11:29:09.303 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/202 2024-02-18T11:29:09.338 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/202 2024-02-18T11:29:09.342 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/202 2024-02-18T11:29:09.378 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/202 2024-02-18T11:29:09.418 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/202 2024-02-18T11:29:09.482 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/202 2024-02-18T11:29:09.525 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/202 2024-02-18T11:29:09.569 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/202 2024-02-18T11:29:09.601 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/202 2024-02-18T11:29:09.629 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/202 2024-02-18T11:29:09.700 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/202 2024-02-18T11:29:09.734 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-ceph-common-2:19.0.0-1555.g1c920afa.el9. 39/202 2024-02-18T11:29:09.778 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-ceph-argparse-2:19.0.0-1555.g1c920afa.el 40/202 2024-02-18T11:29:09.815 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cephfs-2:19.0.0-1555.g1c920afa.el9.x86_6 41/202 2024-02-18T11:29:09.883 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/202 2024-02-18T11:29:09.889 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/202 2024-02-18T11:29:09.938 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/202 2024-02-18T11:29:09.976 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/202 2024-02-18T11:29:09.988 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 110/202 2024-02-18T11:29:10.011 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/202 2024-02-18T11:29:10.039 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 111/202 2024-02-18T11:29:10.058 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/202 2024-02-18T11:29:10.073 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 112/202 2024-02-18T11:29:10.110 INFO:teuthology.orchestra.run.smithi012.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/202 2024-02-18T11:29:10.118 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/202 2024-02-18T11:29:10.120 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 113/202 2024-02-18T11:29:10.145 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/202 2024-02-18T11:29:10.158 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/202 2024-02-18T11:29:10.195 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : librados-devel-2:19.0.0-1555.g1c920afa.el9.x86_6 52/202 2024-02-18T11:29:10.198 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/202 2024-02-18T11:29:10.202 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 114/202 2024-02-18T11:29:10.249 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 115/202 2024-02-18T11:29:10.426 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : openblas-0.3.21-2.el9.x86_64 116/202 2024-02-18T11:29:10.447 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 117/202 2024-02-18T11:29:10.473 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 53/202 2024-02-18T11:29:10.524 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 118/202 2024-02-18T11:29:10.619 INFO:teuthology.orchestra.run.smithi012.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 54/202 2024-02-18T11:29:10.670 INFO:teuthology.orchestra.run.smithi012.stdout: Upgrading : iproute-6.7.0-2.el9.x86_64 55/202 2024-02-18T11:29:10.715 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/202 2024-02-18T11:29:10.802 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/202 2024-02-18T11:29:10.880 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/202 2024-02-18T11:29:10.923 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/202 2024-02-18T11:29:10.931 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/202 2024-02-18T11:29:10.958 INFO:teuthology.orchestra.run.smithi012.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 56/202 2024-02-18T11:29:10.969 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/202 2024-02-18T11:29:11.033 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : librados-devel-2:19.0.0-1555.g1c920afa.el9.x86_6 52/202 2024-02-18T11:29:11.064 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 53/202 2024-02-18T11:29:11.112 INFO:teuthology.orchestra.run.smithi012.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 57/202 2024-02-18T11:29:11.171 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 119/202 2024-02-18T11:29:11.173 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 54/202 2024-02-18T11:29:11.187 INFO:teuthology.orchestra.run.smithi012.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 58/202 2024-02-18T11:29:11.223 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : iproute-6.7.0-2.el9.x86_64 55/202 2024-02-18T11:29:11.233 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 59/202 2024-02-18T11:29:11.295 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 120/202 2024-02-18T11:29:11.374 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 60/202 2024-02-18T11:29:11.418 INFO:teuthology.orchestra.run.smithi012.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 61/202 2024-02-18T11:29:11.467 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 56/202 2024-02-18T11:29:11.474 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 62/202 2024-02-18T11:29:11.548 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 63/202 2024-02-18T11:29:11.585 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 64/202 2024-02-18T11:29:11.622 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 65/202 2024-02-18T11:29:11.632 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 57/202 2024-02-18T11:29:11.669 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 66/202 2024-02-18T11:29:11.701 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 58/202 2024-02-18T11:29:11.704 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 67/202 2024-02-18T11:29:11.745 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 59/202 2024-02-18T11:29:11.748 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 68/202 2024-02-18T11:29:11.790 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 69/202 2024-02-18T11:29:11.885 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 60/202 2024-02-18T11:29:11.929 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 70/202 2024-02-18T11:29:11.940 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 61/202 2024-02-18T11:29:11.978 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 71/202 2024-02-18T11:29:11.994 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 62/202 2024-02-18T11:29:12.009 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 72/202 2024-02-18T11:29:12.051 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 73/202 2024-02-18T11:29:12.147 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 74/202 2024-02-18T11:29:12.150 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 63/202 2024-02-18T11:29:12.183 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-google-auth-1:2.27.0-1.el9.noarch 75/202 2024-02-18T11:29:12.224 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 76/202 2024-02-18T11:29:12.250 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 77/202 2024-02-18T11:29:12.295 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 78/202 2024-02-18T11:29:12.304 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 64/202 2024-02-18T11:29:12.340 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 65/202 2024-02-18T11:29:12.387 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 66/202 2024-02-18T11:29:12.423 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 67/202 2024-02-18T11:29:12.467 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 68/202 2024-02-18T11:29:12.498 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 79/202 2024-02-18T11:29:12.508 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 69/202 2024-02-18T11:29:12.517 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 121/202 2024-02-18T11:29:12.593 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 122/202 2024-02-18T11:29:12.620 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 123/202 2024-02-18T11:29:12.641 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 70/202 2024-02-18T11:29:12.689 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 71/202 2024-02-18T11:29:12.719 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 72/202 2024-02-18T11:29:12.760 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 73/202 2024-02-18T11:29:12.840 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 80/202 2024-02-18T11:29:12.863 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 74/202 2024-02-18T11:29:12.926 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 124/202 2024-02-18T11:29:12.936 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-google-auth-1:2.27.0-1.el9.noarch 75/202 2024-02-18T11:29:12.944 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 81/202 2024-02-18T11:29:12.957 INFO:teuthology.orchestra.run.smithi158.stdout: Upgrading : librbd1-2:19.0.0-1555.g1c920afa.el9.x86_64 125/202 2024-02-18T11:29:12.975 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 76/202 2024-02-18T11:29:12.991 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 82/202 2024-02-18T11:29:13.002 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 77/202 2024-02-18T11:29:13.026 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 78/202 2024-02-18T11:29:13.048 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: librbd1-2:19.0.0-1555.g1c920afa.el9.x86_64 125/202 2024-02-18T11:29:13.048 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-ply-3.11-14.el9.noarch 83/202 2024-02-18T11:29:13.079 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-rbd-2:19.0.0-1555.g1c920afa.el9.x86_64 126/202 2024-02-18T11:29:13.098 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 79/202 2024-02-18T11:29:13.104 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 84/202 2024-02-18T11:29:13.115 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 127/202 2024-02-18T11:29:13.234 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 80/202 2024-02-18T11:29:13.293 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 85/202 2024-02-18T11:29:13.342 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 81/202 2024-02-18T11:29:13.345 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 86/202 2024-02-18T11:29:13.385 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 82/202 2024-02-18T11:29:13.433 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-ply-3.11-14.el9.noarch 83/202 2024-02-18T11:29:13.486 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 87/202 2024-02-18T11:29:13.489 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 84/202 2024-02-18T11:29:13.527 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 88/202 2024-02-18T11:29:13.559 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 89/202 2024-02-18T11:29:13.566 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 128/202 2024-02-18T11:29:13.590 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 90/202 2024-02-18T11:29:13.594 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : librgw2-2:19.0.0-1555.g1c920afa.el9.x86_64 129/202 2024-02-18T11:29:13.626 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : jq-1.6-16.el9.x86_64 91/202 2024-02-18T11:29:13.656 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : fuse-2.9.9-15.el9.x86_64 92/202 2024-02-18T11:29:13.665 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: librgw2-2:19.0.0-1555.g1c920afa.el9.x86_64 129/202 2024-02-18T11:29:13.672 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 85/202 2024-02-18T11:29:13.684 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 93/202 2024-02-18T11:29:13.695 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-rgw-2:19.0.0-1555.g1c920afa.el9.x86_64 130/202 2024-02-18T11:29:13.719 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 94/202 2024-02-18T11:29:13.723 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 86/202 2024-02-18T11:29:13.758 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : uuid-1.6.2-55.el9.x86_64 95/202 2024-02-18T11:29:13.790 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 96/202 2024-02-18T11:29:13.819 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 97/202 2024-02-18T11:29:13.858 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 87/202 2024-02-18T11:29:13.897 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 98/202 2024-02-18T11:29:13.904 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 88/202 2024-02-18T11:29:13.930 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 99/202 2024-02-18T11:29:13.937 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 89/202 2024-02-18T11:29:13.963 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 100/202 2024-02-18T11:29:13.969 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 90/202 2024-02-18T11:29:14.004 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : jq-1.6-16.el9.x86_64 91/202 2024-02-18T11:29:14.005 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 101/202 2024-02-18T11:29:14.035 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : fuse-2.9.9-15.el9.x86_64 92/202 2024-02-18T11:29:14.040 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 102/202 2024-02-18T11:29:14.073 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 93/202 2024-02-18T11:29:14.114 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 94/202 2024-02-18T11:29:14.131 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 103/202 2024-02-18T11:29:14.153 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : uuid-1.6.2-55.el9.x86_64 95/202 2024-02-18T11:29:14.185 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 96/202 2024-02-18T11:29:14.224 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 97/202 2024-02-18T11:29:14.228 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 104/202 2024-02-18T11:29:14.271 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 105/202 2024-02-18T11:29:14.298 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 98/202 2024-02-18T11:29:14.301 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 106/202 2024-02-18T11:29:14.334 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 99/202 2024-02-18T11:29:14.349 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 107/202 2024-02-18T11:29:14.367 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 100/202 2024-02-18T11:29:14.409 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 101/202 2024-02-18T11:29:14.443 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 102/202 2024-02-18T11:29:14.449 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-packaging-20.9-5.el9.noarch 108/202 2024-02-18T11:29:14.503 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 109/202 2024-02-18T11:29:14.540 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 103/202 2024-02-18T11:29:14.622 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 104/202 2024-02-18T11:29:14.683 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 105/202 2024-02-18T11:29:14.723 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 106/202 2024-02-18T11:29:14.771 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 107/202 2024-02-18T11:29:14.874 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-packaging-20.9-5.el9.noarch 108/202 2024-02-18T11:29:14.924 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 109/202 2024-02-18T11:29:15.263 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 110/202 2024-02-18T11:29:15.309 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 111/202 2024-02-18T11:29:15.342 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 112/202 2024-02-18T11:29:15.391 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 113/202 2024-02-18T11:29:15.480 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 114/202 2024-02-18T11:29:15.518 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 115/202 2024-02-18T11:29:15.650 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 110/202 2024-02-18T11:29:15.696 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 111/202 2024-02-18T11:29:15.702 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : openblas-0.3.21-2.el9.x86_64 116/202 2024-02-18T11:29:15.730 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 112/202 2024-02-18T11:29:15.732 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 117/202 2024-02-18T11:29:15.779 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 113/202 2024-02-18T11:29:15.820 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 118/202 2024-02-18T11:29:15.865 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 114/202 2024-02-18T11:29:15.965 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 115/202 2024-02-18T11:29:16.144 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : openblas-0.3.21-2.el9.x86_64 116/202 2024-02-18T11:29:16.171 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 117/202 2024-02-18T11:29:16.248 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 118/202 2024-02-18T11:29:16.257 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: ceph-common-2:19.0.0-1555.g1c920afa.el9.x86_64 131/202 2024-02-18T11:29:16.292 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : ceph-common-2:19.0.0-1555.g1c920afa.el9.x86_64 131/202 2024-02-18T11:29:16.496 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 119/202 2024-02-18T11:29:16.616 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 120/202 2024-02-18T11:29:16.827 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: ceph-common-2:19.0.0-1555.g1c920afa.el9.x86_64 131/202 2024-02-18T11:29:16.862 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : ceph-base-2:19.0.0-1555.g1c920afa.el9.x86_64 132/202 2024-02-18T11:29:16.885 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 119/202 2024-02-18T11:29:16.943 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: ceph-base-2:19.0.0-1555.g1c920afa.el9.x86_64 132/202 2024-02-18T11:29:16.944 INFO:teuthology.orchestra.run.smithi158.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-02-18T11:29:16.944 INFO:teuthology.orchestra.run.smithi158.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-02-18T11:29:16.944 INFO:teuthology.orchestra.run.smithi158.stdout: 2024-02-18T11:29:16.972 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : ceph-selinux-2:19.0.0-1555.g1c920afa.el9.x86_64 133/202 2024-02-18T11:29:16.993 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 120/202 2024-02-18T11:29:17.913 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 121/202 2024-02-18T11:29:17.993 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 122/202 2024-02-18T11:29:18.021 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 123/202 2024-02-18T11:29:18.202 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 121/202 2024-02-18T11:29:18.284 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 122/202 2024-02-18T11:29:18.311 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 123/202 2024-02-18T11:29:18.346 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 124/202 2024-02-18T11:29:18.375 INFO:teuthology.orchestra.run.smithi012.stdout: Upgrading : librbd1-2:19.0.0-1555.g1c920afa.el9.x86_64 125/202 2024-02-18T11:29:18.466 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: librbd1-2:19.0.0-1555.g1c920afa.el9.x86_64 125/202 2024-02-18T11:29:18.497 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-rbd-2:19.0.0-1555.g1c920afa.el9.x86_64 126/202 2024-02-18T11:29:18.535 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 127/202 2024-02-18T11:29:18.617 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 124/202 2024-02-18T11:29:18.649 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : librbd1-2:19.0.0-1555.g1c920afa.el9.x86_64 125/202 2024-02-18T11:29:18.739 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librbd1-2:19.0.0-1555.g1c920afa.el9.x86_64 125/202 2024-02-18T11:29:18.771 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-rbd-2:19.0.0-1555.g1c920afa.el9.x86_64 126/202 2024-02-18T11:29:18.807 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 127/202 2024-02-18T11:29:19.011 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 128/202 2024-02-18T11:29:19.036 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : librgw2-2:19.0.0-1555.g1c920afa.el9.x86_64 129/202 2024-02-18T11:29:19.091 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: librgw2-2:19.0.0-1555.g1c920afa.el9.x86_64 129/202 2024-02-18T11:29:19.121 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-rgw-2:19.0.0-1555.g1c920afa.el9.x86_64 130/202 2024-02-18T11:29:19.272 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 128/202 2024-02-18T11:29:19.498 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : librgw2-2:19.0.0-1555.g1c920afa.el9.x86_64 129/202 2024-02-18T11:29:19.632 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librgw2-2:19.0.0-1555.g1c920afa.el9.x86_64 129/202 2024-02-18T11:29:19.662 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-rgw-2:19.0.0-1555.g1c920afa.el9.x86_64 130/202 2024-02-18T11:29:21.378 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-common-2:19.0.0-1555.g1c920afa.el9.x86_64 131/202 2024-02-18T11:29:21.409 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-common-2:19.0.0-1555.g1c920afa.el9.x86_64 131/202 2024-02-18T11:29:21.843 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-common-2:19.0.0-1555.g1c920afa.el9.x86_64 131/202 2024-02-18T11:29:21.884 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-common-2:19.0.0-1555.g1c920afa.el9.x86_64 131/202 2024-02-18T11:29:21.980 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-common-2:19.0.0-1555.g1c920afa.el9.x86_64 131/202 2024-02-18T11:29:22.013 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-base-2:19.0.0-1555.g1c920afa.el9.x86_64 132/202 2024-02-18T11:29:22.104 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-base-2:19.0.0-1555.g1c920afa.el9.x86_64 132/202 2024-02-18T11:29:22.104 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-02-18T11:29:22.104 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-02-18T11:29:22.104 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:29:22.139 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-selinux-2:19.0.0-1555.g1c920afa.el9.x86_64 133/202 2024-02-18T11:29:22.419 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-common-2:19.0.0-1555.g1c920afa.el9.x86_64 131/202 2024-02-18T11:29:22.454 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-base-2:19.0.0-1555.g1c920afa.el9.x86_64 132/202 2024-02-18T11:29:22.527 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-base-2:19.0.0-1555.g1c920afa.el9.x86_64 132/202 2024-02-18T11:29:22.527 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-02-18T11:29:22.527 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-02-18T11:29:22.527 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-02-18T11:29:22.555 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-selinux-2:19.0.0-1555.g1c920afa.el9.x86_64 133/202 2024-02-18T11:29:28.784 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1555.g1c920afa.el9.x86_64 133/202 2024-02-18T11:29:28.784 INFO:teuthology.orchestra.run.smithi158.stdout:skipping the directory /sys 2024-02-18T11:29:28.784 INFO:teuthology.orchestra.run.smithi158.stdout:skipping the directory /proc 2024-02-18T11:29:28.785 INFO:teuthology.orchestra.run.smithi158.stdout:skipping the directory /mnt 2024-02-18T11:29:28.785 INFO:teuthology.orchestra.run.smithi158.stdout:skipping the directory /var/tmp 2024-02-18T11:29:28.785 INFO:teuthology.orchestra.run.smithi158.stdout:skipping the directory /home 2024-02-18T11:29:28.785 INFO:teuthology.orchestra.run.smithi158.stdout:skipping the directory /root 2024-02-18T11:29:28.785 INFO:teuthology.orchestra.run.smithi158.stdout:skipping the directory /tmp 2024-02-18T11:29:28.785 INFO:teuthology.orchestra.run.smithi158.stdout: 2024-02-18T11:29:28.946 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : ceph-volume-2:19.0.0-1555.g1c920afa.el9.noarch 134/202 2024-02-18T11:29:28.962 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: ceph-volume-2:19.0.0-1555.g1c920afa.el9.noarch 134/202 2024-02-18T11:29:28.963 INFO:teuthology.orchestra.run.smithi158.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-18T11:29:28.963 INFO:teuthology.orchestra.run.smithi158.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-02-18T11:29:28.963 INFO:teuthology.orchestra.run.smithi158.stdout: 2024-02-18T11:29:30.424 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : ceph-osd-2:19.0.0-1555.g1c920afa.el9.x86_64 135/202 2024-02-18T11:29:30.450 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: ceph-osd-2:19.0.0-1555.g1c920afa.el9.x86_64 135/202 2024-02-18T11:29:30.450 INFO:teuthology.orchestra.run.smithi158.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-18T11:29:30.450 INFO:teuthology.orchestra.run.smithi158.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-02-18T11:29:30.450 INFO:teuthology.orchestra.run.smithi158.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-18T11:29:30.451 INFO:teuthology.orchestra.run.smithi158.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-18T11:29:30.451 INFO:teuthology.orchestra.run.smithi158.stdout: 2024-02-18T11:29:30.658 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : ceph-mds-2:19.0.0-1555.g1c920afa.el9.x86_64 136/202 2024-02-18T11:29:30.681 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: ceph-mds-2:19.0.0-1555.g1c920afa.el9.x86_64 136/202 2024-02-18T11:29:30.681 INFO:teuthology.orchestra.run.smithi158.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-18T11:29:30.681 INFO:teuthology.orchestra.run.smithi158.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-02-18T11:29:30.681 INFO:teuthology.orchestra.run.smithi158.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-18T11:29:30.681 INFO:teuthology.orchestra.run.smithi158.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-18T11:29:30.681 INFO:teuthology.orchestra.run.smithi158.stdout: 2024-02-18T11:29:31.125 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : ceph-mon-2:19.0.0-1555.g1c920afa.el9.x86_64 137/202 2024-02-18T11:29:31.148 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: ceph-mon-2:19.0.0-1555.g1c920afa.el9.x86_64 137/202 2024-02-18T11:29:31.148 INFO:teuthology.orchestra.run.smithi158.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-18T11:29:31.149 INFO:teuthology.orchestra.run.smithi158.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-02-18T11:29:31.149 INFO:teuthology.orchestra.run.smithi158.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-18T11:29:31.149 INFO:teuthology.orchestra.run.smithi158.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-18T11:29:31.149 INFO:teuthology.orchestra.run.smithi158.stdout: 2024-02-18T11:29:31.244 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : automake-1.16.2-8.el9.noarch 138/202 2024-02-18T11:29:31.604 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: cephadm-2:19.0.0-1555.g1c920afa.el9.noarch 139/202 2024-02-18T11:29:31.634 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : cephadm-2:19.0.0-1555.g1c920afa.el9.noarch 139/202 2024-02-18T11:29:31.665 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1555.g1c920afa.e 140/202 2024-02-18T11:29:31.727 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1555.g1c920afa. 141/202 2024-02-18T11:29:31.757 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1555.g1c920afa.el9.noa 142/202 2024-02-18T11:29:31.988 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1555.g1c920afa.el9.noa 142/202 2024-02-18T11:29:32.016 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1555.g1c920afa.el9.n 143/202 2024-02-18T11:29:32.983 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1555.g1c920afa.el9.n 143/202 2024-02-18T11:29:33.008 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1555.g1c9 144/202 2024-02-18T11:29:33.112 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1555.g1c9 144/202 2024-02-18T11:29:33.279 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1555.g1c920afa.el 145/202 2024-02-18T11:29:33.311 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : ceph-mgr-2:19.0.0-1555.g1c920afa.el9.x86_64 146/202 2024-02-18T11:29:33.334 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1555.g1c920afa.el9.x86_64 146/202 2024-02-18T11:29:33.334 INFO:teuthology.orchestra.run.smithi158.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-18T11:29:33.334 INFO:teuthology.orchestra.run.smithi158.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-02-18T11:29:33.334 INFO:teuthology.orchestra.run.smithi158.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-18T11:29:33.334 INFO:teuthology.orchestra.run.smithi158.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-18T11:29:33.334 INFO:teuthology.orchestra.run.smithi158.stdout: 2024-02-18T11:29:33.386 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : ceph-mgr-rook-2:19.0.0-1555.g1c920afa.el9.noarch 147/202 2024-02-18T11:29:33.397 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1555.g1c920afa.el9.noarch 147/202 2024-02-18T11:29:33.443 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : ceph-2:19.0.0-1555.g1c920afa.el9.x86_64 148/202 2024-02-18T11:29:34.295 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : libtool-2.4.6-45.el9.x86_64 149/202 2024-02-18T11:29:34.330 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : ceph-radosgw-2:19.0.0-1555.g1c920afa.el9.x86_64 150/202 2024-02-18T11:29:34.337 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1555.g1c920afa.el9.x86_64 133/202 2024-02-18T11:29:34.337 INFO:teuthology.orchestra.run.smithi012.stdout:skipping the directory /sys 2024-02-18T11:29:34.337 INFO:teuthology.orchestra.run.smithi012.stdout:skipping the directory /proc 2024-02-18T11:29:34.337 INFO:teuthology.orchestra.run.smithi012.stdout:skipping the directory /mnt 2024-02-18T11:29:34.337 INFO:teuthology.orchestra.run.smithi012.stdout:skipping the directory /var/tmp 2024-02-18T11:29:34.337 INFO:teuthology.orchestra.run.smithi012.stdout:skipping the directory /home 2024-02-18T11:29:34.337 INFO:teuthology.orchestra.run.smithi012.stdout:skipping the directory /root 2024-02-18T11:29:34.337 INFO:teuthology.orchestra.run.smithi012.stdout:skipping the directory /tmp 2024-02-18T11:29:34.338 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:29:34.353 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1555.g1c920afa.el9.x86_64 150/202 2024-02-18T11:29:34.353 INFO:teuthology.orchestra.run.smithi158.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-18T11:29:34.353 INFO:teuthology.orchestra.run.smithi158.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-02-18T11:29:34.353 INFO:teuthology.orchestra.run.smithi158.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-18T11:29:34.353 INFO:teuthology.orchestra.run.smithi158.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-18T11:29:34.353 INFO:teuthology.orchestra.run.smithi158.stdout: 2024-02-18T11:29:34.428 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1555.g1c920 151/202 2024-02-18T11:29:34.450 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1555.g1c920 151/202 2024-02-18T11:29:34.450 INFO:teuthology.orchestra.run.smithi158.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-18T11:29:34.451 INFO:teuthology.orchestra.run.smithi158.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-02-18T11:29:34.451 INFO:teuthology.orchestra.run.smithi158.stdout: 2024-02-18T11:29:34.465 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1555.g1c920afa.el9.x86_64 133/202 2024-02-18T11:29:34.465 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /sys 2024-02-18T11:29:34.465 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /proc 2024-02-18T11:29:34.466 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /mnt 2024-02-18T11:29:34.466 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /var/tmp 2024-02-18T11:29:34.466 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /home 2024-02-18T11:29:34.466 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /root 2024-02-18T11:29:34.466 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /tmp 2024-02-18T11:29:34.466 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-02-18T11:29:34.498 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : cephfs-mirror-2:19.0.0-1555.g1c920afa.el9.x86_64 152/202 2024-02-18T11:29:34.500 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-volume-2:19.0.0-1555.g1c920afa.el9.noarch 134/202 2024-02-18T11:29:34.515 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-volume-2:19.0.0-1555.g1c920afa.el9.noarch 134/202 2024-02-18T11:29:34.516 INFO:teuthology.orchestra.run.smithi012.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-18T11:29:34.516 INFO:teuthology.orchestra.run.smithi012.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-02-18T11:29:34.516 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:29:34.520 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1555.g1c920afa.el9.x86_64 152/202 2024-02-18T11:29:34.521 INFO:teuthology.orchestra.run.smithi158.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-18T11:29:34.521 INFO:teuthology.orchestra.run.smithi158.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-02-18T11:29:34.521 INFO:teuthology.orchestra.run.smithi158.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-02-18T11:29:34.521 INFO:teuthology.orchestra.run.smithi158.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-02-18T11:29:34.521 INFO:teuthology.orchestra.run.smithi158.stdout: 2024-02-18T11:29:34.627 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-volume-2:19.0.0-1555.g1c920afa.el9.noarch 134/202 2024-02-18T11:29:34.642 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-volume-2:19.0.0-1555.g1c920afa.el9.noarch 134/202 2024-02-18T11:29:34.642 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-18T11:29:34.642 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-02-18T11:29:34.642 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-02-18T11:29:34.849 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : rbd-mirror-2:19.0.0-1555.g1c920afa.el9.x86_64 153/202 2024-02-18T11:29:34.871 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1555.g1c920afa.el9.x86_64 153/202 2024-02-18T11:29:34.872 INFO:teuthology.orchestra.run.smithi158.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-18T11:29:34.872 INFO:teuthology.orchestra.run.smithi158.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-02-18T11:29:34.872 INFO:teuthology.orchestra.run.smithi158.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-18T11:29:34.872 INFO:teuthology.orchestra.run.smithi158.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-18T11:29:34.872 INFO:teuthology.orchestra.run.smithi158.stdout: 2024-02-18T11:29:36.053 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-osd-2:19.0.0-1555.g1c920afa.el9.x86_64 135/202 2024-02-18T11:29:36.078 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-osd-2:19.0.0-1555.g1c920afa.el9.x86_64 135/202 2024-02-18T11:29:36.079 INFO:teuthology.orchestra.run.smithi012.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-18T11:29:36.079 INFO:teuthology.orchestra.run.smithi012.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-02-18T11:29:36.079 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-18T11:29:36.079 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-18T11:29:36.079 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:29:36.121 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-osd-2:19.0.0-1555.g1c920afa.el9.x86_64 135/202 2024-02-18T11:29:36.146 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-osd-2:19.0.0-1555.g1c920afa.el9.x86_64 135/202 2024-02-18T11:29:36.147 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-18T11:29:36.147 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-02-18T11:29:36.147 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-18T11:29:36.147 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-18T11:29:36.147 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-02-18T11:29:36.296 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-mds-2:19.0.0-1555.g1c920afa.el9.x86_64 136/202 2024-02-18T11:29:36.318 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-mds-2:19.0.0-1555.g1c920afa.el9.x86_64 136/202 2024-02-18T11:29:36.318 INFO:teuthology.orchestra.run.smithi012.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-18T11:29:36.318 INFO:teuthology.orchestra.run.smithi012.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-02-18T11:29:36.319 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-18T11:29:36.319 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-18T11:29:36.319 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:29:36.359 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mds-2:19.0.0-1555.g1c920afa.el9.x86_64 136/202 2024-02-18T11:29:36.381 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mds-2:19.0.0-1555.g1c920afa.el9.x86_64 136/202 2024-02-18T11:29:36.381 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-18T11:29:36.381 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-02-18T11:29:36.382 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-18T11:29:36.382 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-18T11:29:36.382 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-02-18T11:29:36.780 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-mon-2:19.0.0-1555.g1c920afa.el9.x86_64 137/202 2024-02-18T11:29:36.801 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-mon-2:19.0.0-1555.g1c920afa.el9.x86_64 137/202 2024-02-18T11:29:36.801 INFO:teuthology.orchestra.run.smithi012.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-18T11:29:36.802 INFO:teuthology.orchestra.run.smithi012.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-02-18T11:29:36.802 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-18T11:29:36.802 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-18T11:29:36.802 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:29:36.827 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mon-2:19.0.0-1555.g1c920afa.el9.x86_64 137/202 2024-02-18T11:29:36.849 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mon-2:19.0.0-1555.g1c920afa.el9.x86_64 137/202 2024-02-18T11:29:36.849 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-18T11:29:36.849 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-02-18T11:29:36.849 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-18T11:29:36.849 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-18T11:29:36.850 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-02-18T11:29:36.890 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : automake-1.16.2-8.el9.noarch 138/202 2024-02-18T11:29:36.945 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : automake-1.16.2-8.el9.noarch 138/202 2024-02-18T11:29:37.334 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: cephadm-2:19.0.0-1555.g1c920afa.el9.noarch 139/202 2024-02-18T11:29:37.337 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: cephadm-2:19.0.0-1555.g1c920afa.el9.noarch 139/202 2024-02-18T11:29:37.363 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : cephadm-2:19.0.0-1555.g1c920afa.el9.noarch 139/202 2024-02-18T11:29:37.368 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : cephadm-2:19.0.0-1555.g1c920afa.el9.noarch 139/202 2024-02-18T11:29:37.395 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1555.g1c920afa.e 140/202 2024-02-18T11:29:37.400 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1555.g1c920afa.e 140/202 2024-02-18T11:29:37.458 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1555.g1c920afa. 141/202 2024-02-18T11:29:37.460 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1555.g1c920afa. 141/202 2024-02-18T11:29:37.486 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1555.g1c920afa.el9.noa 142/202 2024-02-18T11:29:37.508 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1555.g1c920afa.el9.noa 142/202 2024-02-18T11:29:37.726 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1555.g1c920afa.el9.noa 142/202 2024-02-18T11:29:37.736 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1555.g1c920afa.el9.noa 142/202 2024-02-18T11:29:37.762 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1555.g1c920afa.el9.n 143/202 2024-02-18T11:29:37.768 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1555.g1c920afa.el9.n 143/202 2024-02-18T11:29:38.730 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1555.g1c920afa.el9.n 143/202 2024-02-18T11:29:38.768 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1555.g1c9 144/202 2024-02-18T11:29:38.776 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1555.g1c920afa.el9.n 143/202 2024-02-18T11:29:38.804 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1555.g1c9 144/202 2024-02-18T11:29:38.868 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1555.g1c9 144/202 2024-02-18T11:29:38.909 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1555.g1c9 144/202 2024-02-18T11:29:39.038 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1555.g1c920afa.el 145/202 2024-02-18T11:29:39.070 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-2:19.0.0-1555.g1c920afa.el9.x86_64 146/202 2024-02-18T11:29:39.073 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1555.g1c920afa.el 145/202 2024-02-18T11:29:39.093 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1555.g1c920afa.el9.x86_64 146/202 2024-02-18T11:29:39.093 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-18T11:29:39.093 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-02-18T11:29:39.094 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-18T11:29:39.094 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-18T11:29:39.094 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-02-18T11:29:39.106 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-mgr-2:19.0.0-1555.g1c920afa.el9.x86_64 146/202 2024-02-18T11:29:39.128 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1555.g1c920afa.el9.x86_64 146/202 2024-02-18T11:29:39.128 INFO:teuthology.orchestra.run.smithi012.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-18T11:29:39.128 INFO:teuthology.orchestra.run.smithi012.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-02-18T11:29:39.129 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-18T11:29:39.129 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-18T11:29:39.129 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:29:39.138 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-rook-2:19.0.0-1555.g1c920afa.el9.noarch 147/202 2024-02-18T11:29:39.149 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1555.g1c920afa.el9.noarch 147/202 2024-02-18T11:29:39.173 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-mgr-rook-2:19.0.0-1555.g1c920afa.el9.noarch 147/202 2024-02-18T11:29:39.184 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1555.g1c920afa.el9.noarch 147/202 2024-02-18T11:29:39.194 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-2:19.0.0-1555.g1c920afa.el9.x86_64 148/202 2024-02-18T11:29:39.234 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-2:19.0.0-1555.g1c920afa.el9.x86_64 148/202 2024-02-18T11:29:39.287 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : ceph-test-2:19.0.0-1555.g1c920afa.el9.x86_64 154/202 2024-02-18T11:29:39.327 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : rbd-fuse-2:19.0.0-1555.g1c920afa.el9.x86_64 155/202 2024-02-18T11:29:39.358 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : rbd-nbd-2:19.0.0-1555.g1c920afa.el9.x86_64 156/202 2024-02-18T11:29:39.385 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 157/202 2024-02-18T11:29:39.570 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : cephfs-shell-2:19.0.0-1555.g1c920afa.el9.x86_64 158/202 2024-02-18T11:29:39.692 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 159/202 2024-02-18T11:29:39.733 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : ceph-fuse-2:19.0.0-1555.g1c920afa.el9.x86_64 160/202 2024-02-18T11:29:39.772 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 161/202 2024-02-18T11:29:39.850 INFO:teuthology.orchestra.run.smithi158.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 162/202 2024-02-18T11:29:39.933 INFO:teuthology.orchestra.run.smithi158.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 163/202 2024-02-18T11:29:39.986 INFO:teuthology.orchestra.run.smithi158.stdout: Upgrading : iproute-tc-6.7.0-2.el9.x86_64 164/202 2024-02-18T11:29:40.029 INFO:teuthology.orchestra.run.smithi158.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 165/202 2024-02-18T11:29:40.045 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libtool-2.4.6-45.el9.x86_64 149/202 2024-02-18T11:29:40.059 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : libcephfs-devel-2:19.0.0-1555.g1c920afa.el9.x86_ 166/202 2024-02-18T11:29:40.073 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-radosgw-2:19.0.0-1555.g1c920afa.el9.x86_64 150/202 2024-02-18T11:29:40.089 INFO:teuthology.orchestra.run.smithi158.stdout: Upgrading : NetworkManager-team-1:1.45.91-1.el9.x86_64 167/202 2024-02-18T11:29:40.096 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1555.g1c920afa.el9.x86_64 150/202 2024-02-18T11:29:40.096 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-18T11:29:40.096 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-02-18T11:29:40.096 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-18T11:29:40.097 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-18T11:29:40.097 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-02-18T11:29:40.137 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libtool-2.4.6-45.el9.x86_64 149/202 2024-02-18T11:29:40.141 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1555.g1c920 151/202 2024-02-18T11:29:40.153 INFO:teuthology.orchestra.run.smithi158.stdout: Upgrading : NetworkManager-tui-1:1.45.91-1.el9.x86_64 168/202 2024-02-18T11:29:40.162 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1555.g1c920 151/202 2024-02-18T11:29:40.163 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-18T11:29:40.163 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-02-18T11:29:40.163 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-02-18T11:29:40.167 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-radosgw-2:19.0.0-1555.g1c920afa.el9.x86_64 150/202 2024-02-18T11:29:40.189 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1555.g1c920afa.el9.x86_64 150/202 2024-02-18T11:29:40.189 INFO:teuthology.orchestra.run.smithi012.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-18T11:29:40.189 INFO:teuthology.orchestra.run.smithi012.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-02-18T11:29:40.189 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-18T11:29:40.189 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-18T11:29:40.189 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:29:40.194 INFO:teuthology.orchestra.run.smithi158.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 169/202 2024-02-18T11:29:40.216 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : cephfs-mirror-2:19.0.0-1555.g1c920afa.el9.x86_64 152/202 2024-02-18T11:29:40.229 INFO:teuthology.orchestra.run.smithi158.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 170/202 2024-02-18T11:29:40.238 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1555.g1c920afa.el9.x86_64 152/202 2024-02-18T11:29:40.238 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-18T11:29:40.238 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-02-18T11:29:40.238 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-02-18T11:29:40.238 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-02-18T11:29:40.238 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-02-18T11:29:40.242 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1555.g1c920 151/202 2024-02-18T11:29:40.264 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1555.g1c920 151/202 2024-02-18T11:29:40.264 INFO:teuthology.orchestra.run.smithi012.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-18T11:29:40.264 INFO:teuthology.orchestra.run.smithi012.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-02-18T11:29:40.264 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:29:40.310 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : cephfs-mirror-2:19.0.0-1555.g1c920afa.el9.x86_64 152/202 2024-02-18T11:29:40.332 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1555.g1c920afa.el9.x86_64 152/202 2024-02-18T11:29:40.332 INFO:teuthology.orchestra.run.smithi012.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-18T11:29:40.332 INFO:teuthology.orchestra.run.smithi012.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-02-18T11:29:40.332 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-02-18T11:29:40.332 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-02-18T11:29:40.332 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:29:40.551 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : rbd-mirror-2:19.0.0-1555.g1c920afa.el9.x86_64 153/202 2024-02-18T11:29:40.573 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1555.g1c920afa.el9.x86_64 153/202 2024-02-18T11:29:40.573 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-18T11:29:40.573 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-02-18T11:29:40.573 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-18T11:29:40.573 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-18T11:29:40.573 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-02-18T11:29:40.644 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : rbd-mirror-2:19.0.0-1555.g1c920afa.el9.x86_64 153/202 2024-02-18T11:29:40.665 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1555.g1c920afa.el9.x86_64 153/202 2024-02-18T11:29:40.665 INFO:teuthology.orchestra.run.smithi012.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-18T11:29:40.666 INFO:teuthology.orchestra.run.smithi012.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-02-18T11:29:40.666 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-18T11:29:40.666 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-18T11:29:40.666 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:29:40.818 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 171/202 2024-02-18T11:29:40.852 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 171/202 2024-02-18T11:29:40.873 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 171/202 2024-02-18T11:29:41.718 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : cephfs-top-2:19.0.0-1555.g1c920afa.el9.noarch 172/202 2024-02-18T11:29:41.789 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 173/202 2024-02-18T11:29:41.830 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 174/202 2024-02-18T11:29:41.913 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : flex-2.6.4-9.el9.x86_64 175/202 2024-02-18T11:29:41.952 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : bison-3.7.4-5.el9.x86_64 176/202 2024-02-18T11:29:41.984 INFO:teuthology.orchestra.run.smithi158.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 177/202 2024-02-18T11:29:42.010 INFO:teuthology.orchestra.run.smithi158.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 178/202 2024-02-18T11:29:42.036 INFO:teuthology.orchestra.run.smithi158.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 179/202 2024-02-18T11:29:42.036 INFO:teuthology.orchestra.run.smithi158.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 180/202 2024-02-18T11:29:42.121 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 180/202 2024-02-18T11:29:42.149 INFO:teuthology.orchestra.run.smithi158.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 181/202 2024-02-18T11:29:42.179 INFO:teuthology.orchestra.run.smithi158.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 182/202 2024-02-18T11:29:42.179 INFO:teuthology.orchestra.run.smithi158.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 183/202 2024-02-18T11:29:42.264 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 183/202 2024-02-18T11:29:42.290 INFO:teuthology.orchestra.run.smithi158.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 184/202 2024-02-18T11:29:42.332 INFO:teuthology.orchestra.run.smithi158.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 185/202 2024-02-18T11:29:42.357 INFO:teuthology.orchestra.run.smithi158.stdout: Cleanup : iproute-tc-6.2.0-5.el9.x86_64 186/202 2024-02-18T11:29:42.384 INFO:teuthology.orchestra.run.smithi158.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 187/202 2024-02-18T11:29:42.417 INFO:teuthology.orchestra.run.smithi158.stdout: Cleanup : iproute-6.2.0-5.el9.x86_64 188/202 2024-02-18T11:29:42.450 INFO:teuthology.orchestra.run.smithi158.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 189/202 2024-02-18T11:29:42.489 INFO:teuthology.orchestra.run.smithi158.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 190/202 2024-02-18T11:29:42.522 INFO:teuthology.orchestra.run.smithi158.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 191/202 2024-02-18T11:29:42.547 INFO:teuthology.orchestra.run.smithi158.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 192/202 2024-02-18T11:29:42.572 INFO:teuthology.orchestra.run.smithi158.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 193/202 2024-02-18T11:29:42.598 INFO:teuthology.orchestra.run.smithi158.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 194/202 2024-02-18T11:29:42.623 INFO:teuthology.orchestra.run.smithi158.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 195/202 2024-02-18T11:29:42.630 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 196/202 2024-02-18T11:29:42.630 INFO:teuthology.orchestra.run.smithi158.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 196/202 2024-02-18T11:29:42.834 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 196/202 2024-02-18T11:29:42.834 INFO:teuthology.orchestra.run.smithi158.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 197/202 2024-02-18T11:29:42.917 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 197/202 2024-02-18T11:29:42.942 INFO:teuthology.orchestra.run.smithi158.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 198/202 2024-02-18T11:29:42.967 INFO:teuthology.orchestra.run.smithi158.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 199/202 2024-02-18T11:29:42.992 INFO:teuthology.orchestra.run.smithi158.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 200/202 2024-02-18T11:29:43.025 INFO:teuthology.orchestra.run.smithi158.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 201/202 2024-02-18T11:29:43.090 INFO:teuthology.orchestra.run.smithi158.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 202/202 2024-02-18T11:29:44.972 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-test-2:19.0.0-1555.g1c920afa.el9.x86_64 154/202 2024-02-18T11:29:45.012 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : rbd-fuse-2:19.0.0-1555.g1c920afa.el9.x86_64 155/202 2024-02-18T11:29:45.066 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : rbd-nbd-2:19.0.0-1555.g1c920afa.el9.x86_64 156/202 2024-02-18T11:29:45.096 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 157/202 2024-02-18T11:29:45.275 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : cephfs-shell-2:19.0.0-1555.g1c920afa.el9.x86_64 158/202 2024-02-18T11:29:45.357 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-test-2:19.0.0-1555.g1c920afa.el9.x86_64 154/202 2024-02-18T11:29:45.398 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : rbd-fuse-2:19.0.0-1555.g1c920afa.el9.x86_64 155/202 2024-02-18T11:29:45.419 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 159/202 2024-02-18T11:29:45.430 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : rbd-nbd-2:19.0.0-1555.g1c920afa.el9.x86_64 156/202 2024-02-18T11:29:45.464 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 157/202 2024-02-18T11:29:45.468 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-fuse-2:19.0.0-1555.g1c920afa.el9.x86_64 160/202 2024-02-18T11:29:45.516 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 161/202 2024-02-18T11:29:45.586 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 162/202 2024-02-18T11:29:45.652 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : cephfs-shell-2:19.0.0-1555.g1c920afa.el9.x86_64 158/202 2024-02-18T11:29:45.672 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 163/202 2024-02-18T11:29:45.712 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : iproute-tc-6.7.0-2.el9.x86_64 164/202 2024-02-18T11:29:45.756 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 165/202 2024-02-18T11:29:45.766 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 159/202 2024-02-18T11:29:45.787 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libcephfs-devel-2:19.0.0-1555.g1c920afa.el9.x86_ 166/202 2024-02-18T11:29:45.812 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-fuse-2:19.0.0-1555.g1c920afa.el9.x86_64 160/202 2024-02-18T11:29:45.816 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : NetworkManager-team-1:1.45.91-1.el9.x86_64 167/202 2024-02-18T11:29:45.857 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 161/202 2024-02-18T11:29:45.889 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : NetworkManager-tui-1:1.45.91-1.el9.x86_64 168/202 2024-02-18T11:29:45.911 INFO:teuthology.orchestra.run.smithi012.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 162/202 2024-02-18T11:29:45.944 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 169/202 2024-02-18T11:29:45.982 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 170/202 2024-02-18T11:29:45.994 INFO:teuthology.orchestra.run.smithi012.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 163/202 2024-02-18T11:29:46.038 INFO:teuthology.orchestra.run.smithi012.stdout: Upgrading : iproute-tc-6.7.0-2.el9.x86_64 164/202 2024-02-18T11:29:46.081 INFO:teuthology.orchestra.run.smithi012.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 165/202 2024-02-18T11:29:46.112 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libcephfs-devel-2:19.0.0-1555.g1c920afa.el9.x86_ 166/202 2024-02-18T11:29:46.141 INFO:teuthology.orchestra.run.smithi012.stdout: Upgrading : NetworkManager-team-1:1.45.91-1.el9.x86_64 167/202 2024-02-18T11:29:46.225 INFO:teuthology.orchestra.run.smithi012.stdout: Upgrading : NetworkManager-tui-1:1.45.91-1.el9.x86_64 168/202 2024-02-18T11:29:46.288 INFO:teuthology.orchestra.run.smithi012.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 169/202 2024-02-18T11:29:46.323 INFO:teuthology.orchestra.run.smithi012.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 170/202 2024-02-18T11:29:46.584 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 171/202 2024-02-18T11:29:46.622 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 171/202 2024-02-18T11:29:46.641 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 171/202 2024-02-18T11:29:46.953 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 171/202 2024-02-18T11:29:46.996 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 171/202 2024-02-18T11:29:47.016 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 171/202 2024-02-18T11:29:47.455 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : cephfs-top-2:19.0.0-1555.g1c920afa.el9.noarch 172/202 2024-02-18T11:29:47.526 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 173/202 2024-02-18T11:29:47.566 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 174/202 2024-02-18T11:29:47.648 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : flex-2.6.4-9.el9.x86_64 175/202 2024-02-18T11:29:47.688 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : bison-3.7.4-5.el9.x86_64 176/202 2024-02-18T11:29:47.713 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 177/202 2024-02-18T11:29:47.738 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 178/202 2024-02-18T11:29:47.765 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 179/202 2024-02-18T11:29:47.765 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 180/202 2024-02-18T11:29:47.849 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 180/202 2024-02-18T11:29:47.877 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 181/202 2024-02-18T11:29:47.879 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : cephfs-top-2:19.0.0-1555.g1c920afa.el9.noarch 172/202 2024-02-18T11:29:47.908 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 182/202 2024-02-18T11:29:47.908 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 183/202 2024-02-18T11:29:47.950 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: openssl-libs-1:3.0.7-24.el9.x86_64 202/202 2024-02-18T11:29:47.950 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : ceph-2:19.0.0-1555.g1c920afa.el9.x86_64 1/202 2024-02-18T11:29:47.951 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : ceph-base-2:19.0.0-1555.g1c920afa.el9.x86_64 2/202 2024-02-18T11:29:47.951 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : ceph-common-2:19.0.0-1555.g1c920afa.el9.x86_64 3/202 2024-02-18T11:29:47.951 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : ceph-fuse-2:19.0.0-1555.g1c920afa.el9.x86_64 4/202 2024-02-18T11:29:47.951 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1555.g1c920 5/202 2024-02-18T11:29:47.951 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : ceph-mds-2:19.0.0-1555.g1c920afa.el9.x86_64 6/202 2024-02-18T11:29:47.951 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : ceph-mgr-2:19.0.0-1555.g1c920afa.el9.x86_64 7/202 2024-02-18T11:29:47.951 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : ceph-mon-2:19.0.0-1555.g1c920afa.el9.x86_64 8/202 2024-02-18T11:29:47.951 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : ceph-osd-2:19.0.0-1555.g1c920afa.el9.x86_64 9/202 2024-02-18T11:29:47.951 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : ceph-radosgw-2:19.0.0-1555.g1c920afa.el9.x86_64 10/202 2024-02-18T11:29:47.951 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : ceph-selinux-2:19.0.0-1555.g1c920afa.el9.x86_64 11/202 2024-02-18T11:29:47.951 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : ceph-test-2:19.0.0-1555.g1c920afa.el9.x86_64 12/202 2024-02-18T11:29:47.951 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : cephfs-mirror-2:19.0.0-1555.g1c920afa.el9.x86_64 13/202 2024-02-18T11:29:47.951 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : cephfs-shell-2:19.0.0-1555.g1c920afa.el9.x86_64 14/202 2024-02-18T11:29:47.951 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libcephfs-devel-2:19.0.0-1555.g1c920afa.el9.x86_ 15/202 2024-02-18T11:29:47.952 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libcephfs2-2:19.0.0-1555.g1c920afa.el9.x86_64 16/202 2024-02-18T11:29:47.952 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libcephsqlite-2:19.0.0-1555.g1c920afa.el9.x86_64 17/202 2024-02-18T11:29:47.952 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : librados-devel-2:19.0.0-1555.g1c920afa.el9.x86_6 18/202 2024-02-18T11:29:47.952 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libradosstriper1-2:19.0.0-1555.g1c920afa.el9.x86 19/202 2024-02-18T11:29:47.952 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : librgw2-2:19.0.0-1555.g1c920afa.el9.x86_64 20/202 2024-02-18T11:29:47.952 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1555.g1c920afa.el 21/202 2024-02-18T11:29:47.952 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-ceph-common-2:19.0.0-1555.g1c920afa.el9. 22/202 2024-02-18T11:29:47.952 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-cephfs-2:19.0.0-1555.g1c920afa.el9.x86_6 23/202 2024-02-18T11:29:47.952 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-rados-2:19.0.0-1555.g1c920afa.el9.x86_64 24/202 2024-02-18T11:29:47.952 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-rbd-2:19.0.0-1555.g1c920afa.el9.x86_64 25/202 2024-02-18T11:29:47.952 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-rgw-2:19.0.0-1555.g1c920afa.el9.x86_64 26/202 2024-02-18T11:29:47.952 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : rbd-fuse-2:19.0.0-1555.g1c920afa.el9.x86_64 27/202 2024-02-18T11:29:47.952 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : rbd-mirror-2:19.0.0-1555.g1c920afa.el9.x86_64 28/202 2024-02-18T11:29:47.953 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : rbd-nbd-2:19.0.0-1555.g1c920afa.el9.x86_64 29/202 2024-02-18T11:29:47.953 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1555.g1c920afa. 30/202 2024-02-18T11:29:47.953 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1555.g1c920afa.el9.noa 31/202 2024-02-18T11:29:47.953 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1555.g1c920afa.el9.n 32/202 2024-02-18T11:29:47.953 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1555.g1c9 33/202 2024-02-18T11:29:47.953 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1555.g1c920afa.el 34/202 2024-02-18T11:29:47.953 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1555.g1c920afa.el9.noarch 35/202 2024-02-18T11:29:47.953 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1555.g1c920afa.e 36/202 2024-02-18T11:29:47.953 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : ceph-volume-2:19.0.0-1555.g1c920afa.el9.noarch 37/202 2024-02-18T11:29:47.953 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : cephadm-2:19.0.0-1555.g1c920afa.el9.noarch 38/202 2024-02-18T11:29:47.953 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : cephfs-top-2:19.0.0-1555.g1c920afa.el9.noarch 39/202 2024-02-18T11:29:47.953 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/202 2024-02-18T11:29:47.953 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/202 2024-02-18T11:29:47.953 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/202 2024-02-18T11:29:47.954 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 43/202 2024-02-18T11:29:47.954 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : flex-2.6.4-9.el9.x86_64 44/202 2024-02-18T11:29:47.954 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 45/202 2024-02-18T11:29:47.954 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 46/202 2024-02-18T11:29:47.954 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 47/202 2024-02-18T11:29:47.954 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 48/202 2024-02-18T11:29:47.954 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 49/202 2024-02-18T11:29:47.954 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 50/202 2024-02-18T11:29:47.954 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 51/202 2024-02-18T11:29:47.954 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 52/202 2024-02-18T11:29:47.954 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 53/202 2024-02-18T11:29:47.954 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 54/202 2024-02-18T11:29:47.954 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 55/202 2024-02-18T11:29:47.954 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : lua-5.4.4-4.el9.x86_64 56/202 2024-02-18T11:29:47.955 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 57/202 2024-02-18T11:29:47.955 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 58/202 2024-02-18T11:29:47.955 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 59/202 2024-02-18T11:29:47.955 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : postgresql-13.11-1.el9.x86_64 60/202 2024-02-18T11:29:47.955 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 61/202 2024-02-18T11:29:47.955 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 62/202 2024-02-18T11:29:47.955 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 63/202 2024-02-18T11:29:47.955 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 64/202 2024-02-18T11:29:47.955 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 65/202 2024-02-18T11:29:47.955 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 66/202 2024-02-18T11:29:47.955 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 67/202 2024-02-18T11:29:47.955 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 68/202 2024-02-18T11:29:47.955 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 69/202 2024-02-18T11:29:47.955 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 70/202 2024-02-18T11:29:47.955 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 71/202 2024-02-18T11:29:47.956 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 72/202 2024-02-18T11:29:47.956 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 73/202 2024-02-18T11:29:47.956 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 74/202 2024-02-18T11:29:47.956 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 75/202 2024-02-18T11:29:47.956 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 76/202 2024-02-18T11:29:47.956 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 77/202 2024-02-18T11:29:47.956 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 78/202 2024-02-18T11:29:47.956 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 79/202 2024-02-18T11:29:47.956 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 80/202 2024-02-18T11:29:47.956 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 81/202 2024-02-18T11:29:47.956 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 82/202 2024-02-18T11:29:47.956 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 83/202 2024-02-18T11:29:47.956 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 84/202 2024-02-18T11:29:47.956 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 85/202 2024-02-18T11:29:47.957 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 86/202 2024-02-18T11:29:47.957 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 87/202 2024-02-18T11:29:47.957 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 88/202 2024-02-18T11:29:47.957 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 89/202 2024-02-18T11:29:47.957 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 90/202 2024-02-18T11:29:47.957 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 91/202 2024-02-18T11:29:47.957 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 92/202 2024-02-18T11:29:47.957 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 93/202 2024-02-18T11:29:47.957 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : jq-1.6-16.el9.x86_64 94/202 2024-02-18T11:29:47.957 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 95/202 2024-02-18T11:29:47.957 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 96/202 2024-02-18T11:29:47.957 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 97/202 2024-02-18T11:29:47.957 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 98/202 2024-02-18T11:29:47.957 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 99/202 2024-02-18T11:29:47.958 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-ply-3.11-14.el9.noarch 100/202 2024-02-18T11:29:47.958 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 101/202 2024-02-18T11:29:47.958 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 102/202 2024-02-18T11:29:47.958 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 103/202 2024-02-18T11:29:47.958 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 104/202 2024-02-18T11:29:47.958 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/202 2024-02-18T11:29:47.958 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/202 2024-02-18T11:29:47.958 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/202 2024-02-18T11:29:47.958 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/202 2024-02-18T11:29:47.958 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/202 2024-02-18T11:29:47.958 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/202 2024-02-18T11:29:47.958 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/202 2024-02-18T11:29:47.958 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/202 2024-02-18T11:29:47.958 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/202 2024-02-18T11:29:47.959 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/202 2024-02-18T11:29:47.959 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/202 2024-02-18T11:29:47.959 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/202 2024-02-18T11:29:47.959 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/202 2024-02-18T11:29:47.959 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/202 2024-02-18T11:29:47.959 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/202 2024-02-18T11:29:47.959 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/202 2024-02-18T11:29:47.959 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/202 2024-02-18T11:29:47.959 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/202 2024-02-18T11:29:47.959 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/202 2024-02-18T11:29:47.959 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-google-auth-1:2.27.0-1.el9.noarch 124/202 2024-02-18T11:29:47.959 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/202 2024-02-18T11:29:47.959 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/202 2024-02-18T11:29:47.959 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/202 2024-02-18T11:29:47.959 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/202 2024-02-18T11:29:47.959 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/202 2024-02-18T11:29:47.960 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/202 2024-02-18T11:29:47.960 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/202 2024-02-18T11:29:47.960 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/202 2024-02-18T11:29:47.960 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/202 2024-02-18T11:29:47.960 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/202 2024-02-18T11:29:47.960 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/202 2024-02-18T11:29:47.960 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/202 2024-02-18T11:29:47.960 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/202 2024-02-18T11:29:47.960 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/202 2024-02-18T11:29:47.960 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/202 2024-02-18T11:29:47.960 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/202 2024-02-18T11:29:47.960 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/202 2024-02-18T11:29:47.960 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/202 2024-02-18T11:29:47.960 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/202 2024-02-18T11:29:47.961 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/202 2024-02-18T11:29:47.961 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/202 2024-02-18T11:29:47.961 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/202 2024-02-18T11:29:47.961 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/202 2024-02-18T11:29:47.961 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/202 2024-02-18T11:29:47.961 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/202 2024-02-18T11:29:47.961 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/202 2024-02-18T11:29:47.961 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : librados2-2:19.0.0-1555.g1c920afa.el9.x86_64 151/202 2024-02-18T11:29:47.961 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/202 2024-02-18T11:29:47.961 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : librbd1-2:19.0.0-1555.g1c920afa.el9.x86_64 153/202 2024-02-18T11:29:47.961 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/202 2024-02-18T11:29:47.961 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 155/202 2024-02-18T11:29:47.961 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 156/202 2024-02-18T11:29:47.961 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : NetworkManager-1:1.45.91-1.el9.x86_64 157/202 2024-02-18T11:29:47.962 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/202 2024-02-18T11:29:47.962 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : NetworkManager-libnm-1:1.45.91-1.el9.x86_64 159/202 2024-02-18T11:29:47.962 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/202 2024-02-18T11:29:47.962 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : NetworkManager-team-1:1.45.91-1.el9.x86_64 161/202 2024-02-18T11:29:47.962 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/202 2024-02-18T11:29:47.962 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : NetworkManager-tui-1:1.45.91-1.el9.x86_64 163/202 2024-02-18T11:29:47.962 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/202 2024-02-18T11:29:47.962 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/202 2024-02-18T11:29:47.962 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/202 2024-02-18T11:29:47.962 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/202 2024-02-18T11:29:47.962 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/202 2024-02-18T11:29:47.962 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/202 2024-02-18T11:29:47.962 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/202 2024-02-18T11:29:47.962 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/202 2024-02-18T11:29:47.962 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/202 2024-02-18T11:29:47.963 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : iproute-6.7.0-2.el9.x86_64 173/202 2024-02-18T11:29:47.963 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : iproute-6.2.0-5.el9.x86_64 174/202 2024-02-18T11:29:47.963 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : iproute-tc-6.7.0-2.el9.x86_64 175/202 2024-02-18T11:29:47.963 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : iproute-tc-6.2.0-5.el9.x86_64 176/202 2024-02-18T11:29:47.963 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 177/202 2024-02-18T11:29:47.963 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 178/202 2024-02-18T11:29:47.963 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 179/202 2024-02-18T11:29:47.963 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 180/202 2024-02-18T11:29:47.963 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 181/202 2024-02-18T11:29:47.963 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 182/202 2024-02-18T11:29:47.963 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 183/202 2024-02-18T11:29:47.963 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 184/202 2024-02-18T11:29:47.963 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 185/202 2024-02-18T11:29:47.963 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 186/202 2024-02-18T11:29:47.964 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 187/202 2024-02-18T11:29:47.964 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 188/202 2024-02-18T11:29:47.964 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 189/202 2024-02-18T11:29:47.964 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 190/202 2024-02-18T11:29:47.964 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 191/202 2024-02-18T11:29:47.964 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 192/202 2024-02-18T11:29:47.964 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 193/202 2024-02-18T11:29:47.964 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 194/202 2024-02-18T11:29:47.964 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 195/202 2024-02-18T11:29:47.964 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 196/202 2024-02-18T11:29:47.964 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 197/202 2024-02-18T11:29:47.964 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 198/202 2024-02-18T11:29:47.964 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 199/202 2024-02-18T11:29:47.964 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 200/202 2024-02-18T11:29:47.965 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 201/202 2024-02-18T11:29:47.965 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 173/202 2024-02-18T11:29:47.984 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 183/202 2024-02-18T11:29:47.993 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 174/202 2024-02-18T11:29:48.019 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 184/202 2024-02-18T11:29:48.083 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : flex-2.6.4-9.el9.x86_64 175/202 2024-02-18T11:29:48.113 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : bison-3.7.4-5.el9.x86_64 176/202 2024-02-18T11:29:48.128 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 185/202 2024-02-18T11:29:48.138 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 177/202 2024-02-18T11:29:48.153 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : iproute-tc-6.2.0-5.el9.x86_64 186/202 2024-02-18T11:29:48.163 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 178/202 2024-02-18T11:29:48.180 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 187/202 2024-02-18T11:29:48.189 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 179/202 2024-02-18T11:29:48.190 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 180/202 2024-02-18T11:29:48.212 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : iproute-6.2.0-5.el9.x86_64 188/202 2024-02-18T11:29:48.245 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 189/202 2024-02-18T11:29:48.274 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 180/202 2024-02-18T11:29:48.285 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 190/202 2024-02-18T11:29:48.302 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 181/202 2024-02-18T11:29:48.326 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 191/202 2024-02-18T11:29:48.332 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 182/202 2024-02-18T11:29:48.332 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 183/202 2024-02-18T11:29:48.351 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 192/202 2024-02-18T11:29:48.377 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 193/202 2024-02-18T11:29:48.402 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 194/202 2024-02-18T11:29:48.417 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 183/202 2024-02-18T11:29:48.427 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 195/202 2024-02-18T11:29:48.433 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 196/202 2024-02-18T11:29:48.433 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 196/202 2024-02-18T11:29:48.443 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 184/202 2024-02-18T11:29:48.477 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 185/202 2024-02-18T11:29:48.502 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : iproute-tc-6.2.0-5.el9.x86_64 186/202 2024-02-18T11:29:48.529 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 187/202 2024-02-18T11:29:48.561 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : iproute-6.2.0-5.el9.x86_64 188/202 2024-02-18T11:29:48.595 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 189/202 2024-02-18T11:29:48.613 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 196/202 2024-02-18T11:29:48.613 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 197/202 2024-02-18T11:29:48.651 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 190/202 2024-02-18T11:29:48.689 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 197/202 2024-02-18T11:29:48.691 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 191/202 2024-02-18T11:29:48.713 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 198/202 2024-02-18T11:29:48.716 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 192/202 2024-02-18T11:29:48.738 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 199/202 2024-02-18T11:29:48.742 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 193/202 2024-02-18T11:29:48.763 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 200/202 2024-02-18T11:29:48.767 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 194/202 2024-02-18T11:29:48.784 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 195/202 2024-02-18T11:29:48.791 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 196/202 2024-02-18T11:29:48.791 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 196/202 2024-02-18T11:29:48.797 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 201/202 2024-02-18T11:29:48.886 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 202/202 2024-02-18T11:29:48.979 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 196/202 2024-02-18T11:29:48.979 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 197/202 2024-02-18T11:29:49.053 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 197/202 2024-02-18T11:29:49.078 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 198/202 2024-02-18T11:29:49.103 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 199/202 2024-02-18T11:29:49.129 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 200/202 2024-02-18T11:29:49.162 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 201/202 2024-02-18T11:29:49.232 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 202/202 2024-02-18T11:29:50.862 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 202/202 2024-02-18T11:29:50.862 INFO:teuthology.orchestra.run.smithi158.stdout: 2024-02-18T11:29:50.862 INFO:teuthology.orchestra.run.smithi158.stdout:Upgraded: 2024-02-18T11:29:50.863 INFO:teuthology.orchestra.run.smithi158.stdout: NetworkManager-1:1.45.91-1.el9.x86_64 2024-02-18T11:29:50.863 INFO:teuthology.orchestra.run.smithi158.stdout: NetworkManager-libnm-1:1.45.91-1.el9.x86_64 2024-02-18T11:29:50.863 INFO:teuthology.orchestra.run.smithi158.stdout: NetworkManager-team-1:1.45.91-1.el9.x86_64 2024-02-18T11:29:50.863 INFO:teuthology.orchestra.run.smithi158.stdout: NetworkManager-tui-1:1.45.91-1.el9.x86_64 2024-02-18T11:29:50.863 INFO:teuthology.orchestra.run.smithi158.stdout: acl-2.3.1-4.el9.x86_64 2024-02-18T11:29:50.863 INFO:teuthology.orchestra.run.smithi158.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-02-18T11:29:50.863 INFO:teuthology.orchestra.run.smithi158.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-02-18T11:29:50.863 INFO:teuthology.orchestra.run.smithi158.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-02-18T11:29:50.863 INFO:teuthology.orchestra.run.smithi158.stdout: iproute-6.7.0-2.el9.x86_64 2024-02-18T11:29:50.863 INFO:teuthology.orchestra.run.smithi158.stdout: iproute-tc-6.7.0-2.el9.x86_64 2024-02-18T11:29:50.863 INFO:teuthology.orchestra.run.smithi158.stdout: libacl-2.3.1-4.el9.x86_64 2024-02-18T11:29:50.863 INFO:teuthology.orchestra.run.smithi158.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-02-18T11:29:50.863 INFO:teuthology.orchestra.run.smithi158.stdout: libblkid-2.37.4-18.el9.x86_64 2024-02-18T11:29:50.863 INFO:teuthology.orchestra.run.smithi158.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-02-18T11:29:50.864 INFO:teuthology.orchestra.run.smithi158.stdout: libmount-2.37.4-18.el9.x86_64 2024-02-18T11:29:50.864 INFO:teuthology.orchestra.run.smithi158.stdout: librados2-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:50.864 INFO:teuthology.orchestra.run.smithi158.stdout: librbd1-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:50.864 INFO:teuthology.orchestra.run.smithi158.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-02-18T11:29:50.864 INFO:teuthology.orchestra.run.smithi158.stdout: libuuid-2.37.4-18.el9.x86_64 2024-02-18T11:29:50.864 INFO:teuthology.orchestra.run.smithi158.stdout: openssl-1:3.0.7-25.el9.x86_64 2024-02-18T11:29:50.864 INFO:teuthology.orchestra.run.smithi158.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2024-02-18T11:29:50.864 INFO:teuthology.orchestra.run.smithi158.stdout: util-linux-2.37.4-18.el9.x86_64 2024-02-18T11:29:50.864 INFO:teuthology.orchestra.run.smithi158.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-02-18T11:29:50.864 INFO:teuthology.orchestra.run.smithi158.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-02-18T11:29:50.864 INFO:teuthology.orchestra.run.smithi158.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-02-18T11:29:50.864 INFO:teuthology.orchestra.run.smithi158.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-02-18T11:29:50.864 INFO:teuthology.orchestra.run.smithi158.stdout:Installed: 2024-02-18T11:29:50.864 INFO:teuthology.orchestra.run.smithi158.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-02-18T11:29:50.865 INFO:teuthology.orchestra.run.smithi158.stdout: automake-1.16.2-8.el9.noarch 2024-02-18T11:29:50.865 INFO:teuthology.orchestra.run.smithi158.stdout: bison-3.7.4-5.el9.x86_64 2024-02-18T11:29:50.865 INFO:teuthology.orchestra.run.smithi158.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-02-18T11:29:50.865 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:50.865 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-base-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:50.865 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-common-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:50.865 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-fuse-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:50.865 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-grafana-dashboards-2:19.0.0-1555.g1c920afa.el9.noarch 2024-02-18T11:29:50.865 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-immutable-object-cache-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:50.865 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-mds-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:50.865 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-mgr-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:50.865 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-mgr-cephadm-2:19.0.0-1555.g1c920afa.el9.noarch 2024-02-18T11:29:50.865 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-mgr-dashboard-2:19.0.0-1555.g1c920afa.el9.noarch 2024-02-18T11:29:50.865 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1555.g1c920afa.el9.noarch 2024-02-18T11:29:50.866 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-mgr-modules-core-2:19.0.0-1555.g1c920afa.el9.noarch 2024-02-18T11:29:50.866 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-mgr-rook-2:19.0.0-1555.g1c920afa.el9.noarch 2024-02-18T11:29:50.866 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-mon-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:50.866 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-osd-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:50.866 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-prometheus-alerts-2:19.0.0-1555.g1c920afa.el9.noarch 2024-02-18T11:29:50.866 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-radosgw-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:50.866 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-selinux-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:50.866 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-test-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:50.866 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-volume-2:19.0.0-1555.g1c920afa.el9.noarch 2024-02-18T11:29:50.866 INFO:teuthology.orchestra.run.smithi158.stdout: cephadm-2:19.0.0-1555.g1c920afa.el9.noarch 2024-02-18T11:29:50.866 INFO:teuthology.orchestra.run.smithi158.stdout: cephfs-mirror-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:50.866 INFO:teuthology.orchestra.run.smithi158.stdout: cephfs-shell-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:50.866 INFO:teuthology.orchestra.run.smithi158.stdout: cephfs-top-2:19.0.0-1555.g1c920afa.el9.noarch 2024-02-18T11:29:50.866 INFO:teuthology.orchestra.run.smithi158.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-02-18T11:29:50.866 INFO:teuthology.orchestra.run.smithi158.stdout: flex-2.6.4-9.el9.x86_64 2024-02-18T11:29:50.867 INFO:teuthology.orchestra.run.smithi158.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-02-18T11:29:50.867 INFO:teuthology.orchestra.run.smithi158.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-02-18T11:29:50.867 INFO:teuthology.orchestra.run.smithi158.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-02-18T11:29:50.867 INFO:teuthology.orchestra.run.smithi158.stdout: fuse-2.9.9-15.el9.x86_64 2024-02-18T11:29:50.867 INFO:teuthology.orchestra.run.smithi158.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-02-18T11:29:50.867 INFO:teuthology.orchestra.run.smithi158.stdout: grpc-data-1.46.7-10.el9.noarch 2024-02-18T11:29:50.867 INFO:teuthology.orchestra.run.smithi158.stdout: jq-1.6-16.el9.x86_64 2024-02-18T11:29:50.867 INFO:teuthology.orchestra.run.smithi158.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-02-18T11:29:50.867 INFO:teuthology.orchestra.run.smithi158.stdout: libarrow-9.0.0-10.el9.x86_64 2024-02-18T11:29:50.867 INFO:teuthology.orchestra.run.smithi158.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-02-18T11:29:50.867 INFO:teuthology.orchestra.run.smithi158.stdout: libcephfs-devel-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:50.867 INFO:teuthology.orchestra.run.smithi158.stdout: libcephfs2-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:50.867 INFO:teuthology.orchestra.run.smithi158.stdout: libcephsqlite-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:50.867 INFO:teuthology.orchestra.run.smithi158.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-02-18T11:29:50.867 INFO:teuthology.orchestra.run.smithi158.stdout: liboath-2.6.7-2.el9.x86_64 2024-02-18T11:29:50.868 INFO:teuthology.orchestra.run.smithi158.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-02-18T11:29:50.868 INFO:teuthology.orchestra.run.smithi158.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-02-18T11:29:50.868 INFO:teuthology.orchestra.run.smithi158.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-02-18T11:29:50.868 INFO:teuthology.orchestra.run.smithi158.stdout: librados-devel-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:50.868 INFO:teuthology.orchestra.run.smithi158.stdout: libradosstriper1-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:50.868 INFO:teuthology.orchestra.run.smithi158.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-02-18T11:29:50.868 INFO:teuthology.orchestra.run.smithi158.stdout: librgw2-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:50.868 INFO:teuthology.orchestra.run.smithi158.stdout: libtool-2.4.6-45.el9.x86_64 2024-02-18T11:29:50.868 INFO:teuthology.orchestra.run.smithi158.stdout: libunwind-1.6.2-1.el9.x86_64 2024-02-18T11:29:50.868 INFO:teuthology.orchestra.run.smithi158.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-02-18T11:29:50.868 INFO:teuthology.orchestra.run.smithi158.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-02-18T11:29:50.868 INFO:teuthology.orchestra.run.smithi158.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-02-18T11:29:50.868 INFO:teuthology.orchestra.run.smithi158.stdout: lua-5.4.4-4.el9.x86_64 2024-02-18T11:29:50.868 INFO:teuthology.orchestra.run.smithi158.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-02-18T11:29:50.869 INFO:teuthology.orchestra.run.smithi158.stdout: luarocks-3.9.2-1.el9.noarch 2024-02-18T11:29:50.869 INFO:teuthology.orchestra.run.smithi158.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-02-18T11:29:50.869 INFO:teuthology.orchestra.run.smithi158.stdout: openblas-0.3.21-2.el9.x86_64 2024-02-18T11:29:50.869 INFO:teuthology.orchestra.run.smithi158.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-02-18T11:29:50.869 INFO:teuthology.orchestra.run.smithi158.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2024-02-18T11:29:50.869 INFO:teuthology.orchestra.run.smithi158.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-02-18T11:29:50.869 INFO:teuthology.orchestra.run.smithi158.stdout: postgresql-13.11-1.el9.x86_64 2024-02-18T11:29:50.869 INFO:teuthology.orchestra.run.smithi158.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-02-18T11:29:50.869 INFO:teuthology.orchestra.run.smithi158.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-02-18T11:29:50.869 INFO:teuthology.orchestra.run.smithi158.stdout: postgresql-server-13.11-1.el9.x86_64 2024-02-18T11:29:50.869 INFO:teuthology.orchestra.run.smithi158.stdout: protobuf-3.14.0-13.el9.x86_64 2024-02-18T11:29:50.869 INFO:teuthology.orchestra.run.smithi158.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-02-18T11:29:50.869 INFO:teuthology.orchestra.run.smithi158.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-02-18T11:29:50.869 INFO:teuthology.orchestra.run.smithi158.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-02-18T11:29:50.869 INFO:teuthology.orchestra.run.smithi158.stdout: python3-babel-2.9.1-2.el9.noarch 2024-02-18T11:29:50.870 INFO:teuthology.orchestra.run.smithi158.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-02-18T11:29:50.870 INFO:teuthology.orchestra.run.smithi158.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-02-18T11:29:50.870 INFO:teuthology.orchestra.run.smithi158.stdout: python3-ceph-argparse-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:50.870 INFO:teuthology.orchestra.run.smithi158.stdout: python3-ceph-common-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:50.870 INFO:teuthology.orchestra.run.smithi158.stdout: python3-cephfs-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:50.870 INFO:teuthology.orchestra.run.smithi158.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-02-18T11:29:50.870 INFO:teuthology.orchestra.run.smithi158.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-02-18T11:29:50.870 INFO:teuthology.orchestra.run.smithi158.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-02-18T11:29:50.870 INFO:teuthology.orchestra.run.smithi158.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-02-18T11:29:50.870 INFO:teuthology.orchestra.run.smithi158.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-02-18T11:29:50.870 INFO:teuthology.orchestra.run.smithi158.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-02-18T11:29:50.870 INFO:teuthology.orchestra.run.smithi158.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-02-18T11:29:50.870 INFO:teuthology.orchestra.run.smithi158.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-02-18T11:29:50.870 INFO:teuthology.orchestra.run.smithi158.stdout: python3-google-auth-1:2.27.0-1.el9.noarch 2024-02-18T11:29:50.871 INFO:teuthology.orchestra.run.smithi158.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-02-18T11:29:50.871 INFO:teuthology.orchestra.run.smithi158.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-02-18T11:29:50.871 INFO:teuthology.orchestra.run.smithi158.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-02-18T11:29:50.871 INFO:teuthology.orchestra.run.smithi158.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-02-18T11:29:50.871 INFO:teuthology.orchestra.run.smithi158.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-02-18T11:29:50.871 INFO:teuthology.orchestra.run.smithi158.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-02-18T11:29:50.871 INFO:teuthology.orchestra.run.smithi158.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-02-18T11:29:50.871 INFO:teuthology.orchestra.run.smithi158.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-02-18T11:29:50.871 INFO:teuthology.orchestra.run.smithi158.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-02-18T11:29:50.871 INFO:teuthology.orchestra.run.smithi158.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-02-18T11:29:50.871 INFO:teuthology.orchestra.run.smithi158.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-02-18T11:29:50.871 INFO:teuthology.orchestra.run.smithi158.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-02-18T11:29:50.871 INFO:teuthology.orchestra.run.smithi158.stdout: python3-mako-1.1.4-6.el9.noarch 2024-02-18T11:29:50.871 INFO:teuthology.orchestra.run.smithi158.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-02-18T11:29:50.871 INFO:teuthology.orchestra.run.smithi158.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-02-18T11:29:50.872 INFO:teuthology.orchestra.run.smithi158.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-02-18T11:29:50.872 INFO:teuthology.orchestra.run.smithi158.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-02-18T11:29:50.872 INFO:teuthology.orchestra.run.smithi158.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-02-18T11:29:50.872 INFO:teuthology.orchestra.run.smithi158.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-02-18T11:29:50.872 INFO:teuthology.orchestra.run.smithi158.stdout: python3-packaging-20.9-5.el9.noarch 2024-02-18T11:29:50.872 INFO:teuthology.orchestra.run.smithi158.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-02-18T11:29:50.872 INFO:teuthology.orchestra.run.smithi158.stdout: python3-ply-3.11-14.el9.noarch 2024-02-18T11:29:50.872 INFO:teuthology.orchestra.run.smithi158.stdout: python3-portend-3.1.0-2.el9.noarch 2024-02-18T11:29:50.872 INFO:teuthology.orchestra.run.smithi158.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-02-18T11:29:50.872 INFO:teuthology.orchestra.run.smithi158.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-02-18T11:29:50.872 INFO:teuthology.orchestra.run.smithi158.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-02-18T11:29:50.872 INFO:teuthology.orchestra.run.smithi158.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-02-18T11:29:50.872 INFO:teuthology.orchestra.run.smithi158.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-02-18T11:29:50.872 INFO:teuthology.orchestra.run.smithi158.stdout: python3-pycparser-2.20-6.el9.noarch 2024-02-18T11:29:50.873 INFO:teuthology.orchestra.run.smithi158.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-02-18T11:29:50.873 INFO:teuthology.orchestra.run.smithi158.stdout: python3-pytz-2021.1-5.el9.noarch 2024-02-18T11:29:50.873 INFO:teuthology.orchestra.run.smithi158.stdout: python3-rados-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:50.873 INFO:teuthology.orchestra.run.smithi158.stdout: python3-rbd-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:50.873 INFO:teuthology.orchestra.run.smithi158.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-02-18T11:29:50.873 INFO:teuthology.orchestra.run.smithi158.stdout: python3-requests-2.25.1-8.el9.noarch 2024-02-18T11:29:50.873 INFO:teuthology.orchestra.run.smithi158.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-02-18T11:29:50.873 INFO:teuthology.orchestra.run.smithi158.stdout: python3-rgw-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:50.873 INFO:teuthology.orchestra.run.smithi158.stdout: python3-routes-2.5.1-5.el9.noarch 2024-02-18T11:29:50.873 INFO:teuthology.orchestra.run.smithi158.stdout: python3-rsa-4.9-2.el9.noarch 2024-02-18T11:29:50.873 INFO:teuthology.orchestra.run.smithi158.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-02-18T11:29:50.873 INFO:teuthology.orchestra.run.smithi158.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-02-18T11:29:50.873 INFO:teuthology.orchestra.run.smithi158.stdout: python3-toml-0.10.2-6.el9.noarch 2024-02-18T11:29:50.873 INFO:teuthology.orchestra.run.smithi158.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-02-18T11:29:50.873 INFO:teuthology.orchestra.run.smithi158.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-02-18T11:29:50.873 INFO:teuthology.orchestra.run.smithi158.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-02-18T11:29:50.874 INFO:teuthology.orchestra.run.smithi158.stdout: python3-webob-1.8.7-6.el9.noarch 2024-02-18T11:29:50.874 INFO:teuthology.orchestra.run.smithi158.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-02-18T11:29:50.874 INFO:teuthology.orchestra.run.smithi158.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-02-18T11:29:50.874 INFO:teuthology.orchestra.run.smithi158.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-02-18T11:29:50.874 INFO:teuthology.orchestra.run.smithi158.stdout: qatlib-23.11.0-1.el9.x86_64 2024-02-18T11:29:50.874 INFO:teuthology.orchestra.run.smithi158.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-02-18T11:29:50.874 INFO:teuthology.orchestra.run.smithi158.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-02-18T11:29:50.874 INFO:teuthology.orchestra.run.smithi158.stdout: rbd-fuse-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:50.874 INFO:teuthology.orchestra.run.smithi158.stdout: rbd-mirror-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:50.874 INFO:teuthology.orchestra.run.smithi158.stdout: rbd-nbd-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:50.874 INFO:teuthology.orchestra.run.smithi158.stdout: re2-1:20211101-3.el9.x86_64 2024-02-18T11:29:50.874 INFO:teuthology.orchestra.run.smithi158.stdout: socat-1.7.4.1-5.el9.x86_64 2024-02-18T11:29:50.874 INFO:teuthology.orchestra.run.smithi158.stdout: thrift-0.15.0-2.el9.x86_64 2024-02-18T11:29:50.874 INFO:teuthology.orchestra.run.smithi158.stdout: uuid-1.6.2-55.el9.x86_64 2024-02-18T11:29:50.874 INFO:teuthology.orchestra.run.smithi158.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-02-18T11:29:50.875 INFO:teuthology.orchestra.run.smithi158.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-02-18T11:29:50.875 INFO:teuthology.orchestra.run.smithi158.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-02-18T11:29:50.875 INFO:teuthology.orchestra.run.smithi158.stdout: 2024-02-18T11:29:50.875 INFO:teuthology.orchestra.run.smithi158.stdout:Complete! 2024-02-18T11:29:51.466 DEBUG:teuthology.parallel:result is None 2024-02-18T11:29:53.885 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: openssl-libs-1:3.0.7-24.el9.x86_64 202/202 2024-02-18T11:29:53.885 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-2:19.0.0-1555.g1c920afa.el9.x86_64 1/202 2024-02-18T11:29:53.885 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-base-2:19.0.0-1555.g1c920afa.el9.x86_64 2/202 2024-02-18T11:29:53.885 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-common-2:19.0.0-1555.g1c920afa.el9.x86_64 3/202 2024-02-18T11:29:53.885 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-fuse-2:19.0.0-1555.g1c920afa.el9.x86_64 4/202 2024-02-18T11:29:53.885 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1555.g1c920 5/202 2024-02-18T11:29:53.885 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mds-2:19.0.0-1555.g1c920afa.el9.x86_64 6/202 2024-02-18T11:29:53.885 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-2:19.0.0-1555.g1c920afa.el9.x86_64 7/202 2024-02-18T11:29:53.885 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mon-2:19.0.0-1555.g1c920afa.el9.x86_64 8/202 2024-02-18T11:29:53.886 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-osd-2:19.0.0-1555.g1c920afa.el9.x86_64 9/202 2024-02-18T11:29:53.886 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-radosgw-2:19.0.0-1555.g1c920afa.el9.x86_64 10/202 2024-02-18T11:29:53.886 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-selinux-2:19.0.0-1555.g1c920afa.el9.x86_64 11/202 2024-02-18T11:29:53.886 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-test-2:19.0.0-1555.g1c920afa.el9.x86_64 12/202 2024-02-18T11:29:53.886 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : cephfs-mirror-2:19.0.0-1555.g1c920afa.el9.x86_64 13/202 2024-02-18T11:29:53.886 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : cephfs-shell-2:19.0.0-1555.g1c920afa.el9.x86_64 14/202 2024-02-18T11:29:53.886 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libcephfs-devel-2:19.0.0-1555.g1c920afa.el9.x86_ 15/202 2024-02-18T11:29:53.886 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libcephfs2-2:19.0.0-1555.g1c920afa.el9.x86_64 16/202 2024-02-18T11:29:53.886 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libcephsqlite-2:19.0.0-1555.g1c920afa.el9.x86_64 17/202 2024-02-18T11:29:53.886 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librados-devel-2:19.0.0-1555.g1c920afa.el9.x86_6 18/202 2024-02-18T11:29:53.886 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libradosstriper1-2:19.0.0-1555.g1c920afa.el9.x86 19/202 2024-02-18T11:29:53.886 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librgw2-2:19.0.0-1555.g1c920afa.el9.x86_64 20/202 2024-02-18T11:29:53.887 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1555.g1c920afa.el 21/202 2024-02-18T11:29:53.887 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-ceph-common-2:19.0.0-1555.g1c920afa.el9. 22/202 2024-02-18T11:29:53.887 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cephfs-2:19.0.0-1555.g1c920afa.el9.x86_6 23/202 2024-02-18T11:29:53.887 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-rados-2:19.0.0-1555.g1c920afa.el9.x86_64 24/202 2024-02-18T11:29:53.887 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-rbd-2:19.0.0-1555.g1c920afa.el9.x86_64 25/202 2024-02-18T11:29:53.887 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-rgw-2:19.0.0-1555.g1c920afa.el9.x86_64 26/202 2024-02-18T11:29:53.887 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : rbd-fuse-2:19.0.0-1555.g1c920afa.el9.x86_64 27/202 2024-02-18T11:29:53.887 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : rbd-mirror-2:19.0.0-1555.g1c920afa.el9.x86_64 28/202 2024-02-18T11:29:53.888 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : rbd-nbd-2:19.0.0-1555.g1c920afa.el9.x86_64 29/202 2024-02-18T11:29:53.904 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1555.g1c920afa. 30/202 2024-02-18T11:29:53.905 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1555.g1c920afa.el9.noa 31/202 2024-02-18T11:29:53.905 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1555.g1c920afa.el9.n 32/202 2024-02-18T11:29:53.905 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1555.g1c9 33/202 2024-02-18T11:29:53.905 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1555.g1c920afa.el 34/202 2024-02-18T11:29:53.905 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1555.g1c920afa.el9.noarch 35/202 2024-02-18T11:29:53.905 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1555.g1c920afa.e 36/202 2024-02-18T11:29:53.905 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-volume-2:19.0.0-1555.g1c920afa.el9.noarch 37/202 2024-02-18T11:29:53.905 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : cephadm-2:19.0.0-1555.g1c920afa.el9.noarch 38/202 2024-02-18T11:29:53.905 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : cephfs-top-2:19.0.0-1555.g1c920afa.el9.noarch 39/202 2024-02-18T11:29:53.905 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/202 2024-02-18T11:29:53.905 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/202 2024-02-18T11:29:53.905 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/202 2024-02-18T11:29:53.905 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 43/202 2024-02-18T11:29:53.905 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : flex-2.6.4-9.el9.x86_64 44/202 2024-02-18T11:29:53.905 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 45/202 2024-02-18T11:29:53.906 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 46/202 2024-02-18T11:29:53.906 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 47/202 2024-02-18T11:29:53.906 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 48/202 2024-02-18T11:29:53.906 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 49/202 2024-02-18T11:29:53.906 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 50/202 2024-02-18T11:29:53.906 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 51/202 2024-02-18T11:29:53.906 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 52/202 2024-02-18T11:29:53.906 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 53/202 2024-02-18T11:29:53.906 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 54/202 2024-02-18T11:29:53.906 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 55/202 2024-02-18T11:29:53.906 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : lua-5.4.4-4.el9.x86_64 56/202 2024-02-18T11:29:53.906 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 57/202 2024-02-18T11:29:53.906 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 58/202 2024-02-18T11:29:53.906 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 59/202 2024-02-18T11:29:53.906 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : postgresql-13.11-1.el9.x86_64 60/202 2024-02-18T11:29:53.907 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 61/202 2024-02-18T11:29:53.907 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 62/202 2024-02-18T11:29:53.907 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 63/202 2024-02-18T11:29:53.907 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 64/202 2024-02-18T11:29:53.907 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 65/202 2024-02-18T11:29:53.907 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 66/202 2024-02-18T11:29:53.907 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 67/202 2024-02-18T11:29:53.907 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 68/202 2024-02-18T11:29:53.907 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 69/202 2024-02-18T11:29:53.907 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 70/202 2024-02-18T11:29:53.907 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 71/202 2024-02-18T11:29:53.907 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 72/202 2024-02-18T11:29:53.907 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 73/202 2024-02-18T11:29:53.907 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 74/202 2024-02-18T11:29:53.908 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 75/202 2024-02-18T11:29:53.908 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 76/202 2024-02-18T11:29:53.908 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 77/202 2024-02-18T11:29:53.908 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 78/202 2024-02-18T11:29:53.908 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 79/202 2024-02-18T11:29:53.908 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 80/202 2024-02-18T11:29:53.908 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 81/202 2024-02-18T11:29:53.908 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 82/202 2024-02-18T11:29:53.908 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 83/202 2024-02-18T11:29:53.908 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 84/202 2024-02-18T11:29:53.908 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 85/202 2024-02-18T11:29:53.908 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 86/202 2024-02-18T11:29:53.908 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 87/202 2024-02-18T11:29:53.908 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 88/202 2024-02-18T11:29:53.908 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 89/202 2024-02-18T11:29:53.909 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 90/202 2024-02-18T11:29:53.909 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 91/202 2024-02-18T11:29:53.909 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 92/202 2024-02-18T11:29:53.909 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 93/202 2024-02-18T11:29:53.909 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : jq-1.6-16.el9.x86_64 94/202 2024-02-18T11:29:53.909 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 95/202 2024-02-18T11:29:53.909 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 96/202 2024-02-18T11:29:53.909 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 97/202 2024-02-18T11:29:53.909 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 98/202 2024-02-18T11:29:53.909 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 99/202 2024-02-18T11:29:53.909 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-ply-3.11-14.el9.noarch 100/202 2024-02-18T11:29:53.909 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 101/202 2024-02-18T11:29:53.909 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 102/202 2024-02-18T11:29:53.909 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 103/202 2024-02-18T11:29:53.909 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 104/202 2024-02-18T11:29:53.910 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/202 2024-02-18T11:29:53.910 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/202 2024-02-18T11:29:53.910 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/202 2024-02-18T11:29:53.910 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/202 2024-02-18T11:29:53.910 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/202 2024-02-18T11:29:53.910 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/202 2024-02-18T11:29:53.910 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/202 2024-02-18T11:29:53.910 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/202 2024-02-18T11:29:53.910 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/202 2024-02-18T11:29:53.910 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/202 2024-02-18T11:29:53.910 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/202 2024-02-18T11:29:53.910 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/202 2024-02-18T11:29:53.910 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/202 2024-02-18T11:29:53.910 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/202 2024-02-18T11:29:53.910 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/202 2024-02-18T11:29:53.911 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/202 2024-02-18T11:29:53.911 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/202 2024-02-18T11:29:53.911 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/202 2024-02-18T11:29:53.911 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/202 2024-02-18T11:29:53.911 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-google-auth-1:2.27.0-1.el9.noarch 124/202 2024-02-18T11:29:53.911 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/202 2024-02-18T11:29:53.911 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/202 2024-02-18T11:29:53.911 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/202 2024-02-18T11:29:53.911 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/202 2024-02-18T11:29:53.911 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/202 2024-02-18T11:29:53.911 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/202 2024-02-18T11:29:53.911 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/202 2024-02-18T11:29:53.911 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/202 2024-02-18T11:29:53.911 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/202 2024-02-18T11:29:53.911 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/202 2024-02-18T11:29:53.912 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/202 2024-02-18T11:29:53.912 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/202 2024-02-18T11:29:53.912 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/202 2024-02-18T11:29:53.912 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/202 2024-02-18T11:29:53.912 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/202 2024-02-18T11:29:53.912 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/202 2024-02-18T11:29:53.912 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/202 2024-02-18T11:29:53.912 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/202 2024-02-18T11:29:53.912 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/202 2024-02-18T11:29:53.912 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/202 2024-02-18T11:29:53.912 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/202 2024-02-18T11:29:53.912 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/202 2024-02-18T11:29:53.912 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/202 2024-02-18T11:29:53.913 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/202 2024-02-18T11:29:53.913 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/202 2024-02-18T11:29:53.913 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/202 2024-02-18T11:29:53.913 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librados2-2:19.0.0-1555.g1c920afa.el9.x86_64 151/202 2024-02-18T11:29:53.913 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/202 2024-02-18T11:29:53.913 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librbd1-2:19.0.0-1555.g1c920afa.el9.x86_64 153/202 2024-02-18T11:29:53.913 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/202 2024-02-18T11:29:53.913 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 155/202 2024-02-18T11:29:53.913 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 156/202 2024-02-18T11:29:53.913 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-1:1.45.91-1.el9.x86_64 157/202 2024-02-18T11:29:53.913 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/202 2024-02-18T11:29:53.913 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-libnm-1:1.45.91-1.el9.x86_64 159/202 2024-02-18T11:29:53.913 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/202 2024-02-18T11:29:53.914 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-team-1:1.45.91-1.el9.x86_64 161/202 2024-02-18T11:29:53.914 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/202 2024-02-18T11:29:53.914 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-tui-1:1.45.91-1.el9.x86_64 163/202 2024-02-18T11:29:53.914 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/202 2024-02-18T11:29:53.914 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/202 2024-02-18T11:29:53.914 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/202 2024-02-18T11:29:53.914 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/202 2024-02-18T11:29:53.914 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/202 2024-02-18T11:29:53.914 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/202 2024-02-18T11:29:53.914 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/202 2024-02-18T11:29:53.914 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/202 2024-02-18T11:29:53.914 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/202 2024-02-18T11:29:53.914 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : iproute-6.7.0-2.el9.x86_64 173/202 2024-02-18T11:29:53.914 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : iproute-6.2.0-5.el9.x86_64 174/202 2024-02-18T11:29:53.914 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : iproute-tc-6.7.0-2.el9.x86_64 175/202 2024-02-18T11:29:53.915 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : iproute-tc-6.2.0-5.el9.x86_64 176/202 2024-02-18T11:29:53.915 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 177/202 2024-02-18T11:29:53.915 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 178/202 2024-02-18T11:29:53.915 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 179/202 2024-02-18T11:29:53.915 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 180/202 2024-02-18T11:29:53.915 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 181/202 2024-02-18T11:29:53.915 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 182/202 2024-02-18T11:29:53.915 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 183/202 2024-02-18T11:29:53.915 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 184/202 2024-02-18T11:29:53.915 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 185/202 2024-02-18T11:29:53.915 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 186/202 2024-02-18T11:29:53.915 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 187/202 2024-02-18T11:29:53.915 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 188/202 2024-02-18T11:29:53.915 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 189/202 2024-02-18T11:29:53.915 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 190/202 2024-02-18T11:29:53.916 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 191/202 2024-02-18T11:29:53.916 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 192/202 2024-02-18T11:29:53.916 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 193/202 2024-02-18T11:29:53.916 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 194/202 2024-02-18T11:29:53.916 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 195/202 2024-02-18T11:29:53.916 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 196/202 2024-02-18T11:29:53.916 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 197/202 2024-02-18T11:29:53.916 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 198/202 2024-02-18T11:29:53.916 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 199/202 2024-02-18T11:29:53.916 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 200/202 2024-02-18T11:29:53.916 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 201/202 2024-02-18T11:29:54.121 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: openssl-libs-1:3.0.7-24.el9.x86_64 202/202 2024-02-18T11:29:54.121 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-2:19.0.0-1555.g1c920afa.el9.x86_64 1/202 2024-02-18T11:29:54.121 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-base-2:19.0.0-1555.g1c920afa.el9.x86_64 2/202 2024-02-18T11:29:54.122 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-common-2:19.0.0-1555.g1c920afa.el9.x86_64 3/202 2024-02-18T11:29:54.122 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-fuse-2:19.0.0-1555.g1c920afa.el9.x86_64 4/202 2024-02-18T11:29:54.122 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1555.g1c920 5/202 2024-02-18T11:29:54.122 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-mds-2:19.0.0-1555.g1c920afa.el9.x86_64 6/202 2024-02-18T11:29:54.122 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-mgr-2:19.0.0-1555.g1c920afa.el9.x86_64 7/202 2024-02-18T11:29:54.122 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-mon-2:19.0.0-1555.g1c920afa.el9.x86_64 8/202 2024-02-18T11:29:54.122 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-osd-2:19.0.0-1555.g1c920afa.el9.x86_64 9/202 2024-02-18T11:29:54.122 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-radosgw-2:19.0.0-1555.g1c920afa.el9.x86_64 10/202 2024-02-18T11:29:54.122 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-selinux-2:19.0.0-1555.g1c920afa.el9.x86_64 11/202 2024-02-18T11:29:54.122 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-test-2:19.0.0-1555.g1c920afa.el9.x86_64 12/202 2024-02-18T11:29:54.122 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : cephfs-mirror-2:19.0.0-1555.g1c920afa.el9.x86_64 13/202 2024-02-18T11:29:54.122 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : cephfs-shell-2:19.0.0-1555.g1c920afa.el9.x86_64 14/202 2024-02-18T11:29:54.122 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libcephfs-devel-2:19.0.0-1555.g1c920afa.el9.x86_ 15/202 2024-02-18T11:29:54.122 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libcephfs2-2:19.0.0-1555.g1c920afa.el9.x86_64 16/202 2024-02-18T11:29:54.122 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libcephsqlite-2:19.0.0-1555.g1c920afa.el9.x86_64 17/202 2024-02-18T11:29:54.123 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : librados-devel-2:19.0.0-1555.g1c920afa.el9.x86_6 18/202 2024-02-18T11:29:54.123 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libradosstriper1-2:19.0.0-1555.g1c920afa.el9.x86 19/202 2024-02-18T11:29:54.123 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : librgw2-2:19.0.0-1555.g1c920afa.el9.x86_64 20/202 2024-02-18T11:29:54.123 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1555.g1c920afa.el 21/202 2024-02-18T11:29:54.123 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-ceph-common-2:19.0.0-1555.g1c920afa.el9. 22/202 2024-02-18T11:29:54.123 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-cephfs-2:19.0.0-1555.g1c920afa.el9.x86_6 23/202 2024-02-18T11:29:54.123 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-rados-2:19.0.0-1555.g1c920afa.el9.x86_64 24/202 2024-02-18T11:29:54.123 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-rbd-2:19.0.0-1555.g1c920afa.el9.x86_64 25/202 2024-02-18T11:29:54.123 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-rgw-2:19.0.0-1555.g1c920afa.el9.x86_64 26/202 2024-02-18T11:29:54.123 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : rbd-fuse-2:19.0.0-1555.g1c920afa.el9.x86_64 27/202 2024-02-18T11:29:54.123 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : rbd-mirror-2:19.0.0-1555.g1c920afa.el9.x86_64 28/202 2024-02-18T11:29:54.123 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : rbd-nbd-2:19.0.0-1555.g1c920afa.el9.x86_64 29/202 2024-02-18T11:29:54.123 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1555.g1c920afa. 30/202 2024-02-18T11:29:54.123 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1555.g1c920afa.el9.noa 31/202 2024-02-18T11:29:54.124 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1555.g1c920afa.el9.n 32/202 2024-02-18T11:29:54.124 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1555.g1c9 33/202 2024-02-18T11:29:54.124 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1555.g1c920afa.el 34/202 2024-02-18T11:29:54.124 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1555.g1c920afa.el9.noarch 35/202 2024-02-18T11:29:54.124 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1555.g1c920afa.e 36/202 2024-02-18T11:29:54.124 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-volume-2:19.0.0-1555.g1c920afa.el9.noarch 37/202 2024-02-18T11:29:54.124 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : cephadm-2:19.0.0-1555.g1c920afa.el9.noarch 38/202 2024-02-18T11:29:54.124 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : cephfs-top-2:19.0.0-1555.g1c920afa.el9.noarch 39/202 2024-02-18T11:29:54.124 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/202 2024-02-18T11:29:54.124 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/202 2024-02-18T11:29:54.124 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/202 2024-02-18T11:29:54.124 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 43/202 2024-02-18T11:29:54.124 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : flex-2.6.4-9.el9.x86_64 44/202 2024-02-18T11:29:54.124 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 45/202 2024-02-18T11:29:54.125 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 46/202 2024-02-18T11:29:54.125 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 47/202 2024-02-18T11:29:54.125 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 48/202 2024-02-18T11:29:54.125 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 49/202 2024-02-18T11:29:54.125 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 50/202 2024-02-18T11:29:54.125 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 51/202 2024-02-18T11:29:54.125 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 52/202 2024-02-18T11:29:54.125 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 53/202 2024-02-18T11:29:54.125 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 54/202 2024-02-18T11:29:54.125 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 55/202 2024-02-18T11:29:54.125 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : lua-5.4.4-4.el9.x86_64 56/202 2024-02-18T11:29:54.125 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 57/202 2024-02-18T11:29:54.125 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 58/202 2024-02-18T11:29:54.125 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 59/202 2024-02-18T11:29:54.126 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : postgresql-13.11-1.el9.x86_64 60/202 2024-02-18T11:29:54.126 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 61/202 2024-02-18T11:29:54.126 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 62/202 2024-02-18T11:29:54.126 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 63/202 2024-02-18T11:29:54.126 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 64/202 2024-02-18T11:29:54.126 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 65/202 2024-02-18T11:29:54.126 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 66/202 2024-02-18T11:29:54.126 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 67/202 2024-02-18T11:29:54.126 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 68/202 2024-02-18T11:29:54.126 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 69/202 2024-02-18T11:29:54.126 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 70/202 2024-02-18T11:29:54.126 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 71/202 2024-02-18T11:29:54.126 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 72/202 2024-02-18T11:29:54.126 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 73/202 2024-02-18T11:29:54.126 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 74/202 2024-02-18T11:29:54.127 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 75/202 2024-02-18T11:29:54.127 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 76/202 2024-02-18T11:29:54.127 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 77/202 2024-02-18T11:29:54.127 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 78/202 2024-02-18T11:29:54.127 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 79/202 2024-02-18T11:29:54.127 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 80/202 2024-02-18T11:29:54.127 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 81/202 2024-02-18T11:29:54.127 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 82/202 2024-02-18T11:29:54.127 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 83/202 2024-02-18T11:29:54.127 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 84/202 2024-02-18T11:29:54.127 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 85/202 2024-02-18T11:29:54.127 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 86/202 2024-02-18T11:29:54.127 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 87/202 2024-02-18T11:29:54.127 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 88/202 2024-02-18T11:29:54.128 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 89/202 2024-02-18T11:29:54.128 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 90/202 2024-02-18T11:29:54.128 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 91/202 2024-02-18T11:29:54.128 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 92/202 2024-02-18T11:29:54.128 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 93/202 2024-02-18T11:29:54.128 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : jq-1.6-16.el9.x86_64 94/202 2024-02-18T11:29:54.128 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 95/202 2024-02-18T11:29:54.128 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 96/202 2024-02-18T11:29:54.128 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 97/202 2024-02-18T11:29:54.128 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 98/202 2024-02-18T11:29:54.128 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 99/202 2024-02-18T11:29:54.128 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-ply-3.11-14.el9.noarch 100/202 2024-02-18T11:29:54.128 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 101/202 2024-02-18T11:29:54.128 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 102/202 2024-02-18T11:29:54.129 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 103/202 2024-02-18T11:29:54.129 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 104/202 2024-02-18T11:29:54.129 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/202 2024-02-18T11:29:54.129 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/202 2024-02-18T11:29:54.129 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/202 2024-02-18T11:29:54.129 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/202 2024-02-18T11:29:54.129 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/202 2024-02-18T11:29:54.129 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/202 2024-02-18T11:29:54.129 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/202 2024-02-18T11:29:54.129 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/202 2024-02-18T11:29:54.129 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/202 2024-02-18T11:29:54.129 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/202 2024-02-18T11:29:54.129 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/202 2024-02-18T11:29:54.129 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/202 2024-02-18T11:29:54.130 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/202 2024-02-18T11:29:54.130 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/202 2024-02-18T11:29:54.130 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/202 2024-02-18T11:29:54.130 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/202 2024-02-18T11:29:54.130 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/202 2024-02-18T11:29:54.130 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/202 2024-02-18T11:29:54.130 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/202 2024-02-18T11:29:54.130 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-google-auth-1:2.27.0-1.el9.noarch 124/202 2024-02-18T11:29:54.130 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/202 2024-02-18T11:29:54.130 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/202 2024-02-18T11:29:54.130 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/202 2024-02-18T11:29:54.130 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/202 2024-02-18T11:29:54.130 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/202 2024-02-18T11:29:54.130 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/202 2024-02-18T11:29:54.130 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/202 2024-02-18T11:29:54.131 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/202 2024-02-18T11:29:54.131 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/202 2024-02-18T11:29:54.131 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/202 2024-02-18T11:29:54.131 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/202 2024-02-18T11:29:54.131 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/202 2024-02-18T11:29:54.131 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/202 2024-02-18T11:29:54.131 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/202 2024-02-18T11:29:54.131 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/202 2024-02-18T11:29:54.131 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/202 2024-02-18T11:29:54.131 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/202 2024-02-18T11:29:54.131 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/202 2024-02-18T11:29:54.131 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/202 2024-02-18T11:29:54.131 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/202 2024-02-18T11:29:54.131 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/202 2024-02-18T11:29:54.132 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/202 2024-02-18T11:29:54.132 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/202 2024-02-18T11:29:54.132 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/202 2024-02-18T11:29:54.132 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/202 2024-02-18T11:29:54.132 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/202 2024-02-18T11:29:54.132 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : librados2-2:19.0.0-1555.g1c920afa.el9.x86_64 151/202 2024-02-18T11:29:54.132 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/202 2024-02-18T11:29:54.132 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : librbd1-2:19.0.0-1555.g1c920afa.el9.x86_64 153/202 2024-02-18T11:29:54.132 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/202 2024-02-18T11:29:54.132 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 155/202 2024-02-18T11:29:54.132 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 156/202 2024-02-18T11:29:54.132 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : NetworkManager-1:1.45.91-1.el9.x86_64 157/202 2024-02-18T11:29:54.132 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/202 2024-02-18T11:29:54.132 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : NetworkManager-libnm-1:1.45.91-1.el9.x86_64 159/202 2024-02-18T11:29:54.133 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/202 2024-02-18T11:29:54.133 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : NetworkManager-team-1:1.45.91-1.el9.x86_64 161/202 2024-02-18T11:29:54.133 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/202 2024-02-18T11:29:54.133 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : NetworkManager-tui-1:1.45.91-1.el9.x86_64 163/202 2024-02-18T11:29:54.133 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/202 2024-02-18T11:29:54.133 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/202 2024-02-18T11:29:54.133 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/202 2024-02-18T11:29:54.133 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/202 2024-02-18T11:29:54.133 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/202 2024-02-18T11:29:54.133 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/202 2024-02-18T11:29:54.133 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/202 2024-02-18T11:29:54.133 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/202 2024-02-18T11:29:54.133 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/202 2024-02-18T11:29:54.133 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : iproute-6.7.0-2.el9.x86_64 173/202 2024-02-18T11:29:54.133 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : iproute-6.2.0-5.el9.x86_64 174/202 2024-02-18T11:29:54.134 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : iproute-tc-6.7.0-2.el9.x86_64 175/202 2024-02-18T11:29:54.134 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : iproute-tc-6.2.0-5.el9.x86_64 176/202 2024-02-18T11:29:54.134 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 177/202 2024-02-18T11:29:54.134 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 178/202 2024-02-18T11:29:54.134 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 179/202 2024-02-18T11:29:54.134 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 180/202 2024-02-18T11:29:54.134 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 181/202 2024-02-18T11:29:54.134 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 182/202 2024-02-18T11:29:54.134 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 183/202 2024-02-18T11:29:54.134 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 184/202 2024-02-18T11:29:54.134 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 185/202 2024-02-18T11:29:54.134 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 186/202 2024-02-18T11:29:54.134 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 187/202 2024-02-18T11:29:54.134 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 188/202 2024-02-18T11:29:54.134 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 189/202 2024-02-18T11:29:54.134 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 190/202 2024-02-18T11:29:54.135 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 191/202 2024-02-18T11:29:54.135 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 192/202 2024-02-18T11:29:54.135 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 193/202 2024-02-18T11:29:54.135 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 194/202 2024-02-18T11:29:54.135 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 195/202 2024-02-18T11:29:54.135 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 196/202 2024-02-18T11:29:54.135 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 197/202 2024-02-18T11:29:54.135 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 198/202 2024-02-18T11:29:54.135 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 199/202 2024-02-18T11:29:54.135 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 200/202 2024-02-18T11:29:54.135 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 201/202 2024-02-18T11:29:56.833 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 202/202 2024-02-18T11:29:56.834 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:29:56.834 INFO:teuthology.orchestra.run.smithi012.stdout:Upgraded: 2024-02-18T11:29:56.834 INFO:teuthology.orchestra.run.smithi012.stdout: NetworkManager-1:1.45.91-1.el9.x86_64 2024-02-18T11:29:56.834 INFO:teuthology.orchestra.run.smithi012.stdout: NetworkManager-libnm-1:1.45.91-1.el9.x86_64 2024-02-18T11:29:56.834 INFO:teuthology.orchestra.run.smithi012.stdout: NetworkManager-team-1:1.45.91-1.el9.x86_64 2024-02-18T11:29:56.834 INFO:teuthology.orchestra.run.smithi012.stdout: NetworkManager-tui-1:1.45.91-1.el9.x86_64 2024-02-18T11:29:56.835 INFO:teuthology.orchestra.run.smithi012.stdout: acl-2.3.1-4.el9.x86_64 2024-02-18T11:29:56.835 INFO:teuthology.orchestra.run.smithi012.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-02-18T11:29:56.835 INFO:teuthology.orchestra.run.smithi012.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-02-18T11:29:56.835 INFO:teuthology.orchestra.run.smithi012.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-02-18T11:29:56.835 INFO:teuthology.orchestra.run.smithi012.stdout: iproute-6.7.0-2.el9.x86_64 2024-02-18T11:29:56.835 INFO:teuthology.orchestra.run.smithi012.stdout: iproute-tc-6.7.0-2.el9.x86_64 2024-02-18T11:29:56.835 INFO:teuthology.orchestra.run.smithi012.stdout: libacl-2.3.1-4.el9.x86_64 2024-02-18T11:29:56.835 INFO:teuthology.orchestra.run.smithi012.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-02-18T11:29:56.835 INFO:teuthology.orchestra.run.smithi012.stdout: libblkid-2.37.4-18.el9.x86_64 2024-02-18T11:29:56.835 INFO:teuthology.orchestra.run.smithi012.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-02-18T11:29:56.835 INFO:teuthology.orchestra.run.smithi012.stdout: libmount-2.37.4-18.el9.x86_64 2024-02-18T11:29:56.835 INFO:teuthology.orchestra.run.smithi012.stdout: librados2-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:56.835 INFO:teuthology.orchestra.run.smithi012.stdout: librbd1-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:56.835 INFO:teuthology.orchestra.run.smithi012.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-02-18T11:29:56.836 INFO:teuthology.orchestra.run.smithi012.stdout: libuuid-2.37.4-18.el9.x86_64 2024-02-18T11:29:56.836 INFO:teuthology.orchestra.run.smithi012.stdout: openssl-1:3.0.7-25.el9.x86_64 2024-02-18T11:29:56.836 INFO:teuthology.orchestra.run.smithi012.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2024-02-18T11:29:56.836 INFO:teuthology.orchestra.run.smithi012.stdout: util-linux-2.37.4-18.el9.x86_64 2024-02-18T11:29:56.836 INFO:teuthology.orchestra.run.smithi012.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-02-18T11:29:56.836 INFO:teuthology.orchestra.run.smithi012.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-02-18T11:29:56.836 INFO:teuthology.orchestra.run.smithi012.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-02-18T11:29:56.836 INFO:teuthology.orchestra.run.smithi012.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-02-18T11:29:56.836 INFO:teuthology.orchestra.run.smithi012.stdout:Installed: 2024-02-18T11:29:56.836 INFO:teuthology.orchestra.run.smithi012.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-02-18T11:29:56.836 INFO:teuthology.orchestra.run.smithi012.stdout: automake-1.16.2-8.el9.noarch 2024-02-18T11:29:56.836 INFO:teuthology.orchestra.run.smithi012.stdout: bison-3.7.4-5.el9.x86_64 2024-02-18T11:29:56.836 INFO:teuthology.orchestra.run.smithi012.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-02-18T11:29:56.837 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:56.837 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-base-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:56.837 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-common-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:56.837 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-fuse-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:56.837 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-grafana-dashboards-2:19.0.0-1555.g1c920afa.el9.noarch 2024-02-18T11:29:56.837 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-immutable-object-cache-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:56.837 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mds-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:56.837 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:56.837 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-cephadm-2:19.0.0-1555.g1c920afa.el9.noarch 2024-02-18T11:29:56.837 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-dashboard-2:19.0.0-1555.g1c920afa.el9.noarch 2024-02-18T11:29:56.837 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1555.g1c920afa.el9.noarch 2024-02-18T11:29:56.837 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-modules-core-2:19.0.0-1555.g1c920afa.el9.noarch 2024-02-18T11:29:56.837 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-rook-2:19.0.0-1555.g1c920afa.el9.noarch 2024-02-18T11:29:56.837 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mon-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:56.838 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-osd-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:56.838 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-prometheus-alerts-2:19.0.0-1555.g1c920afa.el9.noarch 2024-02-18T11:29:56.838 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-radosgw-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:56.838 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-selinux-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:56.838 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-test-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:56.838 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-volume-2:19.0.0-1555.g1c920afa.el9.noarch 2024-02-18T11:29:56.838 INFO:teuthology.orchestra.run.smithi012.stdout: cephadm-2:19.0.0-1555.g1c920afa.el9.noarch 2024-02-18T11:29:56.838 INFO:teuthology.orchestra.run.smithi012.stdout: cephfs-mirror-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:56.838 INFO:teuthology.orchestra.run.smithi012.stdout: cephfs-shell-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:56.838 INFO:teuthology.orchestra.run.smithi012.stdout: cephfs-top-2:19.0.0-1555.g1c920afa.el9.noarch 2024-02-18T11:29:56.838 INFO:teuthology.orchestra.run.smithi012.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-02-18T11:29:56.838 INFO:teuthology.orchestra.run.smithi012.stdout: flex-2.6.4-9.el9.x86_64 2024-02-18T11:29:56.838 INFO:teuthology.orchestra.run.smithi012.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-02-18T11:29:56.839 INFO:teuthology.orchestra.run.smithi012.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-02-18T11:29:56.839 INFO:teuthology.orchestra.run.smithi012.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-02-18T11:29:56.839 INFO:teuthology.orchestra.run.smithi012.stdout: fuse-2.9.9-15.el9.x86_64 2024-02-18T11:29:56.839 INFO:teuthology.orchestra.run.smithi012.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-02-18T11:29:56.839 INFO:teuthology.orchestra.run.smithi012.stdout: grpc-data-1.46.7-10.el9.noarch 2024-02-18T11:29:56.839 INFO:teuthology.orchestra.run.smithi012.stdout: jq-1.6-16.el9.x86_64 2024-02-18T11:29:56.839 INFO:teuthology.orchestra.run.smithi012.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-02-18T11:29:56.839 INFO:teuthology.orchestra.run.smithi012.stdout: libarrow-9.0.0-10.el9.x86_64 2024-02-18T11:29:56.839 INFO:teuthology.orchestra.run.smithi012.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-02-18T11:29:56.839 INFO:teuthology.orchestra.run.smithi012.stdout: libcephfs-devel-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:56.839 INFO:teuthology.orchestra.run.smithi012.stdout: libcephfs2-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:56.839 INFO:teuthology.orchestra.run.smithi012.stdout: libcephsqlite-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:56.839 INFO:teuthology.orchestra.run.smithi012.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-02-18T11:29:56.839 INFO:teuthology.orchestra.run.smithi012.stdout: liboath-2.6.7-2.el9.x86_64 2024-02-18T11:29:56.839 INFO:teuthology.orchestra.run.smithi012.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-02-18T11:29:56.840 INFO:teuthology.orchestra.run.smithi012.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-02-18T11:29:56.840 INFO:teuthology.orchestra.run.smithi012.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-02-18T11:29:56.840 INFO:teuthology.orchestra.run.smithi012.stdout: librados-devel-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:56.840 INFO:teuthology.orchestra.run.smithi012.stdout: libradosstriper1-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:56.840 INFO:teuthology.orchestra.run.smithi012.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-02-18T11:29:56.840 INFO:teuthology.orchestra.run.smithi012.stdout: librgw2-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:56.840 INFO:teuthology.orchestra.run.smithi012.stdout: libtool-2.4.6-45.el9.x86_64 2024-02-18T11:29:56.840 INFO:teuthology.orchestra.run.smithi012.stdout: libunwind-1.6.2-1.el9.x86_64 2024-02-18T11:29:56.840 INFO:teuthology.orchestra.run.smithi012.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-02-18T11:29:56.840 INFO:teuthology.orchestra.run.smithi012.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-02-18T11:29:56.840 INFO:teuthology.orchestra.run.smithi012.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-02-18T11:29:56.840 INFO:teuthology.orchestra.run.smithi012.stdout: lua-5.4.4-4.el9.x86_64 2024-02-18T11:29:56.840 INFO:teuthology.orchestra.run.smithi012.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-02-18T11:29:56.840 INFO:teuthology.orchestra.run.smithi012.stdout: luarocks-3.9.2-1.el9.noarch 2024-02-18T11:29:56.841 INFO:teuthology.orchestra.run.smithi012.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-02-18T11:29:56.841 INFO:teuthology.orchestra.run.smithi012.stdout: openblas-0.3.21-2.el9.x86_64 2024-02-18T11:29:56.841 INFO:teuthology.orchestra.run.smithi012.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-02-18T11:29:56.841 INFO:teuthology.orchestra.run.smithi012.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2024-02-18T11:29:56.841 INFO:teuthology.orchestra.run.smithi012.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-02-18T11:29:56.841 INFO:teuthology.orchestra.run.smithi012.stdout: postgresql-13.11-1.el9.x86_64 2024-02-18T11:29:56.841 INFO:teuthology.orchestra.run.smithi012.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-02-18T11:29:56.841 INFO:teuthology.orchestra.run.smithi012.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-02-18T11:29:56.841 INFO:teuthology.orchestra.run.smithi012.stdout: postgresql-server-13.11-1.el9.x86_64 2024-02-18T11:29:56.841 INFO:teuthology.orchestra.run.smithi012.stdout: protobuf-3.14.0-13.el9.x86_64 2024-02-18T11:29:56.841 INFO:teuthology.orchestra.run.smithi012.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-02-18T11:29:56.842 INFO:teuthology.orchestra.run.smithi012.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-02-18T11:29:56.842 INFO:teuthology.orchestra.run.smithi012.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-02-18T11:29:56.842 INFO:teuthology.orchestra.run.smithi012.stdout: python3-babel-2.9.1-2.el9.noarch 2024-02-18T11:29:56.842 INFO:teuthology.orchestra.run.smithi012.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-02-18T11:29:56.842 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-02-18T11:29:56.842 INFO:teuthology.orchestra.run.smithi012.stdout: python3-ceph-argparse-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:56.842 INFO:teuthology.orchestra.run.smithi012.stdout: python3-ceph-common-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:56.842 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cephfs-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:56.842 INFO:teuthology.orchestra.run.smithi012.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-02-18T11:29:56.843 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-02-18T11:29:56.843 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-02-18T11:29:56.843 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-02-18T11:29:56.843 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-02-18T11:29:56.843 INFO:teuthology.orchestra.run.smithi012.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-02-18T11:29:56.843 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-02-18T11:29:56.843 INFO:teuthology.orchestra.run.smithi012.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-02-18T11:29:56.843 INFO:teuthology.orchestra.run.smithi012.stdout: python3-google-auth-1:2.27.0-1.el9.noarch 2024-02-18T11:29:56.843 INFO:teuthology.orchestra.run.smithi012.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-02-18T11:29:56.843 INFO:teuthology.orchestra.run.smithi012.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-02-18T11:29:56.843 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-02-18T11:29:56.843 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-02-18T11:29:56.844 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-02-18T11:29:56.844 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-02-18T11:29:56.844 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-02-18T11:29:56.844 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-02-18T11:29:56.844 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-02-18T11:29:56.844 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-02-18T11:29:56.844 INFO:teuthology.orchestra.run.smithi012.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-02-18T11:29:56.844 INFO:teuthology.orchestra.run.smithi012.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-02-18T11:29:56.844 INFO:teuthology.orchestra.run.smithi012.stdout: python3-mako-1.1.4-6.el9.noarch 2024-02-18T11:29:56.844 INFO:teuthology.orchestra.run.smithi012.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-02-18T11:29:56.845 INFO:teuthology.orchestra.run.smithi012.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-02-18T11:29:56.845 INFO:teuthology.orchestra.run.smithi012.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-02-18T11:29:56.845 INFO:teuthology.orchestra.run.smithi012.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-02-18T11:29:56.845 INFO:teuthology.orchestra.run.smithi012.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-02-18T11:29:56.845 INFO:teuthology.orchestra.run.smithi012.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-02-18T11:29:56.845 INFO:teuthology.orchestra.run.smithi012.stdout: python3-packaging-20.9-5.el9.noarch 2024-02-18T11:29:56.845 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-02-18T11:29:56.845 INFO:teuthology.orchestra.run.smithi012.stdout: python3-ply-3.11-14.el9.noarch 2024-02-18T11:29:56.845 INFO:teuthology.orchestra.run.smithi012.stdout: python3-portend-3.1.0-2.el9.noarch 2024-02-18T11:29:56.845 INFO:teuthology.orchestra.run.smithi012.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-02-18T11:29:56.845 INFO:teuthology.orchestra.run.smithi012.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-02-18T11:29:56.845 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-02-18T11:29:56.845 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-02-18T11:29:56.846 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-02-18T11:29:56.846 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pycparser-2.20-6.el9.noarch 2024-02-18T11:29:56.846 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-02-18T11:29:56.846 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pytz-2021.1-5.el9.noarch 2024-02-18T11:29:56.846 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rados-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:56.846 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rbd-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:56.846 INFO:teuthology.orchestra.run.smithi012.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-02-18T11:29:56.846 INFO:teuthology.orchestra.run.smithi012.stdout: python3-requests-2.25.1-8.el9.noarch 2024-02-18T11:29:56.846 INFO:teuthology.orchestra.run.smithi012.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-02-18T11:29:56.846 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rgw-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:56.846 INFO:teuthology.orchestra.run.smithi012.stdout: python3-routes-2.5.1-5.el9.noarch 2024-02-18T11:29:56.846 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rsa-4.9-2.el9.noarch 2024-02-18T11:29:56.846 INFO:teuthology.orchestra.run.smithi012.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-02-18T11:29:56.846 INFO:teuthology.orchestra.run.smithi012.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-02-18T11:29:56.847 INFO:teuthology.orchestra.run.smithi012.stdout: python3-toml-0.10.2-6.el9.noarch 2024-02-18T11:29:56.847 INFO:teuthology.orchestra.run.smithi012.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-02-18T11:29:56.847 INFO:teuthology.orchestra.run.smithi012.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-02-18T11:29:56.847 INFO:teuthology.orchestra.run.smithi012.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-02-18T11:29:56.847 INFO:teuthology.orchestra.run.smithi012.stdout: python3-webob-1.8.7-6.el9.noarch 2024-02-18T11:29:56.847 INFO:teuthology.orchestra.run.smithi012.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-02-18T11:29:56.847 INFO:teuthology.orchestra.run.smithi012.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-02-18T11:29:56.847 INFO:teuthology.orchestra.run.smithi012.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-02-18T11:29:56.847 INFO:teuthology.orchestra.run.smithi012.stdout: qatlib-23.11.0-1.el9.x86_64 2024-02-18T11:29:56.847 INFO:teuthology.orchestra.run.smithi012.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-02-18T11:29:56.847 INFO:teuthology.orchestra.run.smithi012.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-02-18T11:29:56.847 INFO:teuthology.orchestra.run.smithi012.stdout: rbd-fuse-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:56.848 INFO:teuthology.orchestra.run.smithi012.stdout: rbd-mirror-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:56.848 INFO:teuthology.orchestra.run.smithi012.stdout: rbd-nbd-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:56.848 INFO:teuthology.orchestra.run.smithi012.stdout: re2-1:20211101-3.el9.x86_64 2024-02-18T11:29:56.848 INFO:teuthology.orchestra.run.smithi012.stdout: socat-1.7.4.1-5.el9.x86_64 2024-02-18T11:29:56.848 INFO:teuthology.orchestra.run.smithi012.stdout: thrift-0.15.0-2.el9.x86_64 2024-02-18T11:29:56.848 INFO:teuthology.orchestra.run.smithi012.stdout: uuid-1.6.2-55.el9.x86_64 2024-02-18T11:29:56.848 INFO:teuthology.orchestra.run.smithi012.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-02-18T11:29:56.848 INFO:teuthology.orchestra.run.smithi012.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-02-18T11:29:56.848 INFO:teuthology.orchestra.run.smithi012.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-02-18T11:29:56.849 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:29:56.849 INFO:teuthology.orchestra.run.smithi012.stdout:Complete! 2024-02-18T11:29:56.857 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 202/202 2024-02-18T11:29:56.858 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-02-18T11:29:56.858 INFO:teuthology.orchestra.run.smithi188.stdout:Upgraded: 2024-02-18T11:29:56.858 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-1:1.45.91-1.el9.x86_64 2024-02-18T11:29:56.858 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-libnm-1:1.45.91-1.el9.x86_64 2024-02-18T11:29:56.858 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-team-1:1.45.91-1.el9.x86_64 2024-02-18T11:29:56.858 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-tui-1:1.45.91-1.el9.x86_64 2024-02-18T11:29:56.858 INFO:teuthology.orchestra.run.smithi188.stdout: acl-2.3.1-4.el9.x86_64 2024-02-18T11:29:56.859 INFO:teuthology.orchestra.run.smithi188.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-02-18T11:29:56.859 INFO:teuthology.orchestra.run.smithi188.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-02-18T11:29:56.859 INFO:teuthology.orchestra.run.smithi188.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-02-18T11:29:56.859 INFO:teuthology.orchestra.run.smithi188.stdout: iproute-6.7.0-2.el9.x86_64 2024-02-18T11:29:56.859 INFO:teuthology.orchestra.run.smithi188.stdout: iproute-tc-6.7.0-2.el9.x86_64 2024-02-18T11:29:56.859 INFO:teuthology.orchestra.run.smithi188.stdout: libacl-2.3.1-4.el9.x86_64 2024-02-18T11:29:56.859 INFO:teuthology.orchestra.run.smithi188.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-02-18T11:29:56.859 INFO:teuthology.orchestra.run.smithi188.stdout: libblkid-2.37.4-18.el9.x86_64 2024-02-18T11:29:56.860 INFO:teuthology.orchestra.run.smithi188.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-02-18T11:29:56.860 INFO:teuthology.orchestra.run.smithi188.stdout: libmount-2.37.4-18.el9.x86_64 2024-02-18T11:29:56.860 INFO:teuthology.orchestra.run.smithi188.stdout: librados2-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:56.860 INFO:teuthology.orchestra.run.smithi188.stdout: librbd1-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:56.860 INFO:teuthology.orchestra.run.smithi188.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-02-18T11:29:56.860 INFO:teuthology.orchestra.run.smithi188.stdout: libuuid-2.37.4-18.el9.x86_64 2024-02-18T11:29:56.860 INFO:teuthology.orchestra.run.smithi188.stdout: openssl-1:3.0.7-25.el9.x86_64 2024-02-18T11:29:56.860 INFO:teuthology.orchestra.run.smithi188.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2024-02-18T11:29:56.860 INFO:teuthology.orchestra.run.smithi188.stdout: util-linux-2.37.4-18.el9.x86_64 2024-02-18T11:29:56.860 INFO:teuthology.orchestra.run.smithi188.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-02-18T11:29:56.860 INFO:teuthology.orchestra.run.smithi188.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-02-18T11:29:56.861 INFO:teuthology.orchestra.run.smithi188.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-02-18T11:29:56.861 INFO:teuthology.orchestra.run.smithi188.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-02-18T11:29:56.861 INFO:teuthology.orchestra.run.smithi188.stdout:Installed: 2024-02-18T11:29:56.861 INFO:teuthology.orchestra.run.smithi188.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-02-18T11:29:56.861 INFO:teuthology.orchestra.run.smithi188.stdout: automake-1.16.2-8.el9.noarch 2024-02-18T11:29:56.861 INFO:teuthology.orchestra.run.smithi188.stdout: bison-3.7.4-5.el9.x86_64 2024-02-18T11:29:56.861 INFO:teuthology.orchestra.run.smithi188.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-02-18T11:29:56.861 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:56.861 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-base-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:56.861 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-common-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:56.861 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-fuse-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:56.861 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-grafana-dashboards-2:19.0.0-1555.g1c920afa.el9.noarch 2024-02-18T11:29:56.862 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-immutable-object-cache-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:56.862 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mds-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:56.862 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:56.862 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-cephadm-2:19.0.0-1555.g1c920afa.el9.noarch 2024-02-18T11:29:56.862 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-dashboard-2:19.0.0-1555.g1c920afa.el9.noarch 2024-02-18T11:29:56.862 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1555.g1c920afa.el9.noarch 2024-02-18T11:29:56.862 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-modules-core-2:19.0.0-1555.g1c920afa.el9.noarch 2024-02-18T11:29:56.862 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-rook-2:19.0.0-1555.g1c920afa.el9.noarch 2024-02-18T11:29:56.862 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mon-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:56.862 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-osd-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:56.862 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-prometheus-alerts-2:19.0.0-1555.g1c920afa.el9.noarch 2024-02-18T11:29:56.862 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-radosgw-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:56.862 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-selinux-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:56.862 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-test-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:56.862 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-volume-2:19.0.0-1555.g1c920afa.el9.noarch 2024-02-18T11:29:56.863 INFO:teuthology.orchestra.run.smithi188.stdout: cephadm-2:19.0.0-1555.g1c920afa.el9.noarch 2024-02-18T11:29:56.863 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-mirror-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:56.863 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-shell-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:56.863 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-top-2:19.0.0-1555.g1c920afa.el9.noarch 2024-02-18T11:29:56.863 INFO:teuthology.orchestra.run.smithi188.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-02-18T11:29:56.863 INFO:teuthology.orchestra.run.smithi188.stdout: flex-2.6.4-9.el9.x86_64 2024-02-18T11:29:56.863 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-02-18T11:29:56.863 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-02-18T11:29:56.863 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-02-18T11:29:56.863 INFO:teuthology.orchestra.run.smithi188.stdout: fuse-2.9.9-15.el9.x86_64 2024-02-18T11:29:56.863 INFO:teuthology.orchestra.run.smithi188.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-02-18T11:29:56.863 INFO:teuthology.orchestra.run.smithi188.stdout: grpc-data-1.46.7-10.el9.noarch 2024-02-18T11:29:56.863 INFO:teuthology.orchestra.run.smithi188.stdout: jq-1.6-16.el9.x86_64 2024-02-18T11:29:56.863 INFO:teuthology.orchestra.run.smithi188.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-02-18T11:29:56.864 INFO:teuthology.orchestra.run.smithi188.stdout: libarrow-9.0.0-10.el9.x86_64 2024-02-18T11:29:56.864 INFO:teuthology.orchestra.run.smithi188.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-02-18T11:29:56.864 INFO:teuthology.orchestra.run.smithi188.stdout: libcephfs-devel-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:56.864 INFO:teuthology.orchestra.run.smithi188.stdout: libcephfs2-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:56.864 INFO:teuthology.orchestra.run.smithi188.stdout: libcephsqlite-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:56.864 INFO:teuthology.orchestra.run.smithi188.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-02-18T11:29:56.864 INFO:teuthology.orchestra.run.smithi188.stdout: liboath-2.6.7-2.el9.x86_64 2024-02-18T11:29:56.864 INFO:teuthology.orchestra.run.smithi188.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-02-18T11:29:56.864 INFO:teuthology.orchestra.run.smithi188.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-02-18T11:29:56.864 INFO:teuthology.orchestra.run.smithi188.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-02-18T11:29:56.864 INFO:teuthology.orchestra.run.smithi188.stdout: librados-devel-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:56.864 INFO:teuthology.orchestra.run.smithi188.stdout: libradosstriper1-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:56.864 INFO:teuthology.orchestra.run.smithi188.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-02-18T11:29:56.865 INFO:teuthology.orchestra.run.smithi188.stdout: librgw2-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:56.865 INFO:teuthology.orchestra.run.smithi188.stdout: libtool-2.4.6-45.el9.x86_64 2024-02-18T11:29:56.865 INFO:teuthology.orchestra.run.smithi188.stdout: libunwind-1.6.2-1.el9.x86_64 2024-02-18T11:29:56.865 INFO:teuthology.orchestra.run.smithi188.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-02-18T11:29:56.865 INFO:teuthology.orchestra.run.smithi188.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-02-18T11:29:56.865 INFO:teuthology.orchestra.run.smithi188.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-02-18T11:29:56.865 INFO:teuthology.orchestra.run.smithi188.stdout: lua-5.4.4-4.el9.x86_64 2024-02-18T11:29:56.865 INFO:teuthology.orchestra.run.smithi188.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-02-18T11:29:56.865 INFO:teuthology.orchestra.run.smithi188.stdout: luarocks-3.9.2-1.el9.noarch 2024-02-18T11:29:56.865 INFO:teuthology.orchestra.run.smithi188.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-02-18T11:29:56.865 INFO:teuthology.orchestra.run.smithi188.stdout: openblas-0.3.21-2.el9.x86_64 2024-02-18T11:29:56.865 INFO:teuthology.orchestra.run.smithi188.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-02-18T11:29:56.865 INFO:teuthology.orchestra.run.smithi188.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2024-02-18T11:29:56.865 INFO:teuthology.orchestra.run.smithi188.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-02-18T11:29:56.865 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-13.11-1.el9.x86_64 2024-02-18T11:29:56.866 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-02-18T11:29:56.866 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-02-18T11:29:56.866 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-server-13.11-1.el9.x86_64 2024-02-18T11:29:56.866 INFO:teuthology.orchestra.run.smithi188.stdout: protobuf-3.14.0-13.el9.x86_64 2024-02-18T11:29:56.866 INFO:teuthology.orchestra.run.smithi188.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-02-18T11:29:56.866 INFO:teuthology.orchestra.run.smithi188.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-02-18T11:29:56.866 INFO:teuthology.orchestra.run.smithi188.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-02-18T11:29:56.866 INFO:teuthology.orchestra.run.smithi188.stdout: python3-babel-2.9.1-2.el9.noarch 2024-02-18T11:29:56.866 INFO:teuthology.orchestra.run.smithi188.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-02-18T11:29:56.866 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-02-18T11:29:56.867 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ceph-argparse-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:56.867 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ceph-common-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:56.867 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cephfs-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:56.867 INFO:teuthology.orchestra.run.smithi188.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-02-18T11:29:56.867 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-02-18T11:29:56.867 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-02-18T11:29:56.867 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-02-18T11:29:56.867 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-02-18T11:29:56.867 INFO:teuthology.orchestra.run.smithi188.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-02-18T11:29:56.867 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-02-18T11:29:56.867 INFO:teuthology.orchestra.run.smithi188.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-02-18T11:29:56.867 INFO:teuthology.orchestra.run.smithi188.stdout: python3-google-auth-1:2.27.0-1.el9.noarch 2024-02-18T11:29:56.867 INFO:teuthology.orchestra.run.smithi188.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-02-18T11:29:56.867 INFO:teuthology.orchestra.run.smithi188.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-02-18T11:29:56.867 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-02-18T11:29:56.868 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-02-18T11:29:56.868 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-02-18T11:29:56.868 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-02-18T11:29:56.868 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-02-18T11:29:56.868 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-02-18T11:29:56.868 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-02-18T11:29:56.868 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-02-18T11:29:56.868 INFO:teuthology.orchestra.run.smithi188.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-02-18T11:29:56.868 INFO:teuthology.orchestra.run.smithi188.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-02-18T11:29:56.868 INFO:teuthology.orchestra.run.smithi188.stdout: python3-mako-1.1.4-6.el9.noarch 2024-02-18T11:29:56.869 INFO:teuthology.orchestra.run.smithi188.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-02-18T11:29:56.869 INFO:teuthology.orchestra.run.smithi188.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-02-18T11:29:56.869 INFO:teuthology.orchestra.run.smithi188.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-02-18T11:29:56.869 INFO:teuthology.orchestra.run.smithi188.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-02-18T11:29:56.869 INFO:teuthology.orchestra.run.smithi188.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-02-18T11:29:56.869 INFO:teuthology.orchestra.run.smithi188.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-02-18T11:29:56.869 INFO:teuthology.orchestra.run.smithi188.stdout: python3-packaging-20.9-5.el9.noarch 2024-02-18T11:29:56.869 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-02-18T11:29:56.869 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ply-3.11-14.el9.noarch 2024-02-18T11:29:56.869 INFO:teuthology.orchestra.run.smithi188.stdout: python3-portend-3.1.0-2.el9.noarch 2024-02-18T11:29:56.869 INFO:teuthology.orchestra.run.smithi188.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-02-18T11:29:56.869 INFO:teuthology.orchestra.run.smithi188.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-02-18T11:29:56.869 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-02-18T11:29:56.870 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-02-18T11:29:56.870 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-02-18T11:29:56.870 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pycparser-2.20-6.el9.noarch 2024-02-18T11:29:56.870 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-02-18T11:29:56.870 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pytz-2021.1-5.el9.noarch 2024-02-18T11:29:56.870 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rados-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:56.870 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rbd-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:56.870 INFO:teuthology.orchestra.run.smithi188.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-02-18T11:29:56.870 INFO:teuthology.orchestra.run.smithi188.stdout: python3-requests-2.25.1-8.el9.noarch 2024-02-18T11:29:56.870 INFO:teuthology.orchestra.run.smithi188.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-02-18T11:29:56.870 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rgw-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:56.870 INFO:teuthology.orchestra.run.smithi188.stdout: python3-routes-2.5.1-5.el9.noarch 2024-02-18T11:29:56.870 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rsa-4.9-2.el9.noarch 2024-02-18T11:29:56.870 INFO:teuthology.orchestra.run.smithi188.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-02-18T11:29:56.871 INFO:teuthology.orchestra.run.smithi188.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-02-18T11:29:56.871 INFO:teuthology.orchestra.run.smithi188.stdout: python3-toml-0.10.2-6.el9.noarch 2024-02-18T11:29:56.871 INFO:teuthology.orchestra.run.smithi188.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-02-18T11:29:56.871 INFO:teuthology.orchestra.run.smithi188.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-02-18T11:29:56.871 INFO:teuthology.orchestra.run.smithi188.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-02-18T11:29:56.871 INFO:teuthology.orchestra.run.smithi188.stdout: python3-webob-1.8.7-6.el9.noarch 2024-02-18T11:29:56.871 INFO:teuthology.orchestra.run.smithi188.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-02-18T11:29:56.871 INFO:teuthology.orchestra.run.smithi188.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-02-18T11:29:56.871 INFO:teuthology.orchestra.run.smithi188.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-02-18T11:29:56.871 INFO:teuthology.orchestra.run.smithi188.stdout: qatlib-23.11.0-1.el9.x86_64 2024-02-18T11:29:56.871 INFO:teuthology.orchestra.run.smithi188.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-02-18T11:29:56.871 INFO:teuthology.orchestra.run.smithi188.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-02-18T11:29:56.872 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-fuse-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:56.872 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-mirror-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:56.872 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-nbd-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:29:56.872 INFO:teuthology.orchestra.run.smithi188.stdout: re2-1:20211101-3.el9.x86_64 2024-02-18T11:29:56.872 INFO:teuthology.orchestra.run.smithi188.stdout: socat-1.7.4.1-5.el9.x86_64 2024-02-18T11:29:56.872 INFO:teuthology.orchestra.run.smithi188.stdout: thrift-0.15.0-2.el9.x86_64 2024-02-18T11:29:56.872 INFO:teuthology.orchestra.run.smithi188.stdout: uuid-1.6.2-55.el9.x86_64 2024-02-18T11:29:56.872 INFO:teuthology.orchestra.run.smithi188.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-02-18T11:29:56.872 INFO:teuthology.orchestra.run.smithi188.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-02-18T11:29:56.873 INFO:teuthology.orchestra.run.smithi188.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-02-18T11:29:56.873 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-02-18T11:29:56.873 INFO:teuthology.orchestra.run.smithi188.stdout:Complete! 2024-02-18T11:29:57.419 DEBUG:teuthology.parallel:result is None 2024-02-18T11:29:57.473 DEBUG:teuthology.parallel:result is None 2024-02-18T11:29:57.473 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1c920afa8316ea21877f3186850254947531ea22 2024-02-18T11:29:57.692 DEBUG:teuthology.orchestra.run.smithi012:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-18T11:29:57.715 INFO:teuthology.orchestra.run.smithi012.stdout:19.0.0-1555.g1c920afa.el9 2024-02-18T11:29:57.715 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1555.g1c920afa.el9 2024-02-18T11:29:57.715 INFO:teuthology.task.install:The correct ceph version 19.0.0-1555.g1c920afa is installed. 2024-02-18T11:29:57.717 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1c920afa8316ea21877f3186850254947531ea22 2024-02-18T11:29:57.934 DEBUG:teuthology.orchestra.run.smithi158:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-18T11:29:57.960 INFO:teuthology.orchestra.run.smithi158.stdout:19.0.0-1555.g1c920afa.el9 2024-02-18T11:29:57.960 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1555.g1c920afa.el9 2024-02-18T11:29:57.960 INFO:teuthology.task.install:The correct ceph version 19.0.0-1555.g1c920afa is installed. 2024-02-18T11:29:57.962 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1c920afa8316ea21877f3186850254947531ea22 2024-02-18T11:29:58.181 DEBUG:teuthology.orchestra.run.smithi188:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-18T11:29:58.205 INFO:teuthology.orchestra.run.smithi188.stdout:19.0.0-1555.g1c920afa.el9 2024-02-18T11:29:58.206 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1555.g1c920afa.el9 2024-02-18T11:29:58.206 INFO:teuthology.task.install:The correct ceph version 19.0.0-1555.g1c920afa is installed. 2024-02-18T11:29:58.208 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-02-18T11:29:58.208 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-02-18T11:29:58.208 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-18T11:29:58.243 DEBUG:teuthology.orchestra.run.smithi158:> set -ex 2024-02-18T11:29:58.243 DEBUG:teuthology.orchestra.run.smithi158:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-18T11:29:58.277 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-02-18T11:29:58.277 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-18T11:29:58.309 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-02-18T11:29:58.310 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-02-18T11:29:58.310 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/usr/bin/daemon-helper 2024-02-18T11:29:58.339 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-18T11:29:58.410 DEBUG:teuthology.orchestra.run.smithi158:> set -ex 2024-02-18T11:29:58.410 DEBUG:teuthology.orchestra.run.smithi158:> sudo dd of=/usr/bin/daemon-helper 2024-02-18T11:29:58.442 DEBUG:teuthology.orchestra.run.smithi158:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-18T11:29:58.515 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-02-18T11:29:58.515 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/usr/bin/daemon-helper 2024-02-18T11:29:58.546 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-18T11:29:58.615 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-02-18T11:29:58.616 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-02-18T11:29:58.616 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-18T11:29:58.645 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-18T11:29:58.715 DEBUG:teuthology.orchestra.run.smithi158:> set -ex 2024-02-18T11:29:58.715 DEBUG:teuthology.orchestra.run.smithi158:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-18T11:29:58.745 DEBUG:teuthology.orchestra.run.smithi158:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-18T11:29:58.817 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-02-18T11:29:58.817 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-18T11:29:58.846 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-18T11:29:58.914 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-02-18T11:29:58.915 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-02-18T11:29:58.915 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/usr/bin/stdin-killer 2024-02-18T11:29:58.944 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-18T11:29:59.014 DEBUG:teuthology.orchestra.run.smithi158:> set -ex 2024-02-18T11:29:59.014 DEBUG:teuthology.orchestra.run.smithi158:> sudo dd of=/usr/bin/stdin-killer 2024-02-18T11:29:59.045 DEBUG:teuthology.orchestra.run.smithi158:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-18T11:29:59.114 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-02-18T11:29:59.115 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/usr/bin/stdin-killer 2024-02-18T11:29:59.144 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-18T11:29:59.213 INFO:teuthology.run_tasks:Running task cephadm... 2024-02-18T11:29:59.321 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': '1c920afa8316ea21877f3186850254947531ea22', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-02-18T11:29:59.321 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 2024-02-18T11:29:59.321 INFO:tasks.cephadm:Cluster fsid is 0c8bb0de-ce51-11ee-95ba-87774f69a715 2024-02-18T11:29:59.322 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-02-18T11:29:59.322 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.12', 'mon.b': '172.21.15.158', 'mon.c': '172.21.15.188'} 2024-02-18T11:29:59.322 INFO:tasks.cephadm:First mon is mon.a on smithi012 2024-02-18T11:29:59.322 INFO:tasks.cephadm:First mgr is x 2024-02-18T11:29:59.322 INFO:tasks.cephadm:Normalizing hostnames... 2024-02-18T11:29:59.322 DEBUG:teuthology.orchestra.run.smithi012:> sudo hostname $(hostname -s) 2024-02-18T11:29:59.354 DEBUG:teuthology.orchestra.run.smithi158:> sudo hostname $(hostname -s) 2024-02-18T11:29:59.385 DEBUG:teuthology.orchestra.run.smithi188:> sudo hostname $(hostname -s) 2024-02-18T11:29:59.414 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-02-18T11:29:59.414 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1c920afa8316ea21877f3186850254947531ea22 2024-02-18T11:29:59.556 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '1c920afa8316ea21877f3186850254947531ea22', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/76850/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-1555-g1c920afa', 'node_name': '172.21.2.14+braggi14', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-1555.g1c920afa'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240217.015652/1c920afa8316ea21877f3186850254947531ea22/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-02-17 03:35:08.314406', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-vshankar-testing-20240217.015652', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-vshankar-testing-20240217.015652/1c920afa8316ea21877f3186850254947531ea22/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-02-18T11:29:59.778 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref wip-vshankar-testing-20240217.015652, sha1 1c920afa8316ea21877f3186850254947531ea22 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=1c920afa8316ea21877f3186850254947531ea22 2024-02-18T11:29:59.780 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240217.015652/1c920afa8316ea21877f3186850254947531ea22/centos/9/x86_64/flavors/default/cephadm 2024-02-18T11:29:59.780 DEBUG:teuthology.orchestra.run.smithi012:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240217.015652/1c920afa8316ea21877f3186850254947531ea22/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-18T11:30:00.201 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r--. 1 ubuntu ubuntu 766708 Feb 18 11:30 /home/ubuntu/cephtest/cephadm 2024-02-18T11:30:00.202 DEBUG:teuthology.orchestra.run.smithi158:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240217.015652/1c920afa8316ea21877f3186850254947531ea22/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-18T11:30:00.597 INFO:teuthology.orchestra.run.smithi158.stdout:-rw-r--r--. 1 ubuntu ubuntu 766708 Feb 18 11:30 /home/ubuntu/cephtest/cephadm 2024-02-18T11:30:00.597 DEBUG:teuthology.orchestra.run.smithi188:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240217.015652/1c920afa8316ea21877f3186850254947531ea22/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-18T11:30:01.001 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r--. 1 ubuntu ubuntu 766708 Feb 18 11:30 /home/ubuntu/cephtest/cephadm 2024-02-18T11:30:01.001 DEBUG:teuthology.orchestra.run.smithi012:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-18T11:30:01.020 DEBUG:teuthology.orchestra.run.smithi158:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-18T11:30:01.038 DEBUG:teuthology.orchestra.run.smithi188:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-18T11:30:01.092 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 on all hosts... 2024-02-18T11:30:01.092 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 pull 2024-02-18T11:30:01.095 DEBUG:teuthology.orchestra.run.smithi158:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 pull 2024-02-18T11:30:01.097 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 pull 2024-02-18T11:30:01.907 INFO:teuthology.orchestra.run.smithi158.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22... 2024-02-18T11:30:02.021 INFO:teuthology.orchestra.run.smithi188.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22... 2024-02-18T11:30:02.083 INFO:teuthology.orchestra.run.smithi012.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22... 2024-02-18T11:30:27.722 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-02-18T11:30:27.722 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_version": "ceph version 19.0.0-1555-g1c920afa (1c920afa8316ea21877f3186850254947531ea22) squid (dev)", 2024-02-18T11:30:27.722 INFO:teuthology.orchestra.run.smithi012.stdout: "image_id": "53c33ee3bf62dc411b650f86eedbbee7698efb1001b4c063271dcd8d65b41165", 2024-02-18T11:30:27.722 INFO:teuthology.orchestra.run.smithi012.stdout: "repo_digests": [ 2024-02-18T11:30:27.722 INFO:teuthology.orchestra.run.smithi012.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1" 2024-02-18T11:30:27.722 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-02-18T11:30:27.722 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-02-18T11:30:32.223 INFO:teuthology.orchestra.run.smithi158.stdout:{ 2024-02-18T11:30:32.224 INFO:teuthology.orchestra.run.smithi158.stdout: "ceph_version": "ceph version 19.0.0-1555-g1c920afa (1c920afa8316ea21877f3186850254947531ea22) squid (dev)", 2024-02-18T11:30:32.224 INFO:teuthology.orchestra.run.smithi158.stdout: "image_id": "53c33ee3bf62dc411b650f86eedbbee7698efb1001b4c063271dcd8d65b41165", 2024-02-18T11:30:32.224 INFO:teuthology.orchestra.run.smithi158.stdout: "repo_digests": [ 2024-02-18T11:30:32.224 INFO:teuthology.orchestra.run.smithi158.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1" 2024-02-18T11:30:32.224 INFO:teuthology.orchestra.run.smithi158.stdout: ] 2024-02-18T11:30:32.224 INFO:teuthology.orchestra.run.smithi158.stdout:} 2024-02-18T11:30:36.945 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-02-18T11:30:36.945 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_version": "ceph version 19.0.0-1555-g1c920afa (1c920afa8316ea21877f3186850254947531ea22) squid (dev)", 2024-02-18T11:30:36.945 INFO:teuthology.orchestra.run.smithi188.stdout: "image_id": "53c33ee3bf62dc411b650f86eedbbee7698efb1001b4c063271dcd8d65b41165", 2024-02-18T11:30:36.946 INFO:teuthology.orchestra.run.smithi188.stdout: "repo_digests": [ 2024-02-18T11:30:36.946 INFO:teuthology.orchestra.run.smithi188.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1" 2024-02-18T11:30:36.946 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-02-18T11:30:36.946 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-02-18T11:30:36.969 DEBUG:teuthology.orchestra.run.smithi012:> sudo mkdir -p /etc/ceph 2024-02-18T11:30:37.004 DEBUG:teuthology.orchestra.run.smithi158:> sudo mkdir -p /etc/ceph 2024-02-18T11:30:37.040 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /etc/ceph 2024-02-18T11:30:37.069 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 777 /etc/ceph 2024-02-18T11:30:37.099 DEBUG:teuthology.orchestra.run.smithi158:> sudo chmod 777 /etc/ceph 2024-02-18T11:30:37.129 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 777 /etc/ceph 2024-02-18T11:30:37.156 INFO:tasks.cephadm:Writing seed config... 2024-02-18T11:30:37.187 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-02-18T11:30:37.187 INFO:tasks.cephadm: override: [client] debug client = 20 2024-02-18T11:30:37.187 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-02-18T11:30:37.187 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-02-18T11:30:37.187 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-02-18T11:30:37.187 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-02-18T11:30:37.187 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-02-18T11:30:37.187 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-02-18T11:30:37.187 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-02-18T11:30:37.187 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-02-18T11:30:37.187 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-02-18T11:30:37.187 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-02-18T11:30:37.188 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-02-18T11:30:37.188 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-02-18T11:30:37.188 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-02-18T11:30:37.188 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-02-18T11:30:37.188 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-02-18T11:30:37.188 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-02-18T11:30:37.188 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-02-18T11:30:37.188 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-02-18T11:30:37.188 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-02-18T11:30:37.188 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-02-18T11:30:37.188 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-02-18T11:30:37.188 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-02-18T11:30:37.188 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-02-18T11:30:37.188 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-02-18T11:30:37.188 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-02-18T11:30:37.189 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-02-18T11:30:37.189 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-02-18T11:30:37.189 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-02-18T11:30:37.189 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-02-18T11:30:37.189 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-02-18T11:30:37.189 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-02-18T11:30:37.189 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-02-18T11:30:37.189 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-02-18T11:30:37.189 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-02-18T11:30:37.189 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-02-18T11:30:37.189 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-02-18T11:30:37.189 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-02-18T11:30:37.189 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-02-18T11:30:37.189 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-02-18T11:30:37.189 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-02-18T11:30:37.189 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-02-18T11:30:37.190 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-02-18T11:30:37.190 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-02-18T11:30:37.190 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-02-18T11:30:37.190 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-02-18T11:30:37.191 DEBUG:teuthology.orchestra.run.smithi012:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-02-18T11:30:37.208 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 0c8bb0de-ce51-11ee-95ba-87774f69a715 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-02-18T11:30:37.208 DEBUG:teuthology.orchestra.run.smithi012:mon.a> sudo journalctl -f -n 0 -u ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@mon.a.service 2024-02-18T11:30:37.251 DEBUG:teuthology.orchestra.run.smithi012:mgr.x> sudo journalctl -f -n 0 -u ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@mgr.x.service 2024-02-18T11:30:37.294 INFO:tasks.cephadm:Bootstrapping... 2024-02-18T11:30:37.295 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 -v bootstrap --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.12 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-02-18T11:30:37.495 INFO:teuthology.orchestra.run.smithi012.stdout:-------------------------------------------------------------------------------- 2024-02-18T11:30:37.495 INFO:teuthology.orchestra.run.smithi012.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22', '-v', 'bootstrap', '--fsid', '0c8bb0de-ce51-11ee-95ba-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.12', '--skip-admin-label'] 2024-02-18T11:30:37.519 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-18T11:30:37.520 INFO:teuthology.orchestra.run.smithi012.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-02-18T11:30:37.520 INFO:teuthology.orchestra.run.smithi012.stdout:Verifying podman|docker is present... 2024-02-18T11:30:37.544 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-18T11:30:37.544 INFO:teuthology.orchestra.run.smithi012.stdout:Verifying lvm2 is present... 2024-02-18T11:30:37.544 INFO:teuthology.orchestra.run.smithi012.stdout:Verifying time synchronization is in place... 2024-02-18T11:30:37.551 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-02-18T11:30:37.551 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-02-18T11:30:37.557 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-02-18T11:30:37.557 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout inactive 2024-02-18T11:30:37.563 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout enabled 2024-02-18T11:30:37.569 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout active 2024-02-18T11:30:37.570 INFO:teuthology.orchestra.run.smithi012.stdout:Unit chronyd.service is enabled and running 2024-02-18T11:30:37.570 INFO:teuthology.orchestra.run.smithi012.stdout:Repeating the final host check... 2024-02-18T11:30:37.593 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-18T11:30:37.593 INFO:teuthology.orchestra.run.smithi012.stdout:podman (/bin/podman) version 4.9.3 is present 2024-02-18T11:30:37.593 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl is present 2024-02-18T11:30:37.594 INFO:teuthology.orchestra.run.smithi012.stdout:lvcreate is present 2024-02-18T11:30:37.599 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-02-18T11:30:37.599 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-02-18T11:30:37.605 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-02-18T11:30:37.605 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout inactive 2024-02-18T11:30:37.611 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout enabled 2024-02-18T11:30:37.617 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout active 2024-02-18T11:30:37.617 INFO:teuthology.orchestra.run.smithi012.stdout:Unit chronyd.service is enabled and running 2024-02-18T11:30:37.617 INFO:teuthology.orchestra.run.smithi012.stdout:Host looks OK 2024-02-18T11:30:37.617 INFO:teuthology.orchestra.run.smithi012.stdout:Cluster fsid: 0c8bb0de-ce51-11ee-95ba-87774f69a715 2024-02-18T11:30:37.617 INFO:teuthology.orchestra.run.smithi012.stdout:Acquiring lock 140568158166992 on /run/cephadm/0c8bb0de-ce51-11ee-95ba-87774f69a715.lock 2024-02-18T11:30:37.617 INFO:teuthology.orchestra.run.smithi012.stdout:Lock 140568158166992 acquired on /run/cephadm/0c8bb0de-ce51-11ee-95ba-87774f69a715.lock 2024-02-18T11:30:37.618 INFO:teuthology.orchestra.run.smithi012.stdout:Verifying IP 172.21.15.12 port 3300 ... 2024-02-18T11:30:37.618 INFO:teuthology.orchestra.run.smithi012.stdout:Verifying IP 172.21.15.12 port 6789 ... 2024-02-18T11:30:37.618 INFO:teuthology.orchestra.run.smithi012.stdout:Base mon IP(s) is [172.21.15.12:3300, 172.21.15.12:6789], mon addrv is [v2:172.21.15.12:3300,v1:172.21.15.12:6789] 2024-02-18T11:30:37.622 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.12 metric 100 2024-02-18T11:30:37.622 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.12 metric 100 2024-02-18T11:30:37.624 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-02-18T11:30:37.624 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-02-18T11:30:37.624 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-02-18T11:30:37.627 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-02-18T11:30:37.627 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-02-18T11:30:37.627 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-02-18T11:30:37.627 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-02-18T11:30:37.627 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout inet6 fe80::ae1f:6bff:fef8:25ea/64 scope link noprefixroute 2024-02-18T11:30:37.627 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-02-18T11:30:37.628 INFO:teuthology.orchestra.run.smithi012.stdout:Mon IP `172.21.15.12` is in CIDR network `172.21.0.0/20` 2024-02-18T11:30:37.628 INFO:teuthology.orchestra.run.smithi012.stdout:Mon IP `172.21.15.12` is in CIDR network `172.21.0.0/20` 2024-02-18T11:30:37.628 INFO:teuthology.orchestra.run.smithi012.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-02-18T11:30:37.628 INFO:teuthology.orchestra.run.smithi012.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-02-18T11:30:37.629 INFO:teuthology.orchestra.run.smithi012.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22... 2024-02-18T11:30:38.097 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stdout 53c33ee3bf62dc411b650f86eedbbee7698efb1001b4c063271dcd8d65b41165 2024-02-18T11:30:38.097 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22... 2024-02-18T11:30:38.097 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stderr Getting image source signatures 2024-02-18T11:30:38.097 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stderr Copying blob sha256:0f642e61a11afb42c0ccb5b05f5809250ad643ed143780d6ff49a5a4c492fc38 2024-02-18T11:30:38.098 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stderr Copying blob sha256:abdae48b0ac50b3c5f0ea645f05d0bae1f7a14884975c248b7e205a8cf4e9ebd 2024-02-18T11:30:38.098 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stderr Copying config sha256:53c33ee3bf62dc411b650f86eedbbee7698efb1001b4c063271dcd8d65b41165 2024-02-18T11:30:38.098 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stderr Writing manifest to image destination 2024-02-18T11:30:39.027 INFO:teuthology.orchestra.run.smithi012.stdout:ceph: stdout ceph version 19.0.0-1555-g1c920afa (1c920afa8316ea21877f3186850254947531ea22) squid (dev) 2024-02-18T11:30:39.027 INFO:teuthology.orchestra.run.smithi012.stdout:Ceph version: ceph version 19.0.0-1555-g1c920afa (1c920afa8316ea21877f3186850254947531ea22) squid (dev) 2024-02-18T11:30:39.027 INFO:teuthology.orchestra.run.smithi012.stdout:Extracting ceph user uid/gid from container image... 2024-02-18T11:30:40.066 INFO:teuthology.orchestra.run.smithi012.stdout:stat: stdout 167 167 2024-02-18T11:30:40.066 INFO:teuthology.orchestra.run.smithi012.stdout:Creating initial keys... 2024-02-18T11:30:40.929 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph-authtool: stdout AQBg6tFlAliiFxAAi0SVfMJ6eneAtf29Ic0dNg== 2024-02-18T11:30:41.767 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph-authtool: stdout AQBh6tFlPmILDxAAmhJ/xwMUTwUxQV0RQRMxUw== 2024-02-18T11:30:42.683 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph-authtool: stdout AQBi6tFlJG/nBRAAzhUsbXtIj11I+mJNAJT6tw== 2024-02-18T11:30:42.684 INFO:teuthology.orchestra.run.smithi012.stdout:Creating initial monmap... 2024-02-18T11:30:43.613 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-18T11:30:43.613 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-02-18T11:30:43.613 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 0c8bb0de-ce51-11ee-95ba-87774f69a715 2024-02-18T11:30:43.613 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-18T11:30:43.613 INFO:teuthology.orchestra.run.smithi012.stdout:monmaptool for a [v2:172.21.15.12:3300,v1:172.21.15.12:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-18T11:30:43.613 INFO:teuthology.orchestra.run.smithi012.stdout:setting min_mon_release = quincy 2024-02-18T11:30:43.613 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/monmaptool: set fsid to 0c8bb0de-ce51-11ee-95ba-87774f69a715 2024-02-18T11:30:43.613 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-18T11:30:43.613 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:30:43.613 INFO:teuthology.orchestra.run.smithi012.stdout:Creating mon... 2024-02-18T11:30:44.443 INFO:teuthology.orchestra.run.smithi012.stdout:create mon.a on 2024-02-18T11:30:44.657 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-02-18T11:30:44.795 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-02-18T11:30:44.980 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715.target → /etc/systemd/system/ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715.target. 2024-02-18T11:30:44.981 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715.target → /etc/systemd/system/ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715.target. 2024-02-18T11:30:45.690 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@mon.a 2024-02-18T11:30:45.690 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Failed to reset failed state of unit ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@mon.a.service: Unit ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@mon.a.service not loaded. 2024-02-18T11:30:45.954 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715.target.wants/ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@mon.a.service → /etc/systemd/system/ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@.service. 2024-02-18T11:30:46.425 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:30:46 smithi012 podman[32054]: 2024-02-18 11:30:46.096981205 +0000 UTC m=+0.017948396 image pull 53c33ee3bf62dc411b650f86eedbbee7698efb1001b4c063271dcd8d65b41165 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 2024-02-18T11:30:47.241 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:30:47 smithi012 podman[32054]: 2024-02-18 11:30:47.044944158 +0000 UTC m=+0.965911339 container create 5179e2bcc3c6a581b3f05a2d228c976001515dd68e5642c2e08b09cad14da302 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mon-a, org.label-schema.build-date=20240214, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-02-18T11:30:49.425 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:30:49 smithi012 podman[32054]: 2024-02-18 11:30:49.035396177 +0000 UTC m=+2.956363368 container init 5179e2bcc3c6a581b3f05a2d228c976001515dd68e5642c2e08b09cad14da302 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mon-a, GIT_CLEAN=True, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-vshankar-testing-20240217.015652-1c920af) 2024-02-18T11:30:49.425 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:30:49 smithi012 podman[32054]: 2024-02-18 11:30:49.039743188 +0000 UTC m=+2.960710370 container start 5179e2bcc3c6a581b3f05a2d228c976001515dd68e5642c2e08b09cad14da302 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mon-a, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-02-18T11:30:49.425 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:30:49 smithi012 ceph-mon[32084]: mkfs 0c8bb0de-ce51-11ee-95ba-87774f69a715 2024-02-18T11:30:49.425 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:30:49 smithi012 ceph-mon[32084]: mon.a is new leader, mons a in quorum (ranks 0) 2024-02-18T11:30:49.916 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-18T11:30:49.916 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout disabled 2024-02-18T11:30:49.924 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-18T11:30:49.924 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout inactive 2024-02-18T11:30:49.924 INFO:teuthology.orchestra.run.smithi012.stdout:firewalld.service is not enabled 2024-02-18T11:30:49.924 INFO:teuthology.orchestra.run.smithi012.stdout:Not possible to enable service . firewalld.service is not available 2024-02-18T11:30:49.925 INFO:teuthology.orchestra.run.smithi012.stdout:Waiting for mon to start... 2024-02-18T11:30:49.925 INFO:teuthology.orchestra.run.smithi012.stdout:Waiting for mon... 2024-02-18T11:30:50.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:30:49 smithi012 bash[32054]: 5179e2bcc3c6a581b3f05a2d228c976001515dd68e5642c2e08b09cad14da302 2024-02-18T11:30:50.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:30:49 smithi012 systemd[1]: Started Ceph mon.a for 0c8bb0de-ce51-11ee-95ba-87774f69a715. 2024-02-18T11:30:50.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:30:50 smithi012 ceph-mon[32084]: mon.a is new leader, mons a in quorum (ranks 0) 2024-02-18T11:30:50.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:30:50 smithi012 ceph-mon[32084]: monmap e1: 1 mons at {a=[v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-18T11:30:50.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:30:50 smithi012 ceph-mon[32084]: fsmap 2024-02-18T11:30:50.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:30:50 smithi012 ceph-mon[32084]: osdmap e1: 0 total, 0 up, 0 in 2024-02-18T11:30:50.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:30:50 smithi012 ceph-mon[32084]: mgrmap e1: no daemons active 2024-02-18T11:30:53.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:30:53 smithi012 ceph-mon[32084]: from='client.? 172.21.15.12:0/1175411825' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-02-18T11:30:57.602 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout cluster: 2024-02-18T11:30:57.609 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout id: 0c8bb0de-ce51-11ee-95ba-87774f69a715 2024-02-18T11:30:57.609 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-02-18T11:30:57.609 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-02-18T11:30:57.609 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout services: 2024-02-18T11:30:57.609 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 3s) 2024-02-18T11:30:57.610 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-02-18T11:30:57.610 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-02-18T11:30:57.610 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-02-18T11:30:57.610 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout data: 2024-02-18T11:30:57.610 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-02-18T11:30:57.610 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-02-18T11:30:57.610 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-02-18T11:30:57.610 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout pgs: 2024-02-18T11:30:57.610 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-02-18T11:30:57.610 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:52.979+0000 7f324524b700 1 Processor -- start 2024-02-18T11:30:57.610 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:52.979+0000 7f324524b700 1 -- start start 2024-02-18T11:30:57.610 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:52.979+0000 7f324524b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3240108d10 0x7f3240109130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:30:57.610 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:52.979+0000 7f324524b700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3240109700 con 0x7f3240108d10 2024-02-18T11:30:57.610 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:52.979+0000 7f323ed9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3240108d10 0x7f3240109130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:30:57.610 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:52.979+0000 7f323ed9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3240108d10 0x7f3240109130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52416/0 (socket says 172.21.15.12:52416) 2024-02-18T11:30:57.611 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:52.979+0000 7f323ed9d700 1 -- 172.21.15.12:0/1542863823 learned_addr learned my addr 172.21.15.12:0/1542863823 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:30:57.611 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:52.980+0000 7f323ed9d700 1 -- 172.21.15.12:0/1542863823 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3240109840 con 0x7f3240108d10 2024-02-18T11:30:57.611 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:52.980+0000 7f323ed9d700 1 --2- 172.21.15.12:0/1542863823 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3240108d10 0x7f3240109130 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f32300098a0 tx=0x7f3230009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3964447be607fe03 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:30:57.611 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:52.980+0000 7f323dd9b700 1 -- 172.21.15.12:0/1542863823 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3230004160 con 0x7f3240108d10 2024-02-18T11:30:57.611 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:52.980+0000 7f323dd9b700 1 -- 172.21.15.12:0/1542863823 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32300042c0 con 0x7f3240108d10 2024-02-18T11:30:57.611 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:52.980+0000 7f323dd9b700 1 -- 172.21.15.12:0/1542863823 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f32300045a0 con 0x7f3240108d10 2024-02-18T11:30:57.611 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:52.980+0000 7f324524b700 1 -- 172.21.15.12:0/1542863823 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3240108d10 msgr2=0x7f3240109130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:30:57.611 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:52.980+0000 7f324524b700 1 --2- 172.21.15.12:0/1542863823 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3240108d10 0x7f3240109130 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f32300098a0 tx=0x7f3230009bb0 comp rx=0 tx=0).stop 2024-02-18T11:30:57.611 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:52.980+0000 7f324524b700 1 -- 172.21.15.12:0/1542863823 shutdown_connections 2024-02-18T11:30:57.611 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:52.980+0000 7f324524b700 1 --2- 172.21.15.12:0/1542863823 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3240108d10 0x7f3240109130 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:30:57.611 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:52.980+0000 7f324524b700 1 -- 172.21.15.12:0/1542863823 >> 172.21.15.12:0/1542863823 conn(0x7f32401013b0 msgr2=0x7f32401037d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:30:57.611 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:52.980+0000 7f324524b700 1 -- 172.21.15.12:0/1542863823 shutdown_connections 2024-02-18T11:30:57.611 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:52.981+0000 7f324524b700 1 -- 172.21.15.12:0/1542863823 wait complete. 2024-02-18T11:30:57.612 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:52.981+0000 7f324524b700 1 Processor -- start 2024-02-18T11:30:57.612 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:52.981+0000 7f324524b700 1 -- start start 2024-02-18T11:30:57.612 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:52.982+0000 7f324524b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3240108d10 0x7f324019aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:30:57.612 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:52.982+0000 7f324524b700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f324019b1e0 con 0x7f3240108d10 2024-02-18T11:30:57.612 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:52.982+0000 7f323ed9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3240108d10 0x7f324019aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:30:57.612 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:52.982+0000 7f323ed9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3240108d10 0x7f324019aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52420/0 (socket says 172.21.15.12:52420) 2024-02-18T11:30:57.612 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:52.982+0000 7f323ed9d700 1 -- 172.21.15.12:0/1175411825 learned_addr learned my addr 172.21.15.12:0/1175411825 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:30:57.612 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:52.982+0000 7f323ed9d700 1 -- 172.21.15.12:0/1175411825 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3230009550 con 0x7f3240108d10 2024-02-18T11:30:57.612 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:52.982+0000 7f323ed9d700 1 --2- 172.21.15.12:0/1175411825 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3240108d10 0x7f324019aca0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f3230004000 tx=0x7f3230004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:30:57.612 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:52.982+0000 7f3227fff700 1 -- 172.21.15.12:0/1175411825 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3230011630 con 0x7f3240108d10 2024-02-18T11:30:57.612 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:52.982+0000 7f3227fff700 1 -- 172.21.15.12:0/1175411825 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3230011790 con 0x7f3240108d10 2024-02-18T11:30:57.612 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:52.982+0000 7f3227fff700 1 -- 172.21.15.12:0/1175411825 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3230011a30 con 0x7f3240108d10 2024-02-18T11:30:57.612 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:52.982+0000 7f324524b700 1 -- 172.21.15.12:0/1175411825 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f324019b3e0 con 0x7f3240108d10 2024-02-18T11:30:57.612 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:52.983+0000 7f324524b700 1 -- 172.21.15.12:0/1175411825 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f324019b880 con 0x7f3240108d10 2024-02-18T11:30:57.613 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:52.983+0000 7f3227fff700 1 -- 172.21.15.12:0/1175411825 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f3230021070 con 0x7f3240108d10 2024-02-18T11:30:57.613 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:52.983+0000 7f3227fff700 1 -- 172.21.15.12:0/1175411825 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f323001b7d0 con 0x7f3240108d10 2024-02-18T11:30:57.613 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:52.983+0000 7f324524b700 1 -- 172.21.15.12:0/1175411825 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3240194930 con 0x7f3240108d10 2024-02-18T11:30:57.613 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:52.986+0000 7f3227fff700 1 -- 172.21.15.12:0/1175411825 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f323001ba30 con 0x7f3240108d10 2024-02-18T11:30:57.613 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:53.060+0000 7f324524b700 1 -- 172.21.15.12:0/1175411825 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f3240105ae0 con 0x7f3240108d10 2024-02-18T11:30:57.613 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:53.060+0000 7f3227fff700 1 -- 172.21.15.12:0/1175411825 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+310 (secure 0 0 0) 0x7f323001bbd0 con 0x7f3240108d10 2024-02-18T11:30:57.613 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:53.061+0000 7f324524b700 1 -- 172.21.15.12:0/1175411825 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3240108d10 msgr2=0x7f324019aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:30:57.613 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:53.061+0000 7f324524b700 1 --2- 172.21.15.12:0/1175411825 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3240108d10 0x7f324019aca0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f3230004000 tx=0x7f3230004750 comp rx=0 tx=0).stop 2024-02-18T11:30:57.613 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:53.061+0000 7f324524b700 1 -- 172.21.15.12:0/1175411825 shutdown_connections 2024-02-18T11:30:57.613 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:53.061+0000 7f324524b700 1 --2- 172.21.15.12:0/1175411825 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3240108d10 0x7f324019aca0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:30:57.613 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:53.061+0000 7f324524b700 1 -- 172.21.15.12:0/1175411825 >> 172.21.15.12:0/1175411825 conn(0x7f32401013b0 msgr2=0x7f3240191a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:30:57.613 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:53.061+0000 7f324524b700 1 -- 172.21.15.12:0/1175411825 shutdown_connections 2024-02-18T11:30:57.613 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:53.061+0000 7f324524b700 1 -- 172.21.15.12:0/1175411825 wait complete. 2024-02-18T11:30:57.613 INFO:teuthology.orchestra.run.smithi012.stdout:mon is available 2024-02-18T11:30:57.614 INFO:teuthology.orchestra.run.smithi012.stdout:Assimilating anything we can from ceph.conf... 2024-02-18T11:30:58.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:30:58 smithi012 ceph-mon[32084]: from='client.? 172.21.15.12:0/1115369524' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-02-18T11:30:58.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:30:58 smithi012 ceph-mon[32084]: from='client.? 172.21.15.12:0/1115369524' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-02-18T11:30:58.715 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-02-18T11:30:58.715 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout [global] 2024-02-18T11:30:58.715 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout fsid = 0c8bb0de-ce51-11ee-95ba-87774f69a715 2024-02-18T11:30:58.715 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.12:3300,v1:172.21.15.12:6789] 2024-02-18T11:30:58.715 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-02-18T11:30:58.715 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-02-18T11:30:58.715 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-02-18T11:30:58.715 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-02-18T11:30:58.716 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-02-18T11:30:58.716 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout [mgr] 2024-02-18T11:30:58.716 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-02-18T11:30:58.716 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-02-18T11:30:58.716 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout [osd] 2024-02-18T11:30:58.716 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-02-18T11:30:58.716 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-02-18T11:30:58.716 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-02-18T11:30:58.716 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-02-18T11:30:58.716 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-02-18T11:30:58.716 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:58.028+0000 7fb548cf4700 1 Processor -- start 2024-02-18T11:30:58.716 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:58.028+0000 7fb548cf4700 1 -- start start 2024-02-18T11:30:58.716 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:58.028+0000 7fb548cf4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5441076f0 0x7fb544107b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:30:58.716 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:58.028+0000 7fb548cf4700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5441080e0 con 0x7fb5441076f0 2024-02-18T11:30:58.717 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:58.029+0000 7fb54259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5441076f0 0x7fb544107b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:30:58.717 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:58.029+0000 7fb54259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5441076f0 0x7fb544107b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52426/0 (socket says 172.21.15.12:52426) 2024-02-18T11:30:58.717 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:58.029+0000 7fb54259c700 1 -- 172.21.15.12:0/4287870791 learned_addr learned my addr 172.21.15.12:0/4287870791 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:30:58.717 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:58.029+0000 7fb54259c700 1 -- 172.21.15.12:0/4287870791 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5441088a0 con 0x7fb5441076f0 2024-02-18T11:30:58.717 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:58.029+0000 7fb54259c700 1 --2- 172.21.15.12:0/4287870791 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5441076f0 0x7fb544107b10 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fb534009e80 tx=0x7fb53400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96cef383fe715664 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:30:58.717 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:58.029+0000 7fb54159a700 1 -- 172.21.15.12:0/4287870791 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb53400e070 con 0x7fb5441076f0 2024-02-18T11:30:58.717 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:58.029+0000 7fb54159a700 1 -- 172.21.15.12:0/4287870791 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb534003a20 con 0x7fb5441076f0 2024-02-18T11:30:58.717 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:58.029+0000 7fb54159a700 1 -- 172.21.15.12:0/4287870791 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb534003ce0 con 0x7fb5441076f0 2024-02-18T11:30:58.717 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:58.030+0000 7fb548cf4700 1 -- 172.21.15.12:0/4287870791 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5441076f0 msgr2=0x7fb544107b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:30:58.717 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:58.030+0000 7fb548cf4700 1 --2- 172.21.15.12:0/4287870791 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5441076f0 0x7fb544107b10 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fb534009e80 tx=0x7fb53400b330 comp rx=0 tx=0).stop 2024-02-18T11:30:58.717 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:58.030+0000 7fb548cf4700 1 -- 172.21.15.12:0/4287870791 shutdown_connections 2024-02-18T11:30:58.717 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:58.030+0000 7fb548cf4700 1 --2- 172.21.15.12:0/4287870791 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5441076f0 0x7fb544107b10 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:30:58.717 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:58.030+0000 7fb548cf4700 1 -- 172.21.15.12:0/4287870791 >> 172.21.15.12:0/4287870791 conn(0x7fb544102d90 msgr2=0x7fb5441051b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:30:58.717 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:58.030+0000 7fb548cf4700 1 -- 172.21.15.12:0/4287870791 shutdown_connections 2024-02-18T11:30:58.718 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:58.030+0000 7fb548cf4700 1 -- 172.21.15.12:0/4287870791 wait complete. 2024-02-18T11:30:58.718 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:58.031+0000 7fb548cf4700 1 Processor -- start 2024-02-18T11:30:58.718 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:58.031+0000 7fb548cf4700 1 -- start start 2024-02-18T11:30:58.718 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:58.031+0000 7fb548cf4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5441076f0 0x7fb54407cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:30:58.718 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:58.031+0000 7fb548cf4700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb54407d0c0 con 0x7fb5441076f0 2024-02-18T11:30:58.718 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:58.031+0000 7fb54259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5441076f0 0x7fb54407cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:30:58.718 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:58.031+0000 7fb54259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5441076f0 0x7fb54407cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52432/0 (socket says 172.21.15.12:52432) 2024-02-18T11:30:58.718 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:58.031+0000 7fb54259c700 1 -- 172.21.15.12:0/1115369524 learned_addr learned my addr 172.21.15.12:0/1115369524 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:30:58.718 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:58.031+0000 7fb54259c700 1 -- 172.21.15.12:0/1115369524 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb534009b30 con 0x7fb5441076f0 2024-02-18T11:30:58.718 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:58.032+0000 7fb54259c700 1 --2- 172.21.15.12:0/1115369524 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5441076f0 0x7fb54407cb80 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fb534004000 tx=0x7fb534004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:30:58.718 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:58.032+0000 7fb53b7fe700 1 -- 172.21.15.12:0/1115369524 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb534004430 con 0x7fb5441076f0 2024-02-18T11:30:58.718 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:58.032+0000 7fb53b7fe700 1 -- 172.21.15.12:0/1115369524 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5340045b0 con 0x7fb5441076f0 2024-02-18T11:30:58.718 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:58.032+0000 7fb548cf4700 1 -- 172.21.15.12:0/1115369524 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb54407b2b0 con 0x7fb5441076f0 2024-02-18T11:30:58.718 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:58.032+0000 7fb53b7fe700 1 -- 172.21.15.12:0/1115369524 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb534004890 con 0x7fb5441076f0 2024-02-18T11:30:58.718 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:58.032+0000 7fb548cf4700 1 -- 172.21.15.12:0/1115369524 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb54407b750 con 0x7fb5441076f0 2024-02-18T11:30:58.719 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:58.032+0000 7fb53b7fe700 1 -- 172.21.15.12:0/1115369524 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fb5340127c0 con 0x7fb5441076f0 2024-02-18T11:30:58.719 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:58.032+0000 7fb53b7fe700 1 -- 172.21.15.12:0/1115369524 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb53401dce0 con 0x7fb5441076f0 2024-02-18T11:30:58.719 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:58.033+0000 7fb548cf4700 1 -- 172.21.15.12:0/1115369524 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb54407c160 con 0x7fb5441076f0 2024-02-18T11:30:58.719 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:58.035+0000 7fb53b7fe700 1 -- 172.21.15.12:0/1115369524 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fb53402b060 con 0x7fb5441076f0 2024-02-18T11:30:58.719 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:58.105+0000 7fb548cf4700 1 -- 172.21.15.12:0/1115369524 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fb544068df0 con 0x7fb5441076f0 2024-02-18T11:30:58.719 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:58.108+0000 7fb53b7fe700 1 -- 172.21.15.12:0/1115369524 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7fb534021030 con 0x7fb5441076f0 2024-02-18T11:30:58.719 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:58.109+0000 7fb53b7fe700 1 -- 172.21.15.12:0/1115369524 <== mon.0 v2:172.21.15.12:3300/0 8 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb53401d8e0 con 0x7fb5441076f0 2024-02-18T11:30:58.719 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:58.109+0000 7fb548cf4700 1 -- 172.21.15.12:0/1115369524 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5441076f0 msgr2=0x7fb54407cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:30:58.719 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:58.109+0000 7fb548cf4700 1 --2- 172.21.15.12:0/1115369524 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5441076f0 0x7fb54407cb80 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fb534004000 tx=0x7fb534004ad0 comp rx=0 tx=0).stop 2024-02-18T11:30:58.719 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:58.110+0000 7fb548cf4700 1 -- 172.21.15.12:0/1115369524 shutdown_connections 2024-02-18T11:30:58.719 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:58.110+0000 7fb548cf4700 1 --2- 172.21.15.12:0/1115369524 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5441076f0 0x7fb54407cb80 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:30:58.719 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:58.110+0000 7fb548cf4700 1 -- 172.21.15.12:0/1115369524 >> 172.21.15.12:0/1115369524 conn(0x7fb544102d90 msgr2=0x7fb5441919b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:30:58.719 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:58.110+0000 7fb548cf4700 1 -- 172.21.15.12:0/1115369524 shutdown_connections 2024-02-18T11:30:58.719 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:58.110+0000 7fb548cf4700 1 -- 172.21.15.12:0/1115369524 wait complete. 2024-02-18T11:30:58.719 INFO:teuthology.orchestra.run.smithi012.stdout:Generating new minimal ceph.conf... 2024-02-18T11:30:59.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:30:59 smithi012 ceph-mon[32084]: from='client.? 172.21.15.12:0/2740616608' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:30:59.778 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:59.128+0000 7f17ec574700 1 Processor -- start 2024-02-18T11:30:59.778 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:59.128+0000 7f17ec574700 1 -- start start 2024-02-18T11:30:59.779 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:59.128+0000 7f17ec574700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f17e4105a50 0x7f17e4105e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:30:59.779 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:59.128+0000 7f17ec574700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17e4106440 con 0x7f17e4105a50 2024-02-18T11:30:59.779 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:59.128+0000 7f17ea310700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f17e4105a50 0x7f17e4105e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:30:59.779 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:59.128+0000 7f17ea310700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f17e4105a50 0x7f17e4105e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52434/0 (socket says 172.21.15.12:52434) 2024-02-18T11:30:59.779 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:59.128+0000 7f17ea310700 1 -- 172.21.15.12:0/4069569737 learned_addr learned my addr 172.21.15.12:0/4069569737 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:30:59.779 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:59.129+0000 7f17ea310700 1 -- 172.21.15.12:0/4069569737 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17e41089f0 con 0x7f17e4105a50 2024-02-18T11:30:59.779 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:59.129+0000 7f17ea310700 1 --2- 172.21.15.12:0/4069569737 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f17e4105a50 0x7f17e4105e70 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f17e00098a0 tx=0x7f17e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a0f3ceed49853a8 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:30:59.779 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:59.129+0000 7f17e930e700 1 -- 172.21.15.12:0/4069569737 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f17e0004160 con 0x7f17e4105a50 2024-02-18T11:30:59.779 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:59.129+0000 7f17e930e700 1 -- 172.21.15.12:0/4069569737 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f17e00042c0 con 0x7f17e4105a50 2024-02-18T11:30:59.779 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:59.129+0000 7f17e930e700 1 -- 172.21.15.12:0/4069569737 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f17e0011ac0 con 0x7f17e4105a50 2024-02-18T11:30:59.779 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:59.130+0000 7f17ec574700 1 -- 172.21.15.12:0/4069569737 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f17e4105a50 msgr2=0x7f17e4105e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:30:59.779 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:59.130+0000 7f17ec574700 1 --2- 172.21.15.12:0/4069569737 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f17e4105a50 0x7f17e4105e70 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f17e00098a0 tx=0x7f17e0009bb0 comp rx=0 tx=0).stop 2024-02-18T11:30:59.779 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:59.130+0000 7f17ec574700 1 -- 172.21.15.12:0/4069569737 shutdown_connections 2024-02-18T11:30:59.780 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:59.130+0000 7f17ec574700 1 --2- 172.21.15.12:0/4069569737 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f17e4105a50 0x7f17e4105e70 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:30:59.780 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:59.130+0000 7f17ec574700 1 -- 172.21.15.12:0/4069569737 >> 172.21.15.12:0/4069569737 conn(0x7f17e41013b0 msgr2=0x7f17e41037d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:30:59.780 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:59.130+0000 7f17ec574700 1 -- 172.21.15.12:0/4069569737 shutdown_connections 2024-02-18T11:30:59.780 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:59.130+0000 7f17ec574700 1 -- 172.21.15.12:0/4069569737 wait complete. 2024-02-18T11:30:59.780 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:59.130+0000 7f17ec574700 1 Processor -- start 2024-02-18T11:30:59.780 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:59.130+0000 7f17ec574700 1 -- start start 2024-02-18T11:30:59.780 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:59.130+0000 7f17ec574700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f17e4105a50 0x7f17e4108270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:30:59.780 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:59.130+0000 7f17ec574700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17e41087b0 con 0x7f17e4105a50 2024-02-18T11:30:59.780 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:59.131+0000 7f17ea310700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f17e4105a50 0x7f17e4108270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:30:59.780 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:59.131+0000 7f17ea310700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f17e4105a50 0x7f17e4108270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52440/0 (socket says 172.21.15.12:52440) 2024-02-18T11:30:59.780 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:59.131+0000 7f17ea310700 1 -- 172.21.15.12:0/2740616608 learned_addr learned my addr 172.21.15.12:0/2740616608 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:30:59.780 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:59.131+0000 7f17ea310700 1 -- 172.21.15.12:0/2740616608 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17e0009550 con 0x7f17e4105a50 2024-02-18T11:30:59.780 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:59.131+0000 7f17ea310700 1 --2- 172.21.15.12:0/2740616608 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f17e4105a50 0x7f17e4108270 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f17e0009fb0 tx=0x7f17e0011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:30:59.780 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:59.131+0000 7f17db7fe700 1 -- 172.21.15.12:0/2740616608 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f17e00038b0 con 0x7f17e4105a50 2024-02-18T11:30:59.781 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:59.131+0000 7f17db7fe700 1 -- 172.21.15.12:0/2740616608 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f17e0003a10 con 0x7f17e4105a50 2024-02-18T11:30:59.781 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:59.131+0000 7f17ec574700 1 -- 172.21.15.12:0/2740616608 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17e41069a0 con 0x7f17e4105a50 2024-02-18T11:30:59.781 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:59.131+0000 7f17db7fe700 1 -- 172.21.15.12:0/2740616608 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f17e00195e0 con 0x7f17e4105a50 2024-02-18T11:30:59.781 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:59.131+0000 7f17ec574700 1 -- 172.21.15.12:0/2740616608 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17e4106e40 con 0x7f17e4105a50 2024-02-18T11:30:59.781 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:59.131+0000 7f17db7fe700 1 -- 172.21.15.12:0/2740616608 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f17e0019c00 con 0x7f17e4105a50 2024-02-18T11:30:59.781 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:59.132+0000 7f17db7fe700 1 -- 172.21.15.12:0/2740616608 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f17e0024990 con 0x7f17e4105a50 2024-02-18T11:30:59.781 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:59.132+0000 7f17ec574700 1 -- 172.21.15.12:0/2740616608 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17e4068da0 con 0x7f17e4105a50 2024-02-18T11:30:59.781 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:59.135+0000 7f17db7fe700 1 -- 172.21.15.12:0/2740616608 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f17e0003b80 con 0x7f17e4105a50 2024-02-18T11:30:59.781 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:59.205+0000 7f17ec574700 1 -- 172.21.15.12:0/2740616608 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f17e410be30 con 0x7f17e4105a50 2024-02-18T11:30:59.781 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:59.205+0000 7f17db7fe700 1 -- 172.21.15.12:0/2740616608 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f17e001ab00 con 0x7f17e4105a50 2024-02-18T11:30:59.781 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:59.206+0000 7f17ec574700 1 -- 172.21.15.12:0/2740616608 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f17e4105a50 msgr2=0x7f17e4108270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:30:59.781 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:59.206+0000 7f17ec574700 1 --2- 172.21.15.12:0/2740616608 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f17e4105a50 0x7f17e4108270 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f17e0009fb0 tx=0x7f17e0011b20 comp rx=0 tx=0).stop 2024-02-18T11:30:59.781 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:59.206+0000 7f17ec574700 1 -- 172.21.15.12:0/2740616608 shutdown_connections 2024-02-18T11:30:59.781 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:59.206+0000 7f17ec574700 1 --2- 172.21.15.12:0/2740616608 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f17e4105a50 0x7f17e4108270 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:30:59.782 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:59.206+0000 7f17ec574700 1 -- 172.21.15.12:0/2740616608 >> 172.21.15.12:0/2740616608 conn(0x7f17e41013b0 msgr2=0x7f17e4101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:30:59.782 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:59.206+0000 7f17ec574700 1 -- 172.21.15.12:0/2740616608 shutdown_connections 2024-02-18T11:30:59.782 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:30:59.206+0000 7f17ec574700 1 -- 172.21.15.12:0/2740616608 wait complete. 2024-02-18T11:30:59.782 INFO:teuthology.orchestra.run.smithi012.stdout:Restarting the monitor... 2024-02-18T11:31:00.024 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:30:59 smithi012 systemd[1]: Stopping Ceph mon.a for 0c8bb0de-ce51-11ee-95ba-87774f69a715... 2024-02-18T11:31:00.425 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:00 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mon-a[32079]: 2024-02-18T11:31:00.027+0000 7f4f45440700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-18T11:31:00.425 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:00 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mon-a[32079]: 2024-02-18T11:31:00.027+0000 7f4f45440700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-02-18T11:31:00.425 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:00 smithi012 podman[32330]: 2024-02-18 11:31:00.181632304 +0000 UTC m=+0.238632996 container died 5179e2bcc3c6a581b3f05a2d228c976001515dd68e5642c2e08b09cad14da302 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mon-a, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True) 2024-02-18T11:31:00.807 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:00 smithi012 podman[32330]: 2024-02-18 11:31:00.511296909 +0000 UTC m=+0.568297585 container cleanup 5179e2bcc3c6a581b3f05a2d228c976001515dd68e5642c2e08b09cad14da302 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5) 2024-02-18T11:31:00.807 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:00 smithi012 bash[32330]: ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mon-a 2024-02-18T11:31:00.807 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:00 smithi012 podman[32344]: 2024-02-18 11:31:00.67877303 +0000 UTC m=+0.497743294 container remove 5179e2bcc3c6a581b3f05a2d228c976001515dd68e5642c2e08b09cad14da302 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mon-a, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-02-18T11:31:01.133 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:00 smithi012 systemd[1]: ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@mon.a.service: Deactivated successfully. 2024-02-18T11:31:01.133 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:00 smithi012 systemd[1]: Stopped Ceph mon.a for 0c8bb0de-ce51-11ee-95ba-87774f69a715. 2024-02-18T11:31:01.133 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:00 smithi012 systemd[1]: Starting Ceph mon.a for 0c8bb0de-ce51-11ee-95ba-87774f69a715... 2024-02-18T11:31:01.386 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 podman[32442]: 2024-02-18 11:31:01.137920929 +0000 UTC m=+0.109358495 container create 74f2d580255e5a641a85532de12b838833c585e2189c06128eeb7fad38e2d61f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mon-a, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214) 2024-02-18T11:31:01.386 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 podman[32442]: 2024-02-18 11:31:01.046735566 +0000 UTC m=+0.018173090 image pull 53c33ee3bf62dc411b650f86eedbbee7698efb1001b4c063271dcd8d65b41165 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 2024-02-18T11:31:01.386 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 podman[32442]: 2024-02-18 11:31:01.390939037 +0000 UTC m=+0.362376578 container init 74f2d580255e5a641a85532de12b838833c585e2189c06128eeb7fad38e2d61f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2) 2024-02-18T11:31:01.456 INFO:teuthology.orchestra.run.smithi012.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-02-18T11:31:01.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 podman[32442]: 2024-02-18 11:31:01.394129611 +0000 UTC m=+0.365567136 container start 74f2d580255e5a641a85532de12b838833c585e2189c06128eeb7fad38e2d61f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mon-a, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-18T11:31:01.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: set uid:gid to 167:167 (ceph:ceph) 2024-02-18T11:31:01.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: ceph version 19.0.0-1555-g1c920afa (1c920afa8316ea21877f3186850254947531ea22) squid (dev), process ceph-mon, pid 2 2024-02-18T11:31:01.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: pidfile_write: ignore empty --pid-file 2024-02-18T11:31:01.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: load: jerasure load: lrc 2024-02-18T11:31:01.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: RocksDB version: 7.9.2 2024-02-18T11:31:01.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Git sha 0 2024-02-18T11:31:01.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Compile date 2024-02-17 02:14:56 2024-02-18T11:31:01.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: DB SUMMARY 2024-02-18T11:31:01.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: DB Session ID: OVR0FZWK680O0H3KXAJX 2024-02-18T11:31:01.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: CURRENT file: CURRENT 2024-02-18T11:31:01.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: IDENTITY file: IDENTITY 2024-02-18T11:31:01.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-02-18T11:31:01.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-02-18T11:31:01.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 100983 ; 2024-02-18T11:31:01.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.error_if_exists: 0 2024-02-18T11:31:01.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.create_if_missing: 0 2024-02-18T11:31:01.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.paranoid_checks: 1 2024-02-18T11:31:01.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.flush_verify_memtable_count: 1 2024-02-18T11:31:01.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-18T11:31:01.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-02-18T11:31:01.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.env: 0x560f43325ae0 2024-02-18T11:31:01.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.fs: PosixFileSystem 2024-02-18T11:31:01.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.info_log: 0x560f43d6b8e0 2024-02-18T11:31:01.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.max_file_opening_threads: 16 2024-02-18T11:31:01.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.statistics: (nil) 2024-02-18T11:31:01.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.use_fsync: 0 2024-02-18T11:31:01.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.max_log_file_size: 0 2024-02-18T11:31:01.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-18T11:31:01.675 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-18T11:31:01.675 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.keep_log_file_num: 1000 2024-02-18T11:31:01.675 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.recycle_log_file_num: 0 2024-02-18T11:31:01.675 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.allow_fallocate: 1 2024-02-18T11:31:01.675 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.allow_mmap_reads: 0 2024-02-18T11:31:01.675 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.allow_mmap_writes: 0 2024-02-18T11:31:01.675 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.use_direct_reads: 0 2024-02-18T11:31:01.675 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-18T11:31:01.675 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.create_missing_column_families: 0 2024-02-18T11:31:01.675 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.db_log_dir: 2024-02-18T11:31:01.675 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.wal_dir: 2024-02-18T11:31:01.675 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-18T11:31:01.675 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-18T11:31:01.676 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-18T11:31:01.676 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-18T11:31:01.676 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-18T11:31:01.676 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-18T11:31:01.676 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.advise_random_on_open: 1 2024-02-18T11:31:01.676 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.db_write_buffer_size: 0 2024-02-18T11:31:01.676 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.write_buffer_manager: 0x560f43d3f400 2024-02-18T11:31:01.676 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-18T11:31:01.676 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-18T11:31:01.676 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-18T11:31:01.676 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.rate_limiter: (nil) 2024-02-18T11:31:01.676 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-18T11:31:01.676 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.wal_recovery_mode: 2 2024-02-18T11:31:01.677 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.enable_thread_tracking: 0 2024-02-18T11:31:01.677 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.enable_pipelined_write: 0 2024-02-18T11:31:01.677 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.unordered_write: 0 2024-02-18T11:31:01.677 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-18T11:31:01.677 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-18T11:31:01.677 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-18T11:31:01.677 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-18T11:31:01.677 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.row_cache: None 2024-02-18T11:31:01.677 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.wal_filter: None 2024-02-18T11:31:01.677 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-18T11:31:01.677 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.allow_ingest_behind: 0 2024-02-18T11:31:01.677 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.two_write_queues: 0 2024-02-18T11:31:01.677 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.manual_wal_flush: 0 2024-02-18T11:31:01.677 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.wal_compression: 0 2024-02-18T11:31:01.678 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.atomic_flush: 0 2024-02-18T11:31:01.678 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-18T11:31:01.678 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-18T11:31:01.678 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-18T11:31:01.679 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.log_readahead_size: 0 2024-02-18T11:31:01.679 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-18T11:31:01.679 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.best_efforts_recovery: 0 2024-02-18T11:31:01.679 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-18T11:31:01.679 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-18T11:31:01.679 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.allow_data_in_errors: 0 2024-02-18T11:31:01.679 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.db_host_id: __hostname__ 2024-02-18T11:31:01.680 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.enforce_single_del_contracts: true 2024-02-18T11:31:01.680 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.max_background_jobs: 2 2024-02-18T11:31:01.680 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.max_background_compactions: -1 2024-02-18T11:31:01.680 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.max_subcompactions: 1 2024-02-18T11:31:01.680 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-18T11:31:01.680 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-18T11:31:01.680 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-18T11:31:01.680 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.max_total_wal_size: 0 2024-02-18T11:31:01.680 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-18T11:31:01.680 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-18T11:31:01.680 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-18T11:31:01.680 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-18T11:31:01.680 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.max_open_files: -1 2024-02-18T11:31:01.680 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.bytes_per_sync: 0 2024-02-18T11:31:01.681 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-18T11:31:01.681 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-18T11:31:01.681 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.compaction_readahead_size: 0 2024-02-18T11:31:01.681 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.max_background_flushes: -1 2024-02-18T11:31:01.681 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Compression algorithms supported: 2024-02-18T11:31:01.681 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-18T11:31:01.681 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: kZSTD supported: 0 2024-02-18T11:31:01.681 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: kXpressCompression supported: 0 2024-02-18T11:31:01.681 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: kLZ4HCCompression supported: 1 2024-02-18T11:31:01.681 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: kZlibCompression supported: 1 2024-02-18T11:31:01.681 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: kSnappyCompression supported: 1 2024-02-18T11:31:01.681 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: kLZ4Compression supported: 1 2024-02-18T11:31:01.681 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: kBZip2Compression supported: 0 2024-02-18T11:31:01.681 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-18T11:31:01.681 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: DMutex implementation: pthread_mutex_t 2024-02-18T11:31:01.682 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-02-18T11:31:01.682 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-02-18T11:31:01.682 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-18T11:31:01.682 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.merge_operator: 2024-02-18T11:31:01.682 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.compaction_filter: None 2024-02-18T11:31:01.682 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.compaction_filter_factory: None 2024-02-18T11:31:01.682 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.sst_partitioner_factory: None 2024-02-18T11:31:01.682 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-18T11:31:01.682 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-18T11:31:01.682 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x560f43d6b9e0) 2024-02-18T11:31:01.682 INFO:journalctl@ceph.mon.a.smithi012.stdout: cache_index_and_filter_blocks: 1 2024-02-18T11:31:01.682 INFO:journalctl@ceph.mon.a.smithi012.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-18T11:31:01.682 INFO:journalctl@ceph.mon.a.smithi012.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-18T11:31:01.682 INFO:journalctl@ceph.mon.a.smithi012.stdout: pin_top_level_index_and_filter: 1 2024-02-18T11:31:01.683 INFO:journalctl@ceph.mon.a.smithi012.stdout: index_type: 0 2024-02-18T11:31:01.683 INFO:journalctl@ceph.mon.a.smithi012.stdout: data_block_index_type: 0 2024-02-18T11:31:01.683 INFO:journalctl@ceph.mon.a.smithi012.stdout: index_shortening: 1 2024-02-18T11:31:01.683 INFO:journalctl@ceph.mon.a.smithi012.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-18T11:31:01.683 INFO:journalctl@ceph.mon.a.smithi012.stdout: checksum: 4 2024-02-18T11:31:01.683 INFO:journalctl@ceph.mon.a.smithi012.stdout: no_block_cache: 0 2024-02-18T11:31:01.683 INFO:journalctl@ceph.mon.a.smithi012.stdout: block_cache: 0x560f43d6f090 2024-02-18T11:31:01.683 INFO:journalctl@ceph.mon.a.smithi012.stdout: block_cache_name: BinnedLRUCache 2024-02-18T11:31:01.683 INFO:journalctl@ceph.mon.a.smithi012.stdout: block_cache_options: 2024-02-18T11:31:01.683 INFO:journalctl@ceph.mon.a.smithi012.stdout: capacity : 536870912 2024-02-18T11:31:01.683 INFO:journalctl@ceph.mon.a.smithi012.stdout: num_shard_bits : 4 2024-02-18T11:31:01.683 INFO:journalctl@ceph.mon.a.smithi012.stdout: strict_capacity_limit : 0 2024-02-18T11:31:01.684 INFO:journalctl@ceph.mon.a.smithi012.stdout: high_pri_pool_ratio: 0.000 2024-02-18T11:31:01.684 INFO:journalctl@ceph.mon.a.smithi012.stdout: block_cache_compressed: (nil) 2024-02-18T11:31:01.684 INFO:journalctl@ceph.mon.a.smithi012.stdout: persistent_cache: (nil) 2024-02-18T11:31:01.684 INFO:journalctl@ceph.mon.a.smithi012.stdout: block_size: 4096 2024-02-18T11:31:01.684 INFO:journalctl@ceph.mon.a.smithi012.stdout: block_size_deviation: 10 2024-02-18T11:31:01.684 INFO:journalctl@ceph.mon.a.smithi012.stdout: block_restart_interval: 16 2024-02-18T11:31:01.684 INFO:journalctl@ceph.mon.a.smithi012.stdout: index_block_restart_interval: 1 2024-02-18T11:31:01.684 INFO:journalctl@ceph.mon.a.smithi012.stdout: metadata_block_size: 4096 2024-02-18T11:31:01.684 INFO:journalctl@ceph.mon.a.smithi012.stdout: partition_filters: 0 2024-02-18T11:31:01.684 INFO:journalctl@ceph.mon.a.smithi012.stdout: use_delta_encoding: 1 2024-02-18T11:31:01.684 INFO:journalctl@ceph.mon.a.smithi012.stdout: filter_policy: bloomfilter 2024-02-18T11:31:01.684 INFO:journalctl@ceph.mon.a.smithi012.stdout: whole_key_filtering: 1 2024-02-18T11:31:01.684 INFO:journalctl@ceph.mon.a.smithi012.stdout: verify_compression: 0 2024-02-18T11:31:01.684 INFO:journalctl@ceph.mon.a.smithi012.stdout: read_amp_bytes_per_bit: 0 2024-02-18T11:31:01.685 INFO:journalctl@ceph.mon.a.smithi012.stdout: format_version: 5 2024-02-18T11:31:01.685 INFO:journalctl@ceph.mon.a.smithi012.stdout: enable_index_compression: 1 2024-02-18T11:31:01.685 INFO:journalctl@ceph.mon.a.smithi012.stdout: block_align: 0 2024-02-18T11:31:01.685 INFO:journalctl@ceph.mon.a.smithi012.stdout: max_auto_readahead_size: 262144 2024-02-18T11:31:01.685 INFO:journalctl@ceph.mon.a.smithi012.stdout: prepopulate_block_cache: 0 2024-02-18T11:31:01.685 INFO:journalctl@ceph.mon.a.smithi012.stdout: initial_auto_readahead_size: 8192 2024-02-18T11:31:01.685 INFO:journalctl@ceph.mon.a.smithi012.stdout: num_file_reads_for_auto_readahead: 2 2024-02-18T11:31:01.685 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.write_buffer_size: 33554432 2024-02-18T11:31:01.685 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.max_write_buffer_number: 2 2024-02-18T11:31:01.685 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.compression: NoCompression 2024-02-18T11:31:01.685 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.bottommost_compression: Disabled 2024-02-18T11:31:01.685 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.prefix_extractor: nullptr 2024-02-18T11:31:01.686 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-18T11:31:01.686 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.num_levels: 7 2024-02-18T11:31:01.686 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-18T11:31:01.686 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-18T11:31:01.686 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-18T11:31:01.686 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-18T11:31:01.686 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-18T11:31:01.686 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-18T11:31:01.686 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-18T11:31:01.686 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-18T11:31:01.686 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-18T11:31:01.686 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-18T11:31:01.686 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-02-18T11:31:01.686 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-02-18T11:31:01.687 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-18T11:31:01.687 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.compression_opts.level: 32767 2024-02-18T11:31:01.687 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.compression_opts.strategy: 0 2024-02-18T11:31:01.687 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-18T11:31:01.687 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-18T11:31:01.687 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-02-18T11:31:01.687 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-18T11:31:01.687 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.compression_opts.enabled: false 2024-02-18T11:31:01.687 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-02-18T11:31:01.687 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-18T11:31:01.687 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-18T11:31:01.687 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-18T11:31:01.688 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.target_file_size_base: 67108864 2024-02-18T11:31:01.688 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-18T11:31:01.688 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-18T11:31:01.688 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-18T11:31:01.688 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-18T11:31:01.688 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-18T11:31:01.688 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-18T11:31:01.688 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-18T11:31:01.688 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-18T11:31:01.688 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-18T11:31:01.689 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-18T11:31:01.689 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-18T11:31:01.689 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-18T11:31:01.689 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-18T11:31:01.689 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-02-18T11:31:01.689 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.arena_block_size: 1048576 2024-02-18T11:31:01.689 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-18T11:31:01.689 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-18T11:31:01.689 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.disable_auto_compactions: 0 2024-02-18T11:31:01.689 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-18T11:31:01.689 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-18T11:31:01.689 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-18T11:31:01.689 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-18T11:31:01.690 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-18T11:31:01.690 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-18T11:31:01.690 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-18T11:31:01.690 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-18T11:31:01.690 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-18T11:31:01.690 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-18T11:31:01.690 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.table_properties_collectors: 2024-02-18T11:31:01.690 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.inplace_update_support: 0 2024-02-18T11:31:01.690 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-18T11:31:01.690 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-18T11:31:01.690 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-18T11:31:01.690 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-18T11:31:01.690 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.bloom_locality: 0 2024-02-18T11:31:01.690 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.max_successive_merges: 0 2024-02-18T11:31:01.691 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-18T11:31:01.691 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.paranoid_file_checks: 0 2024-02-18T11:31:01.691 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.force_consistency_checks: 1 2024-02-18T11:31:01.691 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.report_bg_io_stats: 0 2024-02-18T11:31:01.691 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.ttl: 2592000 2024-02-18T11:31:01.691 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-18T11:31:01.691 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-02-18T11:31:01.691 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-02-18T11:31:01.691 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.enable_blob_files: false 2024-02-18T11:31:01.691 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.min_blob_size: 0 2024-02-18T11:31:01.691 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.blob_file_size: 268435456 2024-02-18T11:31:01.691 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-18T11:31:01.691 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-18T11:31:01.691 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-18T11:31:01.691 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-02-18T11:31:01.692 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-02-18T11:31:01.692 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.blob_file_starting_level: 0 2024-02-18T11:31:01.692 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-02-18T11:31:01.692 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-02-18T11:31:01.692 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-02-18T11:31:01.692 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 6bebd3f3-50f2-4116-b772-e12e199047cd 2024-02-18T11:31:01.692 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708255861422793, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-02-18T11:31:01.692 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-02-18T11:31:01.692 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708255861423787, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 95922, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 326, "table_properties": {"data_size": 93911, "index_size": 293, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 14049, "raw_average_key_size": 49, "raw_value_size": 86271, "raw_average_value_size": 307, "num_data_blocks": 11, "num_entries": 281, "num_filter_entries": 281, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1708255861, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "6bebd3f3-50f2-4116-b772-e12e199047cd", "db_session_id": "OVR0FZWK680O0H3KXAJX", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-02-18T11:31:01.692 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708255861423870, "job": 1, "event": "recovery_finished"} 2024-02-18T11:31:01.692 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-02-18T11:31:01.692 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-18T11:31:01.692 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x560f43e60000 2024-02-18T11:31:01.692 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: DB pointer 0x560f43e4a000 2024-02-18T11:31:01.693 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-02-18T11:31:01.693 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-02-18T11:31:01.693 INFO:journalctl@ceph.mon.a.smithi012.stdout: ** DB Stats ** 2024-02-18T11:31:01.693 INFO:journalctl@ceph.mon.a.smithi012.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-18T11:31:01.693 INFO:journalctl@ceph.mon.a.smithi012.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-18T11:31:01.693 INFO:journalctl@ceph.mon.a.smithi012.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-18T11:31:01.693 INFO:journalctl@ceph.mon.a.smithi012.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-18T11:31:01.693 INFO:journalctl@ceph.mon.a.smithi012.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-18T11:31:01.693 INFO:journalctl@ceph.mon.a.smithi012.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-18T11:31:01.693 INFO:journalctl@ceph.mon.a.smithi012.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-18T11:31:01.693 INFO:journalctl@ceph.mon.a.smithi012.stdout: 2024-02-18T11:31:01.693 INFO:journalctl@ceph.mon.a.smithi012.stdout: ** Compaction Stats [default] ** 2024-02-18T11:31:01.694 INFO:journalctl@ceph.mon.a.smithi012.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-18T11:31:01.694 INFO:journalctl@ceph.mon.a.smithi012.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-02-18T11:31:01.694 INFO:journalctl@ceph.mon.a.smithi012.stdout: L0 2/0 95.51 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 147.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-18T11:31:01.694 INFO:journalctl@ceph.mon.a.smithi012.stdout: Sum 2/0 95.51 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 147.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-18T11:31:01.694 INFO:journalctl@ceph.mon.a.smithi012.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 147.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-18T11:31:01.694 INFO:journalctl@ceph.mon.a.smithi012.stdout: 2024-02-18T11:31:01.694 INFO:journalctl@ceph.mon.a.smithi012.stdout: ** Compaction Stats [default] ** 2024-02-18T11:31:01.694 INFO:journalctl@ceph.mon.a.smithi012.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-18T11:31:01.694 INFO:journalctl@ceph.mon.a.smithi012.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-18T11:31:01.694 INFO:journalctl@ceph.mon.a.smithi012.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 147.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-18T11:31:01.694 INFO:journalctl@ceph.mon.a.smithi012.stdout: 2024-02-18T11:31:01.694 INFO:journalctl@ceph.mon.a.smithi012.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-02-18T11:31:01.694 INFO:journalctl@ceph.mon.a.smithi012.stdout: 2024-02-18T11:31:01.694 INFO:journalctl@ceph.mon.a.smithi012.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-18T11:31:01.695 INFO:journalctl@ceph.mon.a.smithi012.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-02-18T11:31:01.695 INFO:journalctl@ceph.mon.a.smithi012.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-18T11:31:01.695 INFO:journalctl@ceph.mon.a.smithi012.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-18T11:31:01.695 INFO:journalctl@ceph.mon.a.smithi012.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-18T11:31:01.695 INFO:journalctl@ceph.mon.a.smithi012.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-18T11:31:01.695 INFO:journalctl@ceph.mon.a.smithi012.stdout: Cumulative compaction: 0.00 GB write, 31.07 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-18T11:31:01.695 INFO:journalctl@ceph.mon.a.smithi012.stdout: Interval compaction: 0.00 GB write, 31.07 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-18T11:31:01.695 INFO:journalctl@ceph.mon.a.smithi012.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-18T11:31:01.695 INFO:journalctl@ceph.mon.a.smithi012.stdout: Block cache BinnedLRUCache@0x560f43d6f090#2 capacity: 512.00 MB usage: 1.39 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-02-18T11:31:01.695 INFO:journalctl@ceph.mon.a.smithi012.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-02-18T11:31:01.695 INFO:journalctl@ceph.mon.a.smithi012.stdout: 2024-02-18T11:31:01.695 INFO:journalctl@ceph.mon.a.smithi012.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-18T11:31:01.695 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: starting mon.a rank 0 at public addrs [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] at bind addrs [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 2024-02-18T11:31:01.695 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: mon.a@-1(???) e1 preinit fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 2024-02-18T11:31:01.696 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: mon.a@-1(???).mds e1 new map 2024-02-18T11:31:01.696 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: mon.a@-1(???).mds e1 print_map 2024-02-18T11:31:01.696 INFO:journalctl@ceph.mon.a.smithi012.stdout: e1 2024-02-18T11:31:01.696 INFO:journalctl@ceph.mon.a.smithi012.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-18T11:31:01.696 INFO:journalctl@ceph.mon.a.smithi012.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-18T11:31:01.696 INFO:journalctl@ceph.mon.a.smithi012.stdout: legacy client fscid: -1 2024-02-18T11:31:01.696 INFO:journalctl@ceph.mon.a.smithi012.stdout: 2024-02-18T11:31:01.696 INFO:journalctl@ceph.mon.a.smithi012.stdout: No filesystems configured 2024-02-18T11:31:01.696 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-18T11:31:01.696 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-18T11:31:01.696 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-18T11:31:01.696 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-18T11:31:01.696 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-02-18T11:31:01.696 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 bash[32442]: 74f2d580255e5a641a85532de12b838833c585e2189c06128eeb7fad38e2d61f 2024-02-18T11:31:01.697 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 systemd[1]: Started Ceph mon.a for 0c8bb0de-ce51-11ee-95ba-87774f69a715. 2024-02-18T11:31:01.697 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: mon.a is new leader, mons a in quorum (ranks 0) 2024-02-18T11:31:01.697 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: monmap e1: 1 mons at {a=[v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-18T11:31:01.697 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: fsmap 2024-02-18T11:31:01.697 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: osdmap e1: 0 total, 0 up, 0 in 2024-02-18T11:31:01.697 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:01 smithi012 ceph-mon[32460]: mgrmap e1: no daemons active 2024-02-18T11:31:03.053 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:01.889+0000 7f925a1bb700 1 Processor -- start 2024-02-18T11:31:03.058 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:01.889+0000 7f925a1bb700 1 -- start start 2024-02-18T11:31:03.058 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:01.889+0000 7f925a1bb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92541065b0 0x7f92541069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:31:03.058 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:01.889+0000 7f925a1bb700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9254106fa0 con 0x7f92541065b0 2024-02-18T11:31:03.058 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:01.889+0000 7f92537fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92541065b0 0x7f92541069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:31:03.058 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:01.889+0000 7f92537fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92541065b0 0x7f92541069d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43452/0 (socket says 172.21.15.12:43452) 2024-02-18T11:31:03.058 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:01.889+0000 7f92537fe700 1 -- 172.21.15.12:0/3462122332 learned_addr learned my addr 172.21.15.12:0/3462122332 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:31:03.058 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:01.889+0000 7f92537fe700 1 -- 172.21.15.12:0/3462122332 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92541077b0 con 0x7f92541065b0 2024-02-18T11:31:03.058 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:01.890+0000 7f92537fe700 1 --2- 172.21.15.12:0/3462122332 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92541065b0 0x7f92541069d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f924400bb40 tx=0x7f924400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d356777a5646b6e server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:31:03.058 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:01.890+0000 7f92527fc700 1 -- 172.21.15.12:0/3462122332 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f924400dbe0 con 0x7f92541065b0 2024-02-18T11:31:03.059 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:01.890+0000 7f92527fc700 1 -- 172.21.15.12:0/3462122332 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f92440044f0 con 0x7f92541065b0 2024-02-18T11:31:03.059 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:01.890+0000 7f92527fc700 1 -- 172.21.15.12:0/3462122332 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9244010470 con 0x7f92541065b0 2024-02-18T11:31:03.059 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:01.890+0000 7f925a1bb700 1 -- 172.21.15.12:0/3462122332 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92541065b0 msgr2=0x7f92541069d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:31:03.059 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:01.890+0000 7f925a1bb700 1 --2- 172.21.15.12:0/3462122332 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92541065b0 0x7f92541069d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f924400bb40 tx=0x7f924400be50 comp rx=0 tx=0).stop 2024-02-18T11:31:03.059 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:01.890+0000 7f925a1bb700 1 -- 172.21.15.12:0/3462122332 shutdown_connections 2024-02-18T11:31:03.059 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:01.890+0000 7f925a1bb700 1 --2- 172.21.15.12:0/3462122332 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92541065b0 0x7f92541069d0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:31:03.059 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:01.890+0000 7f925a1bb700 1 -- 172.21.15.12:0/3462122332 >> 172.21.15.12:0/3462122332 conn(0x7f9254101bd0 msgr2=0x7f9254103ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:31:03.059 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:01.890+0000 7f925a1bb700 1 -- 172.21.15.12:0/3462122332 shutdown_connections 2024-02-18T11:31:03.059 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:01.890+0000 7f925a1bb700 1 -- 172.21.15.12:0/3462122332 wait complete. 2024-02-18T11:31:03.059 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:01.891+0000 7f925a1bb700 1 Processor -- start 2024-02-18T11:31:03.059 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:01.891+0000 7f925a1bb700 1 -- start start 2024-02-18T11:31:03.059 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:01.891+0000 7f925a1bb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92541065b0 0x7f925419d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:31:03.059 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:01.891+0000 7f925a1bb700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f925419dc50 con 0x7f92541065b0 2024-02-18T11:31:03.060 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:01.891+0000 7f92537fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92541065b0 0x7f925419d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:31:03.060 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:01.891+0000 7f92537fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92541065b0 0x7f925419d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43466/0 (socket says 172.21.15.12:43466) 2024-02-18T11:31:03.060 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:01.891+0000 7f92537fe700 1 -- 172.21.15.12:0/3421857520 learned_addr learned my addr 172.21.15.12:0/3421857520 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:31:03.060 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:01.892+0000 7f92537fe700 1 -- 172.21.15.12:0/3421857520 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f924400b7f0 con 0x7f92541065b0 2024-02-18T11:31:03.060 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:01.892+0000 7f92537fe700 1 --2- 172.21.15.12:0/3421857520 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92541065b0 0x7f925419d710 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f924400b340 tx=0x7f9244010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:31:03.060 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:01.892+0000 7f9250ff9700 1 -- 172.21.15.12:0/3421857520 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9244004380 con 0x7f92541065b0 2024-02-18T11:31:03.060 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:01.892+0000 7f9250ff9700 1 -- 172.21.15.12:0/3421857520 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9244010b60 con 0x7f92541065b0 2024-02-18T11:31:03.060 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:01.892+0000 7f925a1bb700 1 -- 172.21.15.12:0/3421857520 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f925419de50 con 0x7f92541065b0 2024-02-18T11:31:03.060 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:01.892+0000 7f9250ff9700 1 -- 172.21.15.12:0/3421857520 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9244021c70 con 0x7f92541065b0 2024-02-18T11:31:03.060 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:01.892+0000 7f925a1bb700 1 -- 172.21.15.12:0/3421857520 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f925419e290 con 0x7f92541065b0 2024-02-18T11:31:03.060 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:01.892+0000 7f9250ff9700 1 -- 172.21.15.12:0/3421857520 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f924402a3f0 con 0x7f92541065b0 2024-02-18T11:31:03.060 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:01.892+0000 7f9250ff9700 1 -- 172.21.15.12:0/3421857520 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f92440197a0 con 0x7f92541065b0 2024-02-18T11:31:03.060 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:01.893+0000 7f925a1bb700 1 -- 172.21.15.12:0/3421857520 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92540632e0 con 0x7f92541065b0 2024-02-18T11:31:03.061 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:01.895+0000 7f9250ff9700 1 -- 172.21.15.12:0/3421857520 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f924402f060 con 0x7f92541065b0 2024-02-18T11:31:03.061 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:01.966+0000 7f925a1bb700 1 -- 172.21.15.12:0/3421857520 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f92541929d0 con 0x7f92541065b0 2024-02-18T11:31:03.061 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:01.970+0000 7f9250ff9700 1 -- 172.21.15.12:0/3421857520 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f9244010cd0 con 0x7f92541065b0 2024-02-18T11:31:03.061 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:01.971+0000 7f925a1bb700 1 -- 172.21.15.12:0/3421857520 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92541065b0 msgr2=0x7f925419d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:31:03.061 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:01.971+0000 7f925a1bb700 1 --2- 172.21.15.12:0/3421857520 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92541065b0 0x7f925419d710 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f924400b340 tx=0x7f9244010590 comp rx=0 tx=0).stop 2024-02-18T11:31:03.061 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:01.971+0000 7f925a1bb700 1 -- 172.21.15.12:0/3421857520 shutdown_connections 2024-02-18T11:31:03.061 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:01.971+0000 7f925a1bb700 1 --2- 172.21.15.12:0/3421857520 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92541065b0 0x7f925419d710 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:31:03.061 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:01.971+0000 7f925a1bb700 1 -- 172.21.15.12:0/3421857520 >> 172.21.15.12:0/3421857520 conn(0x7f9254101bd0 msgr2=0x7f925418fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:31:03.061 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:01.971+0000 7f925a1bb700 1 -- 172.21.15.12:0/3421857520 shutdown_connections 2024-02-18T11:31:03.061 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:01.971+0000 7f925a1bb700 1 -- 172.21.15.12:0/3421857520 wait complete. 2024-02-18T11:31:03.061 INFO:teuthology.orchestra.run.smithi012.stdout:Wrote config to /etc/ceph/ceph.conf 2024-02-18T11:31:03.082 INFO:teuthology.orchestra.run.smithi012.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-02-18T11:31:03.082 INFO:teuthology.orchestra.run.smithi012.stdout:Creating mgr... 2024-02-18T11:31:03.083 INFO:teuthology.orchestra.run.smithi012.stdout:Verifying port 0.0.0.0:9283 ... 2024-02-18T11:31:03.084 INFO:teuthology.orchestra.run.smithi012.stdout:Verifying port 0.0.0.0:8765 ... 2024-02-18T11:31:03.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:02 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/3421857520' entity='client.admin' 2024-02-18T11:31:03.335 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@mgr.x 2024-02-18T11:31:03.335 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Failed to reset failed state of unit ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@mgr.x.service: Unit ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@mgr.x.service not loaded. 2024-02-18T11:31:03.485 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715.target.wants/ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@mgr.x.service → /etc/systemd/system/ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@.service. 2024-02-18T11:31:03.987 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-18T11:31:03.987 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout disabled 2024-02-18T11:31:03.994 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-18T11:31:03.994 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout inactive 2024-02-18T11:31:03.994 INFO:teuthology.orchestra.run.smithi012.stdout:firewalld.service is not enabled 2024-02-18T11:31:03.994 INFO:teuthology.orchestra.run.smithi012.stdout:Not possible to enable service . firewalld.service is not available 2024-02-18T11:31:04.002 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-18T11:31:04.002 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout disabled 2024-02-18T11:31:04.009 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-18T11:31:04.009 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout inactive 2024-02-18T11:31:04.009 INFO:teuthology.orchestra.run.smithi012.stdout:firewalld.service is not enabled 2024-02-18T11:31:04.009 INFO:teuthology.orchestra.run.smithi012.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-02-18T11:31:04.009 INFO:teuthology.orchestra.run.smithi012.stdout:Waiting for mgr to start... 2024-02-18T11:31:04.009 INFO:teuthology.orchestra.run.smithi012.stdout:Waiting for mgr... 2024-02-18T11:31:04.130 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:03 smithi012 systemd[1]: Started Ceph mgr.x for 0c8bb0de-ce51-11ee-95ba-87774f69a715. 2024-02-18T11:31:04.810 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:04 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/985545067' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-18T11:31:04.811 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:04 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:31:04.437+0000 7f845f79d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-18T11:31:04.811 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:04 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:31:04.536+0000 7f845f79d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-18T11:31:05.124 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-02-18T11:31:05.124 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout { 2024-02-18T11:31:05.124 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "fsid": "0c8bb0de-ce51-11ee-95ba-87774f69a715", 2024-02-18T11:31:05.124 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "health": { 2024-02-18T11:31:05.124 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-18T11:31:05.124 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-18T11:31:05.124 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-18T11:31:05.124 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-02-18T11:31:05.125 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-18T11:31:05.125 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-18T11:31:05.125 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 0 2024-02-18T11:31:05.125 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-02-18T11:31:05.125 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-18T11:31:05.125 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "a" 2024-02-18T11:31:05.125 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-02-18T11:31:05.125 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-02-18T11:31:05.125 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-18T11:31:05.125 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-18T11:31:05.125 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-18T11:31:05.125 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-18T11:31:05.125 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-02-18T11:31:05.125 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-18T11:31:05.125 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-18T11:31:05.126 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-18T11:31:05.126 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-18T11:31:05.126 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-18T11:31:05.126 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-18T11:31:05.126 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-18T11:31:05.126 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-18T11:31:05.126 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-02-18T11:31:05.126 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-18T11:31:05.126 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-18T11:31:05.126 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-18T11:31:05.126 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-18T11:31:05.126 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-18T11:31:05.126 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-18T11:31:05.126 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-18T11:31:05.126 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-18T11:31:05.127 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-18T11:31:05.127 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-02-18T11:31:05.127 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-18T11:31:05.127 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-18T11:31:05.127 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-18T11:31:05.127 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-18T11:31:05.127 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-02-18T11:31:05.127 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-18T11:31:05.127 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-18T11:31:05.127 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-18T11:31:05.127 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-18T11:31:05.127 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-18T11:31:05.127 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-18T11:31:05.127 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "restful" 2024-02-18T11:31:05.127 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-02-18T11:31:05.128 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-18T11:31:05.128 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-02-18T11:31:05.128 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-18T11:31:05.128 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-18T11:31:05.128 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "modified": "2024-02-18T11:30:49.110551+0000", 2024-02-18T11:31:05.128 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-18T11:31:05.128 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-02-18T11:31:05.128 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-18T11:31:05.128 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout } 2024-02-18T11:31:05.128 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:04.456+0000 7fd6b1a85700 1 Processor -- start 2024-02-18T11:31:05.128 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:04.456+0000 7fd6b1a85700 1 -- start start 2024-02-18T11:31:05.128 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:04.456+0000 7fd6b1a85700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6ac1065b0 0x7fd6ac1069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:31:05.128 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:04.456+0000 7fd6b1a85700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ac106fa0 con 0x7fd6ac1065b0 2024-02-18T11:31:05.128 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:04.456+0000 7fd6ab7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6ac1065b0 0x7fd6ac1069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:31:05.128 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:04.456+0000 7fd6ab7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6ac1065b0 0x7fd6ac1069d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43488/0 (socket says 172.21.15.12:43488) 2024-02-18T11:31:05.129 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:04.456+0000 7fd6ab7fe700 1 -- 172.21.15.12:0/215897267 learned_addr learned my addr 172.21.15.12:0/215897267 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:31:05.129 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:04.457+0000 7fd6ab7fe700 1 -- 172.21.15.12:0/215897267 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6ac1077b0 con 0x7fd6ac1065b0 2024-02-18T11:31:05.129 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:04.457+0000 7fd6ab7fe700 1 --2- 172.21.15.12:0/215897267 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6ac1065b0 0x7fd6ac1069d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fd69c0098a0 tx=0x7fd69c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5e40080c02df6a6 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:31:05.129 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:04.457+0000 7fd6aa7fc700 1 -- 172.21.15.12:0/215897267 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd69c004160 con 0x7fd6ac1065b0 2024-02-18T11:31:05.129 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:04.457+0000 7fd6aa7fc700 1 -- 172.21.15.12:0/215897267 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd69c0042c0 con 0x7fd6ac1065b0 2024-02-18T11:31:05.129 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:04.457+0000 7fd6aa7fc700 1 -- 172.21.15.12:0/215897267 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd69c011ac0 con 0x7fd6ac1065b0 2024-02-18T11:31:05.129 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:04.458+0000 7fd6b1a85700 1 -- 172.21.15.12:0/215897267 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6ac1065b0 msgr2=0x7fd6ac1069d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:31:05.129 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:04.458+0000 7fd6b1a85700 1 --2- 172.21.15.12:0/215897267 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6ac1065b0 0x7fd6ac1069d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fd69c0098a0 tx=0x7fd69c009bb0 comp rx=0 tx=0).stop 2024-02-18T11:31:05.129 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:04.458+0000 7fd6b1a85700 1 -- 172.21.15.12:0/215897267 shutdown_connections 2024-02-18T11:31:05.129 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:04.458+0000 7fd6b1a85700 1 --2- 172.21.15.12:0/215897267 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6ac1065b0 0x7fd6ac1069d0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:31:05.129 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:04.458+0000 7fd6b1a85700 1 -- 172.21.15.12:0/215897267 >> 172.21.15.12:0/215897267 conn(0x7fd6ac101b70 msgr2=0x7fd6ac103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:31:05.129 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:04.458+0000 7fd6b1a85700 1 -- 172.21.15.12:0/215897267 shutdown_connections 2024-02-18T11:31:05.129 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:04.458+0000 7fd6b1a85700 1 -- 172.21.15.12:0/215897267 wait complete. 2024-02-18T11:31:05.129 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:04.459+0000 7fd6b1a85700 1 Processor -- start 2024-02-18T11:31:05.130 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:04.459+0000 7fd6b1a85700 1 -- start start 2024-02-18T11:31:05.130 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:04.459+0000 7fd6b1a85700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6ac1065b0 0x7fd6ac191720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:31:05.130 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:04.459+0000 7fd6b1a85700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ac191c60 con 0x7fd6ac1065b0 2024-02-18T11:31:05.130 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:04.459+0000 7fd6ab7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6ac1065b0 0x7fd6ac191720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:31:05.130 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:04.459+0000 7fd6ab7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6ac1065b0 0x7fd6ac191720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43502/0 (socket says 172.21.15.12:43502) 2024-02-18T11:31:05.130 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:04.459+0000 7fd6ab7fe700 1 -- 172.21.15.12:0/985545067 learned_addr learned my addr 172.21.15.12:0/985545067 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:31:05.130 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:04.459+0000 7fd6ab7fe700 1 -- 172.21.15.12:0/985545067 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd69c009550 con 0x7fd6ac1065b0 2024-02-18T11:31:05.130 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:04.460+0000 7fd6ab7fe700 1 --2- 172.21.15.12:0/985545067 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6ac1065b0 0x7fd6ac191720 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd69c003f70 tx=0x7fd69c011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:31:05.130 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:04.460+0000 7fd6a8ff9700 1 -- 172.21.15.12:0/985545067 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd69c0039f0 con 0x7fd6ac1065b0 2024-02-18T11:31:05.130 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:04.460+0000 7fd6b1a85700 1 -- 172.21.15.12:0/985545067 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6ac18fe50 con 0x7fd6ac1065b0 2024-02-18T11:31:05.130 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:04.460+0000 7fd6a8ff9700 1 -- 172.21.15.12:0/985545067 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd69c003b50 con 0x7fd6ac1065b0 2024-02-18T11:31:05.130 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:04.460+0000 7fd6b1a85700 1 -- 172.21.15.12:0/985545067 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6ac1902f0 con 0x7fd6ac1065b0 2024-02-18T11:31:05.130 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:04.460+0000 7fd6a8ff9700 1 -- 172.21.15.12:0/985545067 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd69c01a690 con 0x7fd6ac1065b0 2024-02-18T11:31:05.130 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:04.461+0000 7fd6a8ff9700 1 -- 172.21.15.12:0/985545067 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fd69c01ad20 con 0x7fd6ac1065b0 2024-02-18T11:31:05.131 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:04.461+0000 7fd6a8ff9700 1 -- 172.21.15.12:0/985545067 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd69c019a10 con 0x7fd6ac1065b0 2024-02-18T11:31:05.131 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:04.461+0000 7fd6b1a85700 1 -- 172.21.15.12:0/985545067 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6ac068da0 con 0x7fd6ac1065b0 2024-02-18T11:31:05.131 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:04.463+0000 7fd6a8ff9700 1 -- 172.21.15.12:0/985545067 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fd69c0249f0 con 0x7fd6ac1065b0 2024-02-18T11:31:05.131 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:04.537+0000 7fd6b1a85700 1 -- 172.21.15.12:0/985545067 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fd6ac10ac10 con 0x7fd6ac1065b0 2024-02-18T11:31:05.131 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:04.537+0000 7fd6a8ff9700 1 -- 172.21.15.12:0/985545067 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7fd69c019c20 con 0x7fd6ac1065b0 2024-02-18T11:31:05.131 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:04.538+0000 7fd6b1a85700 1 -- 172.21.15.12:0/985545067 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6ac1065b0 msgr2=0x7fd6ac191720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:31:05.131 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:04.538+0000 7fd6b1a85700 1 --2- 172.21.15.12:0/985545067 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6ac1065b0 0x7fd6ac191720 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd69c003f70 tx=0x7fd69c011b40 comp rx=0 tx=0).stop 2024-02-18T11:31:05.131 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:04.538+0000 7fd6b1a85700 1 -- 172.21.15.12:0/985545067 shutdown_connections 2024-02-18T11:31:05.131 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:04.538+0000 7fd6b1a85700 1 --2- 172.21.15.12:0/985545067 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6ac1065b0 0x7fd6ac191720 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:31:05.131 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:04.538+0000 7fd6b1a85700 1 -- 172.21.15.12:0/985545067 >> 172.21.15.12:0/985545067 conn(0x7fd6ac101b70 msgr2=0x7fd6ac102a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:31:05.131 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:04.539+0000 7fd6b1a85700 1 -- 172.21.15.12:0/985545067 shutdown_connections 2024-02-18T11:31:05.131 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:04.539+0000 7fd6b1a85700 1 -- 172.21.15.12:0/985545067 wait complete. 2024-02-18T11:31:05.131 INFO:teuthology.orchestra.run.smithi012.stdout:mgr not available, waiting (1/15)... 2024-02-18T11:31:05.424 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:05 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:31:05.208+0000 7f845f79d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-18T11:31:05.425 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:05 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:31:05.303+0000 7f845f79d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-18T11:31:05.894 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:05 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:31:05.447+0000 7f845f79d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-18T11:31:05.894 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:05 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:31:05.545+0000 7f845f79d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-18T11:31:05.895 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:05 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:31:05.630+0000 7f845f79d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-18T11:31:05.895 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:05 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:31:05.897+0000 7f845f79d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-18T11:31:06.174 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:06 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:31:06.010+0000 7f845f79d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-18T11:31:06.174 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:06 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:31:06.098+0000 7f845f79d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-18T11:31:06.674 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:06 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:31:06.195+0000 7f845f79d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-18T11:31:06.674 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:06 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:31:06.373+0000 7f845f79d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-18T11:31:07.514 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:07 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:31:07.381+0000 7f845f79d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-18T11:31:07.806 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:07 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/702789033' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-18T11:31:07.806 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:07 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:31:07.518+0000 7f845f79d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-18T11:31:08.154 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:07 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:31:07.865+0000 7f845f79d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-18T11:31:08.154 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:07 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:31:07.958+0000 7f845f79d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-18T11:31:08.154 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:08 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:31:08.158+0000 7f845f79d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-18T11:31:08.413 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:08 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:31:08.347+0000 7f845f79d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-18T11:31:08.426 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-02-18T11:31:08.426 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout { 2024-02-18T11:31:08.426 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "fsid": "0c8bb0de-ce51-11ee-95ba-87774f69a715", 2024-02-18T11:31:08.426 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "health": { 2024-02-18T11:31:08.426 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-18T11:31:08.426 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-18T11:31:08.426 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-18T11:31:08.426 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-02-18T11:31:08.426 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-18T11:31:08.427 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-18T11:31:08.427 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 0 2024-02-18T11:31:08.427 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-02-18T11:31:08.427 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-18T11:31:08.427 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "a" 2024-02-18T11:31:08.427 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-02-18T11:31:08.427 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-02-18T11:31:08.427 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-18T11:31:08.427 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-18T11:31:08.428 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-18T11:31:08.428 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-18T11:31:08.428 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-02-18T11:31:08.428 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-18T11:31:08.428 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-18T11:31:08.428 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-18T11:31:08.428 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-18T11:31:08.428 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-18T11:31:08.428 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-18T11:31:08.431 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-18T11:31:08.431 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-18T11:31:08.432 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-02-18T11:31:08.432 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-18T11:31:08.432 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-18T11:31:08.432 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-18T11:31:08.432 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-18T11:31:08.432 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-18T11:31:08.432 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-18T11:31:08.432 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-18T11:31:08.432 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-18T11:31:08.432 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-18T11:31:08.432 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-02-18T11:31:08.432 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-18T11:31:08.432 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-18T11:31:08.433 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-18T11:31:08.433 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-18T11:31:08.433 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-02-18T11:31:08.433 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-18T11:31:08.433 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-18T11:31:08.433 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-18T11:31:08.433 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-18T11:31:08.433 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-18T11:31:08.433 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-18T11:31:08.433 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "restful" 2024-02-18T11:31:08.433 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-02-18T11:31:08.434 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-18T11:31:08.434 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-02-18T11:31:08.434 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-18T11:31:08.434 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-18T11:31:08.434 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "modified": "2024-02-18T11:30:49.110551+0000", 2024-02-18T11:31:08.434 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-18T11:31:08.434 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-02-18T11:31:08.434 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-18T11:31:08.434 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout } 2024-02-18T11:31:08.434 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:07.518+0000 7ff2c254f700 1 Processor -- start 2024-02-18T11:31:08.434 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:07.518+0000 7ff2c254f700 1 -- start start 2024-02-18T11:31:08.434 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:07.518+0000 7ff2c254f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2bc1065d0 0x7ff2bc1069f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:31:08.435 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:07.518+0000 7ff2c254f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2bc106fc0 con 0x7ff2bc1065d0 2024-02-18T11:31:08.435 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:07.518+0000 7ff2bbfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2bc1065d0 0x7ff2bc1069f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:31:08.435 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:07.518+0000 7ff2bbfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2bc1065d0 0x7ff2bc1069f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43512/0 (socket says 172.21.15.12:43512) 2024-02-18T11:31:08.435 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:07.518+0000 7ff2bbfff700 1 -- 172.21.15.12:0/2906391300 learned_addr learned my addr 172.21.15.12:0/2906391300 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:31:08.435 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:07.518+0000 7ff2bbfff700 1 -- 172.21.15.12:0/2906391300 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2bc1077d0 con 0x7ff2bc1065d0 2024-02-18T11:31:08.435 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:07.519+0000 7ff2bbfff700 1 --2- 172.21.15.12:0/2906391300 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2bc1065d0 0x7ff2bc1069f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac0098a0 tx=0x7ff2ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeb7ab357db7a4f7 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:31:08.435 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:07.519+0000 7ff2baffd700 1 -- 172.21.15.12:0/2906391300 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff2ac004160 con 0x7ff2bc1065d0 2024-02-18T11:31:08.435 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:07.519+0000 7ff2baffd700 1 -- 172.21.15.12:0/2906391300 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff2ac0042c0 con 0x7ff2bc1065d0 2024-02-18T11:31:08.435 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:07.519+0000 7ff2baffd700 1 -- 172.21.15.12:0/2906391300 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff2ac011ac0 con 0x7ff2bc1065d0 2024-02-18T11:31:08.435 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:07.519+0000 7ff2c254f700 1 -- 172.21.15.12:0/2906391300 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2bc1065d0 msgr2=0x7ff2bc1069f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:31:08.435 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:07.519+0000 7ff2c254f700 1 --2- 172.21.15.12:0/2906391300 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2bc1065d0 0x7ff2bc1069f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac0098a0 tx=0x7ff2ac009bb0 comp rx=0 tx=0).stop 2024-02-18T11:31:08.435 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:07.519+0000 7ff2c254f700 1 -- 172.21.15.12:0/2906391300 shutdown_connections 2024-02-18T11:31:08.436 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:07.519+0000 7ff2c254f700 1 --2- 172.21.15.12:0/2906391300 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2bc1065d0 0x7ff2bc1069f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:31:08.436 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:07.519+0000 7ff2c254f700 1 -- 172.21.15.12:0/2906391300 >> 172.21.15.12:0/2906391300 conn(0x7ff2bc101c10 msgr2=0x7ff2bc104030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:31:08.436 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:07.520+0000 7ff2c254f700 1 -- 172.21.15.12:0/2906391300 shutdown_connections 2024-02-18T11:31:08.436 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:07.520+0000 7ff2c254f700 1 -- 172.21.15.12:0/2906391300 wait complete. 2024-02-18T11:31:08.436 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:07.520+0000 7ff2c254f700 1 Processor -- start 2024-02-18T11:31:08.436 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:07.520+0000 7ff2c254f700 1 -- start start 2024-02-18T11:31:08.436 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:07.520+0000 7ff2c254f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2bc1065d0 0x7ff2bc19b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:31:08.436 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:07.520+0000 7ff2c254f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2bc19bac0 con 0x7ff2bc1065d0 2024-02-18T11:31:08.436 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:07.521+0000 7ff2bbfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2bc1065d0 0x7ff2bc19b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:31:08.436 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:07.521+0000 7ff2bbfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2bc1065d0 0x7ff2bc19b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43516/0 (socket says 172.21.15.12:43516) 2024-02-18T11:31:08.436 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:07.521+0000 7ff2bbfff700 1 -- 172.21.15.12:0/702789033 learned_addr learned my addr 172.21.15.12:0/702789033 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:31:08.436 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:07.521+0000 7ff2bbfff700 1 -- 172.21.15.12:0/702789033 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2ac009550 con 0x7ff2bc1065d0 2024-02-18T11:31:08.436 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:07.521+0000 7ff2bbfff700 1 --2- 172.21.15.12:0/702789033 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2bc1065d0 0x7ff2bc19b580 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac011780 tx=0x7ff2ac011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:31:08.437 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:07.521+0000 7ff2b97fa700 1 -- 172.21.15.12:0/702789033 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff2ac003af0 con 0x7ff2bc1065d0 2024-02-18T11:31:08.437 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:07.521+0000 7ff2b97fa700 1 -- 172.21.15.12:0/702789033 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff2ac003c50 con 0x7ff2bc1065d0 2024-02-18T11:31:08.437 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:07.521+0000 7ff2b97fa700 1 -- 172.21.15.12:0/702789033 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff2ac0197e0 con 0x7ff2bc1065d0 2024-02-18T11:31:08.437 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:07.521+0000 7ff2c254f700 1 -- 172.21.15.12:0/702789033 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2bc19bcc0 con 0x7ff2bc1065d0 2024-02-18T11:31:08.437 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:07.521+0000 7ff2c254f700 1 -- 172.21.15.12:0/702789033 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2bc19c160 con 0x7ff2bc1065d0 2024-02-18T11:31:08.437 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:07.521+0000 7ff2b97fa700 1 -- 172.21.15.12:0/702789033 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7ff2ac025460 con 0x7ff2bc1065d0 2024-02-18T11:31:08.437 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:07.521+0000 7ff2b97fa700 1 -- 172.21.15.12:0/702789033 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff2ac00bc50 con 0x7ff2bc1065d0 2024-02-18T11:31:08.437 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:07.522+0000 7ff2c254f700 1 -- 172.21.15.12:0/702789033 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2bc195670 con 0x7ff2bc1065d0 2024-02-18T11:31:08.437 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:07.525+0000 7ff2b97fa700 1 -- 172.21.15.12:0/702789033 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7ff2ac02a060 con 0x7ff2bc1065d0 2024-02-18T11:31:08.437 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:07.599+0000 7ff2c254f700 1 -- 172.21.15.12:0/702789033 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7ff2bc10ac90 con 0x7ff2bc1065d0 2024-02-18T11:31:08.437 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:07.599+0000 7ff2b97fa700 1 -- 172.21.15.12:0/702789033 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7ff2ac00be60 con 0x7ff2bc1065d0 2024-02-18T11:31:08.437 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:07.600+0000 7ff2c254f700 1 -- 172.21.15.12:0/702789033 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2bc1065d0 msgr2=0x7ff2bc19b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:31:08.438 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:07.600+0000 7ff2c254f700 1 --2- 172.21.15.12:0/702789033 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2bc1065d0 0x7ff2bc19b580 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac011780 tx=0x7ff2ac011b40 comp rx=0 tx=0).stop 2024-02-18T11:31:08.438 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:07.600+0000 7ff2c254f700 1 -- 172.21.15.12:0/702789033 shutdown_connections 2024-02-18T11:31:08.438 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:07.600+0000 7ff2c254f700 1 --2- 172.21.15.12:0/702789033 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2bc1065d0 0x7ff2bc19b580 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:31:08.438 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:07.600+0000 7ff2c254f700 1 -- 172.21.15.12:0/702789033 >> 172.21.15.12:0/702789033 conn(0x7ff2bc101c10 msgr2=0x7ff2bc102af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:31:08.438 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:07.600+0000 7ff2c254f700 1 -- 172.21.15.12:0/702789033 shutdown_connections 2024-02-18T11:31:08.438 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:07.600+0000 7ff2c254f700 1 -- 172.21.15.12:0/702789033 wait complete. 2024-02-18T11:31:08.438 INFO:teuthology.orchestra.run.smithi012.stdout:mgr not available, waiting (2/15)... 2024-02-18T11:31:08.674 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:08 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:31:08.638+0000 7f845f79d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-18T11:31:09.174 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:08 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:31:08.899+0000 7f845f79d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-18T11:31:09.175 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:09 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:31:09.141+0000 7f845f79d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-18T11:31:10.424 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:09 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:31:09.972+0000 7f845f79d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-18T11:31:10.854 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:10 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:31:10.559+0000 7f845f79d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-18T11:31:10.854 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:10 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:31:10.661+0000 7f845f79d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-18T11:31:11.174 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:10 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:31:10.888+0000 7f845f79d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-18T11:31:11.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:10 smithi012 ceph-mon[32460]: Activating manager daemon x 2024-02-18T11:31:11.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:10 smithi012 ceph-mon[32460]: mgrmap e2: x(active, starting, since 0.00323031s) 2024-02-18T11:31:11.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:10 smithi012 ceph-mon[32460]: from='mgr.14100 172.21.15.12:0/90894279' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-18T11:31:11.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:10 smithi012 ceph-mon[32460]: from='mgr.14100 172.21.15.12:0/90894279' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-18T11:31:11.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:10 smithi012 ceph-mon[32460]: from='mgr.14100 172.21.15.12:0/90894279' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-18T11:31:11.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:10 smithi012 ceph-mon[32460]: from='mgr.14100 172.21.15.12:0/90894279' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-18T11:31:11.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:10 smithi012 ceph-mon[32460]: from='mgr.14100 172.21.15.12:0/90894279' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-18T11:31:11.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:10 smithi012 ceph-mon[32460]: Manager daemon x is now available 2024-02-18T11:31:11.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:10 smithi012 ceph-mon[32460]: from='mgr.14100 172.21.15.12:0/90894279' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-02-18T11:31:11.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:10 smithi012 ceph-mon[32460]: from='mgr.14100 172.21.15.12:0/90894279' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-02-18T11:31:11.176 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:10 smithi012 ceph-mon[32460]: from='mgr.14100 172.21.15.12:0/90894279' entity='mgr.x' 2024-02-18T11:31:11.176 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:10 smithi012 ceph-mon[32460]: from='mgr.14100 172.21.15.12:0/90894279' entity='mgr.x' 2024-02-18T11:31:11.505 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-02-18T11:31:11.505 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout { 2024-02-18T11:31:11.505 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "fsid": "0c8bb0de-ce51-11ee-95ba-87774f69a715", 2024-02-18T11:31:11.505 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "health": { 2024-02-18T11:31:11.505 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-18T11:31:11.506 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-18T11:31:11.506 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-18T11:31:11.506 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-02-18T11:31:11.506 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-18T11:31:11.506 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-18T11:31:11.506 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 0 2024-02-18T11:31:11.506 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-02-18T11:31:11.506 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-18T11:31:11.506 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "a" 2024-02-18T11:31:11.506 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-02-18T11:31:11.506 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-02-18T11:31:11.506 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-18T11:31:11.506 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-18T11:31:11.506 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-18T11:31:11.506 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-18T11:31:11.507 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-02-18T11:31:11.507 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-18T11:31:11.507 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-18T11:31:11.507 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-18T11:31:11.507 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-18T11:31:11.507 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-18T11:31:11.507 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-18T11:31:11.507 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-18T11:31:11.507 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-18T11:31:11.507 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-02-18T11:31:11.507 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-18T11:31:11.507 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-18T11:31:11.507 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-18T11:31:11.508 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-18T11:31:11.508 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-18T11:31:11.508 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-18T11:31:11.508 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-18T11:31:11.508 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-18T11:31:11.508 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-18T11:31:11.508 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-02-18T11:31:11.508 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-18T11:31:11.508 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-18T11:31:11.508 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-18T11:31:11.508 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-18T11:31:11.508 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-02-18T11:31:11.508 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-18T11:31:11.508 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-18T11:31:11.509 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-18T11:31:11.509 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-18T11:31:11.509 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-18T11:31:11.509 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-18T11:31:11.509 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "restful" 2024-02-18T11:31:11.509 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-02-18T11:31:11.509 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-18T11:31:11.509 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-02-18T11:31:11.509 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-18T11:31:11.509 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-18T11:31:11.509 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "modified": "2024-02-18T11:30:49.110551+0000", 2024-02-18T11:31:11.509 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-18T11:31:11.509 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-02-18T11:31:11.510 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-18T11:31:11.510 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout } 2024-02-18T11:31:11.510 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:10.882+0000 7fd7b8072700 1 Processor -- start 2024-02-18T11:31:11.510 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:10.882+0000 7fd7b8072700 1 -- start start 2024-02-18T11:31:11.510 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:10.883+0000 7fd7b8072700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7b01081a0 0x7fd7b010a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:31:11.510 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:10.883+0000 7fd7b8072700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b007ae40 con 0x7fd7b01081a0 2024-02-18T11:31:11.510 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:10.883+0000 7fd7b5e0e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7b01081a0 0x7fd7b010a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:31:11.510 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:10.883+0000 7fd7b5e0e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7b01081a0 0x7fd7b010a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43522/0 (socket says 172.21.15.12:43522) 2024-02-18T11:31:11.510 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:10.883+0000 7fd7b5e0e700 1 -- 172.21.15.12:0/214641754 learned_addr learned my addr 172.21.15.12:0/214641754 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:31:11.510 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:10.883+0000 7fd7b5e0e700 1 -- 172.21.15.12:0/214641754 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7b010aad0 con 0x7fd7b01081a0 2024-02-18T11:31:11.510 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:10.883+0000 7fd7b5e0e700 1 --2- 172.21.15.12:0/214641754 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7b01081a0 0x7fd7b010a590 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac0098a0 tx=0x7fd7ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=892bf08622ca70fb server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:31:11.510 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:10.883+0000 7fd7b4e0c700 1 -- 172.21.15.12:0/214641754 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd7ac004160 con 0x7fd7b01081a0 2024-02-18T11:31:11.511 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:10.883+0000 7fd7b4e0c700 1 -- 172.21.15.12:0/214641754 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd7ac0042c0 con 0x7fd7b01081a0 2024-02-18T11:31:11.511 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:10.883+0000 7fd7b4e0c700 1 -- 172.21.15.12:0/214641754 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd7ac011ac0 con 0x7fd7b01081a0 2024-02-18T11:31:11.511 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:10.884+0000 7fd7b8072700 1 -- 172.21.15.12:0/214641754 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7b01081a0 msgr2=0x7fd7b010a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:31:11.511 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:10.884+0000 7fd7b8072700 1 --2- 172.21.15.12:0/214641754 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7b01081a0 0x7fd7b010a590 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac0098a0 tx=0x7fd7ac009bb0 comp rx=0 tx=0).stop 2024-02-18T11:31:11.511 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:10.884+0000 7fd7b8072700 1 -- 172.21.15.12:0/214641754 shutdown_connections 2024-02-18T11:31:11.511 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:10.884+0000 7fd7b8072700 1 --2- 172.21.15.12:0/214641754 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7b01081a0 0x7fd7b010a590 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:31:11.511 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:10.884+0000 7fd7b8072700 1 -- 172.21.15.12:0/214641754 >> 172.21.15.12:0/214641754 conn(0x7fd7b0101bb0 msgr2=0x7fd7b0103fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:31:11.511 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:10.884+0000 7fd7b8072700 1 -- 172.21.15.12:0/214641754 shutdown_connections 2024-02-18T11:31:11.511 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:10.885+0000 7fd7b8072700 1 -- 172.21.15.12:0/214641754 wait complete. 2024-02-18T11:31:11.511 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:10.885+0000 7fd7b8072700 1 Processor -- start 2024-02-18T11:31:11.511 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:10.885+0000 7fd7b8072700 1 -- start start 2024-02-18T11:31:11.511 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:10.886+0000 7fd7b8072700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7b01081a0 0x7fd7b019b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:31:11.511 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:10.886+0000 7fd7b8072700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b019b9a0 con 0x7fd7b01081a0 2024-02-18T11:31:11.511 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:10.886+0000 7fd7b5e0e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7b01081a0 0x7fd7b019b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:31:11.511 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:10.886+0000 7fd7b5e0e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7b01081a0 0x7fd7b019b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43530/0 (socket says 172.21.15.12:43530) 2024-02-18T11:31:11.511 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:10.886+0000 7fd7b5e0e700 1 -- 172.21.15.12:0/2381994342 learned_addr learned my addr 172.21.15.12:0/2381994342 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:31:11.512 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:10.886+0000 7fd7b5e0e700 1 -- 172.21.15.12:0/2381994342 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7ac009550 con 0x7fd7b01081a0 2024-02-18T11:31:11.512 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:10.886+0000 7fd7b5e0e700 1 --2- 172.21.15.12:0/2381994342 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7b01081a0 0x7fd7b019b460 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac009130 tx=0x7fd7ac011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:31:11.512 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:10.886+0000 7fd7a2ffd700 1 -- 172.21.15.12:0/2381994342 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd7ac003aa0 con 0x7fd7b01081a0 2024-02-18T11:31:11.512 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:10.886+0000 7fd7a2ffd700 1 -- 172.21.15.12:0/2381994342 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd7ac003c00 con 0x7fd7b01081a0 2024-02-18T11:31:11.512 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:10.886+0000 7fd7b8072700 1 -- 172.21.15.12:0/2381994342 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7b019bba0 con 0x7fd7b01081a0 2024-02-18T11:31:11.512 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:10.886+0000 7fd7b8072700 1 -- 172.21.15.12:0/2381994342 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7b019c040 con 0x7fd7b01081a0 2024-02-18T11:31:11.512 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:10.886+0000 7fd7a2ffd700 1 -- 172.21.15.12:0/2381994342 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd7ac024800 con 0x7fd7b01081a0 2024-02-18T11:31:11.512 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:10.887+0000 7fd7a2ffd700 1 -- 172.21.15.12:0/2381994342 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fd7ac02d3f0 con 0x7fd7b01081a0 2024-02-18T11:31:11.512 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:10.887+0000 7fd7a2ffd700 1 -- 172.21.15.12:0/2381994342 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd7ac019770 con 0x7fd7b01081a0 2024-02-18T11:31:11.512 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:10.887+0000 7fd7b8072700 1 -- 172.21.15.12:0/2381994342 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7b010b880 con 0x7fd7b01081a0 2024-02-18T11:31:11.512 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:10.889+0000 7fd7a2ffd700 1 -- 172.21.15.12:0/2381994342 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fd7ac01abd0 con 0x7fd7b01081a0 2024-02-18T11:31:11.512 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:10.904+0000 7fd7a2ffd700 1 -- 172.21.15.12:0/2381994342 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mgrmap(e 2) v1 ==== 45847+0+0 (secure 0 0 0) 0x7fd7ac0199d0 con 0x7fd7b01081a0 2024-02-18T11:31:11.512 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:10.979+0000 7fd7b8072700 1 -- 172.21.15.12:0/2381994342 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fd7b010ba90 con 0x7fd7b01081a0 2024-02-18T11:31:11.513 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:10.980+0000 7fd7a2ffd700 1 -- 172.21.15.12:0/2381994342 <== mon.0 v2:172.21.15.12:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7fd7ac02d7e0 con 0x7fd7b01081a0 2024-02-18T11:31:11.513 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:10.980+0000 7fd7b8072700 1 -- 172.21.15.12:0/2381994342 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7b01081a0 msgr2=0x7fd7b019b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:31:11.513 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:10.980+0000 7fd7b8072700 1 --2- 172.21.15.12:0/2381994342 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7b01081a0 0x7fd7b019b460 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac009130 tx=0x7fd7ac011b40 comp rx=0 tx=0).stop 2024-02-18T11:31:11.513 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:10.980+0000 7fd7b8072700 1 -- 172.21.15.12:0/2381994342 shutdown_connections 2024-02-18T11:31:11.513 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:10.980+0000 7fd7b8072700 1 --2- 172.21.15.12:0/2381994342 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7b01081a0 0x7fd7b019b460 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:31:11.513 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:10.980+0000 7fd7b8072700 1 -- 172.21.15.12:0/2381994342 >> 172.21.15.12:0/2381994342 conn(0x7fd7b0101bb0 msgr2=0x7fd7b01026e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:31:11.513 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:10.981+0000 7fd7b8072700 1 -- 172.21.15.12:0/2381994342 shutdown_connections 2024-02-18T11:31:11.513 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:10.981+0000 7fd7b8072700 1 -- 172.21.15.12:0/2381994342 wait complete. 2024-02-18T11:31:11.513 INFO:teuthology.orchestra.run.smithi012.stdout:mgr not available, waiting (3/15)... 2024-02-18T11:31:12.425 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:11 smithi012 ceph-mon[32460]: from='mgr.14100 172.21.15.12:0/90894279' entity='mgr.x' 2024-02-18T11:31:12.425 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:11 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/2381994342' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-18T11:31:12.425 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:11 smithi012 ceph-mon[32460]: mgrmap e3: x(active, since 1.00759s) 2024-02-18T11:31:14.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:13 smithi012 ceph-mon[32460]: mgrmap e4: x(active, since 2s) 2024-02-18T11:31:14.763 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-02-18T11:31:14.763 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout { 2024-02-18T11:31:14.763 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "fsid": "0c8bb0de-ce51-11ee-95ba-87774f69a715", 2024-02-18T11:31:14.763 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "health": { 2024-02-18T11:31:14.763 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-18T11:31:14.763 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-18T11:31:14.763 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-18T11:31:14.763 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-02-18T11:31:14.764 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-18T11:31:14.764 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-18T11:31:14.764 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 0 2024-02-18T11:31:14.764 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-02-18T11:31:14.764 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-18T11:31:14.764 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "a" 2024-02-18T11:31:14.764 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-02-18T11:31:14.764 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-02-18T11:31:14.764 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-18T11:31:14.764 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-18T11:31:14.764 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-18T11:31:14.764 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-18T11:31:14.764 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-02-18T11:31:14.764 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-18T11:31:14.765 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-18T11:31:14.765 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-18T11:31:14.765 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-18T11:31:14.765 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-18T11:31:14.765 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-18T11:31:14.765 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-18T11:31:14.765 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-18T11:31:14.765 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-02-18T11:31:14.765 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-18T11:31:14.765 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-18T11:31:14.765 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-18T11:31:14.765 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-18T11:31:14.765 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-18T11:31:14.765 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-18T11:31:14.765 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-18T11:31:14.766 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-18T11:31:14.766 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-18T11:31:14.766 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-02-18T11:31:14.766 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-18T11:31:14.766 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-18T11:31:14.766 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-18T11:31:14.766 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-18T11:31:14.766 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-02-18T11:31:14.766 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-18T11:31:14.766 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-18T11:31:14.766 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-18T11:31:14.766 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-18T11:31:14.766 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-18T11:31:14.766 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-18T11:31:14.766 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "restful" 2024-02-18T11:31:14.767 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-02-18T11:31:14.767 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-18T11:31:14.767 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-02-18T11:31:14.767 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-18T11:31:14.767 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-18T11:31:14.767 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "modified": "2024-02-18T11:30:49.110551+0000", 2024-02-18T11:31:14.767 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-18T11:31:14.767 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-02-18T11:31:14.767 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-18T11:31:14.767 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout } 2024-02-18T11:31:14.767 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:13.888+0000 7fcb38579700 1 Processor -- start 2024-02-18T11:31:14.767 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:13.888+0000 7fcb38579700 1 -- start start 2024-02-18T11:31:14.767 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:13.888+0000 7fcb38579700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb301076f0 0x7fcb30107b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:31:14.767 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:13.888+0000 7fcb38579700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb301080e0 con 0x7fcb301076f0 2024-02-18T11:31:14.767 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:13.888+0000 7fcb36315700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb301076f0 0x7fcb30107b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:31:14.767 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:13.888+0000 7fcb36315700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb301076f0 0x7fcb30107b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60244/0 (socket says 172.21.15.12:60244) 2024-02-18T11:31:14.768 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:13.888+0000 7fcb36315700 1 -- 172.21.15.12:0/855182213 learned_addr learned my addr 172.21.15.12:0/855182213 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:31:14.768 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:13.889+0000 7fcb36315700 1 -- 172.21.15.12:0/855182213 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb301088a0 con 0x7fcb301076f0 2024-02-18T11:31:14.768 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:13.889+0000 7fcb36315700 1 --2- 172.21.15.12:0/855182213 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb301076f0 0x7fcb30107b10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c0098a0 tx=0x7fcb2c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c824ce98b05f1f5e server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:31:14.768 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:13.889+0000 7fcb35313700 1 -- 172.21.15.12:0/855182213 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcb2c004160 con 0x7fcb301076f0 2024-02-18T11:31:14.768 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:13.889+0000 7fcb35313700 1 -- 172.21.15.12:0/855182213 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcb2c0042c0 con 0x7fcb301076f0 2024-02-18T11:31:14.768 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:13.889+0000 7fcb35313700 1 -- 172.21.15.12:0/855182213 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcb2c011ac0 con 0x7fcb301076f0 2024-02-18T11:31:14.768 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:13.890+0000 7fcb38579700 1 -- 172.21.15.12:0/855182213 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb301076f0 msgr2=0x7fcb30107b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:31:14.768 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:13.890+0000 7fcb38579700 1 --2- 172.21.15.12:0/855182213 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb301076f0 0x7fcb30107b10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c0098a0 tx=0x7fcb2c009bb0 comp rx=0 tx=0).stop 2024-02-18T11:31:14.768 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:13.890+0000 7fcb38579700 1 -- 172.21.15.12:0/855182213 shutdown_connections 2024-02-18T11:31:14.768 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:13.890+0000 7fcb38579700 1 --2- 172.21.15.12:0/855182213 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb301076f0 0x7fcb30107b10 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:31:14.768 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:13.890+0000 7fcb38579700 1 -- 172.21.15.12:0/855182213 >> 172.21.15.12:0/855182213 conn(0x7fcb30102d90 msgr2=0x7fcb301051b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:31:14.768 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:13.890+0000 7fcb38579700 1 -- 172.21.15.12:0/855182213 shutdown_connections 2024-02-18T11:31:14.769 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:13.890+0000 7fcb38579700 1 -- 172.21.15.12:0/855182213 wait complete. 2024-02-18T11:31:14.769 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:13.890+0000 7fcb38579700 1 Processor -- start 2024-02-18T11:31:14.769 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:13.890+0000 7fcb38579700 1 -- start start 2024-02-18T11:31:14.769 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:13.891+0000 7fcb38579700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb301076f0 0x7fcb3019b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:31:14.769 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:13.891+0000 7fcb38579700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3019b9e0 con 0x7fcb301076f0 2024-02-18T11:31:14.769 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:13.891+0000 7fcb36315700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb301076f0 0x7fcb3019b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:31:14.769 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:13.891+0000 7fcb36315700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb301076f0 0x7fcb3019b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60254/0 (socket says 172.21.15.12:60254) 2024-02-18T11:31:14.769 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:13.891+0000 7fcb36315700 1 -- 172.21.15.12:0/4101587888 learned_addr learned my addr 172.21.15.12:0/4101587888 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:31:14.769 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:13.891+0000 7fcb36315700 1 -- 172.21.15.12:0/4101587888 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb2c009550 con 0x7fcb301076f0 2024-02-18T11:31:14.769 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:13.891+0000 7fcb36315700 1 --2- 172.21.15.12:0/4101587888 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb301076f0 0x7fcb3019b4a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c000c00 tx=0x7fcb2c011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:31:14.769 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:13.891+0000 7fcb277fe700 1 -- 172.21.15.12:0/4101587888 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcb2c003aa0 con 0x7fcb301076f0 2024-02-18T11:31:14.769 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:13.891+0000 7fcb277fe700 1 -- 172.21.15.12:0/4101587888 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcb2c003c00 con 0x7fcb301076f0 2024-02-18T11:31:14.769 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:13.891+0000 7fcb38579700 1 -- 172.21.15.12:0/4101587888 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb3019bbe0 con 0x7fcb301076f0 2024-02-18T11:31:14.769 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:13.891+0000 7fcb277fe700 1 -- 172.21.15.12:0/4101587888 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcb2c0248a0 con 0x7fcb301076f0 2024-02-18T11:31:14.770 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:13.891+0000 7fcb38579700 1 -- 172.21.15.12:0/4101587888 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb3019c080 con 0x7fcb301076f0 2024-02-18T11:31:14.770 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:13.892+0000 7fcb38579700 1 -- 172.21.15.12:0/4101587888 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb3010bad0 con 0x7fcb301076f0 2024-02-18T11:31:14.770 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:13.892+0000 7fcb277fe700 1 -- 172.21.15.12:0/4101587888 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 4) v1 ==== 46080+0+0 (secure 0 0 0) 0x7fcb2c01ac60 con 0x7fcb301076f0 2024-02-18T11:31:14.770 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:13.892+0000 7fcb277fe700 1 --2- 172.21.15.12:0/4101587888 >> [v2:172.21.15.12:6800/3925204723,v1:172.21.15.12:6801/3925204723] conn(0x7fcb1c0390c0 0x7fcb1c03b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:31:14.770 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:13.893+0000 7fcb277fe700 1 -- 172.21.15.12:0/4101587888 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fcb2c04cfc0 con 0x7fcb301076f0 2024-02-18T11:31:14.770 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:13.893+0000 7fcb35b14700 1 --2- 172.21.15.12:0/4101587888 >> [v2:172.21.15.12:6800/3925204723,v1:172.21.15.12:6801/3925204723] conn(0x7fcb1c0390c0 0x7fcb1c03b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:31:14.770 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:13.893+0000 7fcb35b14700 1 --2- 172.21.15.12:0/4101587888 >> [v2:172.21.15.12:6800/3925204723,v1:172.21.15.12:6801/3925204723] conn(0x7fcb1c0390c0 0x7fcb1c03b580 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fcb20006fd0 tx=0x7fcb20006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:31:14.770 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:13.898+0000 7fcb277fe700 1 -- 172.21.15.12:0/4101587888 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcb2c01a430 con 0x7fcb301076f0 2024-02-18T11:31:14.770 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:14.182+0000 7fcb38579700 1 -- 172.21.15.12:0/4101587888 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fcb3019ee50 con 0x7fcb301076f0 2024-02-18T11:31:14.770 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:14.182+0000 7fcb277fe700 1 -- 172.21.15.12:0/4101587888 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7fcb2c0314f0 con 0x7fcb301076f0 2024-02-18T11:31:14.770 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:14.184+0000 7fcb38579700 1 -- 172.21.15.12:0/4101587888 >> [v2:172.21.15.12:6800/3925204723,v1:172.21.15.12:6801/3925204723] conn(0x7fcb1c0390c0 msgr2=0x7fcb1c03b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:31:14.770 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:14.184+0000 7fcb38579700 1 --2- 172.21.15.12:0/4101587888 >> [v2:172.21.15.12:6800/3925204723,v1:172.21.15.12:6801/3925204723] conn(0x7fcb1c0390c0 0x7fcb1c03b580 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fcb20006fd0 tx=0x7fcb20006e40 comp rx=0 tx=0).stop 2024-02-18T11:31:14.770 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:14.184+0000 7fcb38579700 1 -- 172.21.15.12:0/4101587888 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb301076f0 msgr2=0x7fcb3019b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:31:14.770 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:14.184+0000 7fcb38579700 1 --2- 172.21.15.12:0/4101587888 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb301076f0 0x7fcb3019b4a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c000c00 tx=0x7fcb2c011b40 comp rx=0 tx=0).stop 2024-02-18T11:31:14.771 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:14.184+0000 7fcb38579700 1 -- 172.21.15.12:0/4101587888 shutdown_connections 2024-02-18T11:31:14.771 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:14.185+0000 7fcb38579700 1 --2- 172.21.15.12:0/4101587888 >> [v2:172.21.15.12:6800/3925204723,v1:172.21.15.12:6801/3925204723] conn(0x7fcb1c0390c0 0x7fcb1c03b580 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:31:14.771 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:14.185+0000 7fcb38579700 1 --2- 172.21.15.12:0/4101587888 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb301076f0 0x7fcb3019b4a0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:31:14.771 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:14.185+0000 7fcb38579700 1 -- 172.21.15.12:0/4101587888 >> 172.21.15.12:0/4101587888 conn(0x7fcb30102d90 msgr2=0x7fcb30103890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:31:14.771 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:14.185+0000 7fcb38579700 1 -- 172.21.15.12:0/4101587888 shutdown_connections 2024-02-18T11:31:14.771 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:14.185+0000 7fcb38579700 1 -- 172.21.15.12:0/4101587888 wait complete. 2024-02-18T11:31:14.771 INFO:teuthology.orchestra.run.smithi012.stdout:mgr is available 2024-02-18T11:31:15.133 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:14 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/4101587888' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-18T11:31:15.921 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-02-18T11:31:15.921 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout [global] 2024-02-18T11:31:15.922 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout fsid = 0c8bb0de-ce51-11ee-95ba-87774f69a715 2024-02-18T11:31:15.922 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-02-18T11:31:15.922 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-02-18T11:31:15.922 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-02-18T11:31:15.922 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-02-18T11:31:15.922 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-02-18T11:31:15.922 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout [mgr] 2024-02-18T11:31:15.922 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-02-18T11:31:15.922 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-02-18T11:31:15.922 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout [osd] 2024-02-18T11:31:15.922 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-02-18T11:31:15.922 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-02-18T11:31:15.922 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-02-18T11:31:15.922 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-02-18T11:31:15.923 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-02-18T11:31:15.923 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:15.166+0000 7f3f5a756700 1 Processor -- start 2024-02-18T11:31:15.923 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:15.166+0000 7f3f5a756700 1 -- start start 2024-02-18T11:31:15.923 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:15.166+0000 7f3f5a756700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f541059e0 0x7f3f54105e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:31:15.923 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:15.166+0000 7f3f5a756700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f541063d0 con 0x7f3f541059e0 2024-02-18T11:31:15.923 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:15.166+0000 7f3f53fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f541059e0 0x7f3f54105e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:31:15.923 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:15.166+0000 7f3f53fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f541059e0 0x7f3f54105e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60260/0 (socket says 172.21.15.12:60260) 2024-02-18T11:31:15.923 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:15.166+0000 7f3f53fff700 1 -- 172.21.15.12:0/908537288 learned_addr learned my addr 172.21.15.12:0/908537288 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:31:15.923 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:15.166+0000 7f3f53fff700 1 -- 172.21.15.12:0/908537288 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f54108970 con 0x7f3f541059e0 2024-02-18T11:31:15.923 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:15.166+0000 7f3f53fff700 1 --2- 172.21.15.12:0/908537288 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f541059e0 0x7f3f54105e00 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3f440098a0 tx=0x7f3f44009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d32abc4a68b7f621 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:31:15.923 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:15.167+0000 7f3f52ffd700 1 -- 172.21.15.12:0/908537288 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3f44004160 con 0x7f3f541059e0 2024-02-18T11:31:15.923 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:15.167+0000 7f3f52ffd700 1 -- 172.21.15.12:0/908537288 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3f440042c0 con 0x7f3f541059e0 2024-02-18T11:31:15.923 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:15.167+0000 7f3f52ffd700 1 -- 172.21.15.12:0/908537288 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3f44011ac0 con 0x7f3f541059e0 2024-02-18T11:31:15.923 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:15.167+0000 7f3f5a756700 1 -- 172.21.15.12:0/908537288 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f541059e0 msgr2=0x7f3f54105e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:31:15.923 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:15.167+0000 7f3f5a756700 1 --2- 172.21.15.12:0/908537288 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f541059e0 0x7f3f54105e00 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3f440098a0 tx=0x7f3f44009bb0 comp rx=0 tx=0).stop 2024-02-18T11:31:15.924 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:15.167+0000 7f3f5a756700 1 -- 172.21.15.12:0/908537288 shutdown_connections 2024-02-18T11:31:15.924 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:15.167+0000 7f3f5a756700 1 --2- 172.21.15.12:0/908537288 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f541059e0 0x7f3f54105e00 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:31:15.924 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:15.167+0000 7f3f5a756700 1 -- 172.21.15.12:0/908537288 >> 172.21.15.12:0/908537288 conn(0x7f3f541013b0 msgr2=0x7f3f541037d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:31:15.924 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:15.167+0000 7f3f5a756700 1 -- 172.21.15.12:0/908537288 shutdown_connections 2024-02-18T11:31:15.924 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:15.167+0000 7f3f5a756700 1 -- 172.21.15.12:0/908537288 wait complete. 2024-02-18T11:31:15.924 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:15.168+0000 7f3f5a756700 1 Processor -- start 2024-02-18T11:31:15.924 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:15.168+0000 7f3f5a756700 1 -- start start 2024-02-18T11:31:15.924 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:15.168+0000 7f3f5a756700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f541059e0 0x7f3f541081f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:31:15.925 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:15.168+0000 7f3f5a756700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f54108730 con 0x7f3f541059e0 2024-02-18T11:31:15.925 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:15.168+0000 7f3f53fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f541059e0 0x7f3f541081f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:31:15.925 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:15.168+0000 7f3f53fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f541059e0 0x7f3f541081f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60272/0 (socket says 172.21.15.12:60272) 2024-02-18T11:31:15.925 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:15.168+0000 7f3f53fff700 1 -- 172.21.15.12:0/1437500809 learned_addr learned my addr 172.21.15.12:0/1437500809 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:31:15.925 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:15.169+0000 7f3f53fff700 1 -- 172.21.15.12:0/1437500809 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f44009550 con 0x7f3f541059e0 2024-02-18T11:31:15.925 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:15.169+0000 7f3f53fff700 1 --2- 172.21.15.12:0/1437500809 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f541059e0 0x7f3f541081f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3f44011780 tx=0x7f3f44011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:31:15.925 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:15.169+0000 7f3f517fa700 1 -- 172.21.15.12:0/1437500809 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3f44003af0 con 0x7f3f541059e0 2024-02-18T11:31:15.925 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:15.169+0000 7f3f517fa700 1 -- 172.21.15.12:0/1437500809 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3f44003c50 con 0x7f3f541059e0 2024-02-18T11:31:15.925 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:15.169+0000 7f3f5a756700 1 -- 172.21.15.12:0/1437500809 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f54106920 con 0x7f3f541059e0 2024-02-18T11:31:15.925 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:15.169+0000 7f3f517fa700 1 -- 172.21.15.12:0/1437500809 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3f440197e0 con 0x7f3f541059e0 2024-02-18T11:31:15.925 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:15.169+0000 7f3f5a756700 1 -- 172.21.15.12:0/1437500809 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f54106dc0 con 0x7f3f541059e0 2024-02-18T11:31:15.925 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:15.170+0000 7f3f517fa700 1 -- 172.21.15.12:0/1437500809 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 4) v1 ==== 46080+0+0 (secure 0 0 0) 0x7f3f4401a430 con 0x7f3f541059e0 2024-02-18T11:31:15.926 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:15.170+0000 7f3f517fa700 1 --2- 172.21.15.12:0/1437500809 >> [v2:172.21.15.12:6800/3925204723,v1:172.21.15.12:6801/3925204723] conn(0x7f3f3c0390c0 0x7f3f3c03b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:31:15.926 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:15.170+0000 7f3f517fa700 1 -- 172.21.15.12:0/1437500809 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f3f4404d510 con 0x7f3f541059e0 2024-02-18T11:31:15.926 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:15.170+0000 7f3f5a756700 1 -- 172.21.15.12:0/1437500809 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f5410bba0 con 0x7f3f541059e0 2024-02-18T11:31:15.926 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:15.170+0000 7f3f537fe700 1 --2- 172.21.15.12:0/1437500809 >> [v2:172.21.15.12:6800/3925204723,v1:172.21.15.12:6801/3925204723] conn(0x7f3f3c0390c0 0x7f3f3c03b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:31:15.926 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:15.170+0000 7f3f537fe700 1 --2- 172.21.15.12:0/1437500809 >> [v2:172.21.15.12:6800/3925204723,v1:172.21.15.12:6801/3925204723] conn(0x7f3f3c0390c0 0x7f3f3c03b580 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3f48006fd0 tx=0x7f3f48006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:31:15.926 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:15.175+0000 7f3f517fa700 1 -- 172.21.15.12:0/1437500809 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3f4402dba0 con 0x7f3f541059e0 2024-02-18T11:31:15.926 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:15.376+0000 7f3f5a756700 1 -- 172.21.15.12:0/1437500809 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f3f5410bd60 con 0x7f3f541059e0 2024-02-18T11:31:15.926 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:15.376+0000 7f3f517fa700 1 -- 172.21.15.12:0/1437500809 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f3f4402dba0 con 0x7f3f541059e0 2024-02-18T11:31:15.926 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:15.379+0000 7f3f5a756700 1 -- 172.21.15.12:0/1437500809 >> [v2:172.21.15.12:6800/3925204723,v1:172.21.15.12:6801/3925204723] conn(0x7f3f3c0390c0 msgr2=0x7f3f3c03b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:31:15.926 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:15.379+0000 7f3f5a756700 1 --2- 172.21.15.12:0/1437500809 >> [v2:172.21.15.12:6800/3925204723,v1:172.21.15.12:6801/3925204723] conn(0x7f3f3c0390c0 0x7f3f3c03b580 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3f48006fd0 tx=0x7f3f48006e40 comp rx=0 tx=0).stop 2024-02-18T11:31:15.926 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:15.379+0000 7f3f5a756700 1 -- 172.21.15.12:0/1437500809 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f541059e0 msgr2=0x7f3f541081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:31:15.926 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:15.379+0000 7f3f5a756700 1 --2- 172.21.15.12:0/1437500809 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f541059e0 0x7f3f541081f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3f44011780 tx=0x7f3f44011b40 comp rx=0 tx=0).stop 2024-02-18T11:31:15.926 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:15.379+0000 7f3f5a756700 1 -- 172.21.15.12:0/1437500809 shutdown_connections 2024-02-18T11:31:15.926 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:15.379+0000 7f3f5a756700 1 --2- 172.21.15.12:0/1437500809 >> [v2:172.21.15.12:6800/3925204723,v1:172.21.15.12:6801/3925204723] conn(0x7f3f3c0390c0 0x7f3f3c03b580 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:31:15.927 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:15.379+0000 7f3f5a756700 1 --2- 172.21.15.12:0/1437500809 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f541059e0 0x7f3f541081f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:31:15.927 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:15.379+0000 7f3f5a756700 1 -- 172.21.15.12:0/1437500809 >> 172.21.15.12:0/1437500809 conn(0x7f3f541013b0 msgr2=0x7f3f54101f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:31:15.927 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:15.379+0000 7f3f5a756700 1 -- 172.21.15.12:0/1437500809 shutdown_connections 2024-02-18T11:31:15.927 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:15.379+0000 7f3f5a756700 1 -- 172.21.15.12:0/1437500809 wait complete. 2024-02-18T11:31:15.927 INFO:teuthology.orchestra.run.smithi012.stdout:Enabling cephadm module... 2024-02-18T11:31:16.215 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:15 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/1437500809' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-02-18T11:31:17.285 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:16 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/1611636725' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-02-18T11:31:17.285 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:17 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: ignoring --setuser ceph since I am not root 2024-02-18T11:31:17.285 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:17 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: ignoring --setgroup ceph since I am not root 2024-02-18T11:31:17.528 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:16.322+0000 7f1f8b181700 1 Processor -- start 2024-02-18T11:31:17.528 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:16.322+0000 7f1f8b181700 1 -- start start 2024-02-18T11:31:17.528 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:16.322+0000 7f1f8b181700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f84108ca0 0x7f1f841090c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:31:17.528 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:16.322+0000 7f1f8b181700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f84109690 con 0x7f1f84108ca0 2024-02-18T11:31:17.528 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:16.323+0000 7f1f88f1d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f84108ca0 0x7f1f841090c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:31:17.528 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:16.323+0000 7f1f88f1d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f84108ca0 0x7f1f841090c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60282/0 (socket says 172.21.15.12:60282) 2024-02-18T11:31:17.528 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:16.323+0000 7f1f88f1d700 1 -- 172.21.15.12:0/3766747674 learned_addr learned my addr 172.21.15.12:0/3766747674 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:31:17.528 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:16.323+0000 7f1f88f1d700 1 -- 172.21.15.12:0/3766747674 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f841097d0 con 0x7f1f84108ca0 2024-02-18T11:31:17.528 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:16.323+0000 7f1f88f1d700 1 --2- 172.21.15.12:0/3766747674 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f84108ca0 0x7f1f841090c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1f780098a0 tx=0x7f1f78009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a296e52636ec9e5 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:31:17.529 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:16.323+0000 7f1f837fe700 1 -- 172.21.15.12:0/3766747674 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1f78004160 con 0x7f1f84108ca0 2024-02-18T11:31:17.529 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:16.323+0000 7f1f837fe700 1 -- 172.21.15.12:0/3766747674 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1f780042c0 con 0x7f1f84108ca0 2024-02-18T11:31:17.529 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:16.323+0000 7f1f837fe700 1 -- 172.21.15.12:0/3766747674 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1f78011ac0 con 0x7f1f84108ca0 2024-02-18T11:31:17.529 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:16.324+0000 7f1f8b181700 1 -- 172.21.15.12:0/3766747674 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f84108ca0 msgr2=0x7f1f841090c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:31:17.529 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:16.324+0000 7f1f8b181700 1 --2- 172.21.15.12:0/3766747674 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f84108ca0 0x7f1f841090c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1f780098a0 tx=0x7f1f78009bb0 comp rx=0 tx=0).stop 2024-02-18T11:31:17.529 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:16.324+0000 7f1f8b181700 1 -- 172.21.15.12:0/3766747674 shutdown_connections 2024-02-18T11:31:17.529 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:16.324+0000 7f1f8b181700 1 --2- 172.21.15.12:0/3766747674 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f84108ca0 0x7f1f841090c0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:31:17.529 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:16.324+0000 7f1f8b181700 1 -- 172.21.15.12:0/3766747674 >> 172.21.15.12:0/3766747674 conn(0x7f1f84101380 msgr2=0x7f1f841037a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:31:17.529 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:16.324+0000 7f1f8b181700 1 -- 172.21.15.12:0/3766747674 shutdown_connections 2024-02-18T11:31:17.529 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:16.324+0000 7f1f8b181700 1 -- 172.21.15.12:0/3766747674 wait complete. 2024-02-18T11:31:17.529 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:16.325+0000 7f1f8b181700 1 Processor -- start 2024-02-18T11:31:17.529 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:16.325+0000 7f1f8b181700 1 -- start start 2024-02-18T11:31:17.529 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:16.325+0000 7f1f8b181700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f84108ca0 0x7f1f8419b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:31:17.529 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:16.325+0000 7f1f8b181700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8419ba10 con 0x7f1f84108ca0 2024-02-18T11:31:17.529 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:16.325+0000 7f1f88f1d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f84108ca0 0x7f1f8419b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:31:17.530 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:16.325+0000 7f1f88f1d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f84108ca0 0x7f1f8419b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60286/0 (socket says 172.21.15.12:60286) 2024-02-18T11:31:17.530 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:16.325+0000 7f1f88f1d700 1 -- 172.21.15.12:0/1611636725 learned_addr learned my addr 172.21.15.12:0/1611636725 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:31:17.530 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:16.325+0000 7f1f88f1d700 1 -- 172.21.15.12:0/1611636725 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f78009550 con 0x7f1f84108ca0 2024-02-18T11:31:17.530 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:16.325+0000 7f1f88f1d700 1 --2- 172.21.15.12:0/1611636725 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f84108ca0 0x7f1f8419b4d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1f78009fb0 tx=0x7f1f78011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:31:17.530 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:16.325+0000 7f1f81ffb700 1 -- 172.21.15.12:0/1611636725 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1f780038b0 con 0x7f1f84108ca0 2024-02-18T11:31:17.530 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:16.325+0000 7f1f81ffb700 1 -- 172.21.15.12:0/1611636725 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1f78003a10 con 0x7f1f84108ca0 2024-02-18T11:31:17.530 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:16.325+0000 7f1f8b181700 1 -- 172.21.15.12:0/1611636725 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f8419bc10 con 0x7f1f84108ca0 2024-02-18T11:31:17.530 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:16.326+0000 7f1f81ffb700 1 -- 172.21.15.12:0/1611636725 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1f780195e0 con 0x7f1f84108ca0 2024-02-18T11:31:17.530 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:16.326+0000 7f1f8b181700 1 -- 172.21.15.12:0/1611636725 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f8419c0b0 con 0x7f1f84108ca0 2024-02-18T11:31:17.530 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:16.326+0000 7f1f81ffb700 1 -- 172.21.15.12:0/1611636725 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 4) v1 ==== 46080+0+0 (secure 0 0 0) 0x7f1f78019850 con 0x7f1f84108ca0 2024-02-18T11:31:17.530 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:16.327+0000 7f1f81ffb700 1 --2- 172.21.15.12:0/1611636725 >> [v2:172.21.15.12:6800/3925204723,v1:172.21.15.12:6801/3925204723] conn(0x7f1f740390c0 0x7f1f7403b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:31:17.530 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:16.327+0000 7f1f81ffb700 1 -- 172.21.15.12:0/1611636725 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f1f7804d280 con 0x7f1f84108ca0 2024-02-18T11:31:17.530 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:16.327+0000 7f1f83fff700 1 --2- 172.21.15.12:0/1611636725 >> [v2:172.21.15.12:6800/3925204723,v1:172.21.15.12:6801/3925204723] conn(0x7f1f740390c0 0x7f1f7403b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:31:17.530 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:16.327+0000 7f1f8b181700 1 -- 172.21.15.12:0/1611636725 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f8410bc50 con 0x7f1f84108ca0 2024-02-18T11:31:17.531 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:16.327+0000 7f1f83fff700 1 --2- 172.21.15.12:0/1611636725 >> [v2:172.21.15.12:6800/3925204723,v1:172.21.15.12:6801/3925204723] conn(0x7f1f740390c0 0x7f1f7403b580 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1f70006fd0 tx=0x7f1f70006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:31:17.531 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:16.332+0000 7f1f81ffb700 1 -- 172.21.15.12:0/1611636725 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1f78019b00 con 0x7f1f84108ca0 2024-02-18T11:31:17.531 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:16.598+0000 7f1f8b181700 1 -- 172.21.15.12:0/1611636725 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f1f8419eba0 con 0x7f1f84108ca0 2024-02-18T11:31:17.531 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:16.965+0000 7f1f81ffb700 1 -- 172.21.15.12:0/1611636725 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f1f78023390 con 0x7f1f84108ca0 2024-02-18T11:31:17.531 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:16.967+0000 7f1f81ffb700 1 -- 172.21.15.12:0/1611636725 <== mon.0 v2:172.21.15.12:3300/0 8 ==== mgrmap(e 5) v1 ==== 46091+0+0 (secure 0 0 0) 0x7f1f78019ce0 con 0x7f1f84108ca0 2024-02-18T11:31:17.531 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:16.968+0000 7f1f8b181700 1 -- 172.21.15.12:0/1611636725 >> [v2:172.21.15.12:6800/3925204723,v1:172.21.15.12:6801/3925204723] conn(0x7f1f740390c0 msgr2=0x7f1f7403b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:31:17.531 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:16.968+0000 7f1f8b181700 1 --2- 172.21.15.12:0/1611636725 >> [v2:172.21.15.12:6800/3925204723,v1:172.21.15.12:6801/3925204723] conn(0x7f1f740390c0 0x7f1f7403b580 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1f70006fd0 tx=0x7f1f70006e40 comp rx=0 tx=0).stop 2024-02-18T11:31:17.531 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:16.968+0000 7f1f8b181700 1 -- 172.21.15.12:0/1611636725 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f84108ca0 msgr2=0x7f1f8419b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:31:17.531 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:16.968+0000 7f1f8b181700 1 --2- 172.21.15.12:0/1611636725 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f84108ca0 0x7f1f8419b4d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1f78009fb0 tx=0x7f1f78011b20 comp rx=0 tx=0).stop 2024-02-18T11:31:17.531 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:16.968+0000 7f1f8b181700 1 -- 172.21.15.12:0/1611636725 shutdown_connections 2024-02-18T11:31:17.531 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:16.968+0000 7f1f8b181700 1 --2- 172.21.15.12:0/1611636725 >> [v2:172.21.15.12:6800/3925204723,v1:172.21.15.12:6801/3925204723] conn(0x7f1f740390c0 0x7f1f7403b580 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:31:17.531 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:16.968+0000 7f1f8b181700 1 --2- 172.21.15.12:0/1611636725 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f84108ca0 0x7f1f8419b4d0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:31:17.531 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:16.968+0000 7f1f8b181700 1 -- 172.21.15.12:0/1611636725 >> 172.21.15.12:0/1611636725 conn(0x7f1f84101380 msgr2=0x7f1f84101f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:31:17.531 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:16.969+0000 7f1f8b181700 1 -- 172.21.15.12:0/1611636725 shutdown_connections 2024-02-18T11:31:17.531 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:16.969+0000 7f1f8b181700 1 -- 172.21.15.12:0/1611636725 wait complete. 2024-02-18T11:31:17.823 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:17 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:31:17.569+0000 7fb3c146c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-18T11:31:17.823 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:17 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:31:17.666+0000 7fb3c146c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-18T11:31:18.128 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:17 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/1611636725' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-02-18T11:31:18.129 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:17 smithi012 ceph-mon[32460]: mgrmap e5: x(active, since 6s) 2024-02-18T11:31:18.424 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:18 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:31:18.339+0000 7fb3c146c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-18T11:31:18.760 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:18 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:31:18.438+0000 7fb3c146c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-18T11:31:18.760 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:18 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:31:18.581+0000 7fb3c146c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-18T11:31:18.760 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:18 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:31:18.680+0000 7fb3c146c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-18T11:31:18.761 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:18 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:31:18.764+0000 7fb3c146c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-18T11:31:18.776 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout { 2024-02-18T11:31:18.776 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-02-18T11:31:18.776 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-18T11:31:18.776 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-02-18T11:31:18.776 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-02-18T11:31:18.776 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout } 2024-02-18T11:31:18.776 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:17.927+0000 7f0a88530700 1 Processor -- start 2024-02-18T11:31:18.776 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:17.927+0000 7f0a88530700 1 -- start start 2024-02-18T11:31:18.776 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:17.927+0000 7f0a88530700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a80108140 0x7f0a8010a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:31:18.776 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:17.927+0000 7f0a88530700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a80075770 con 0x7f0a80108140 2024-02-18T11:31:18.777 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:17.927+0000 7f0a862cc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a80108140 0x7f0a8010a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:31:18.777 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:17.927+0000 7f0a862cc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a80108140 0x7f0a8010a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60322/0 (socket says 172.21.15.12:60322) 2024-02-18T11:31:18.777 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:17.927+0000 7f0a862cc700 1 -- 172.21.15.12:0/504955104 learned_addr learned my addr 172.21.15.12:0/504955104 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:31:18.777 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:17.928+0000 7f0a862cc700 1 -- 172.21.15.12:0/504955104 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a8010aa70 con 0x7f0a80108140 2024-02-18T11:31:18.777 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:17.928+0000 7f0a862cc700 1 --2- 172.21.15.12:0/504955104 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a80108140 0x7f0a8010a530 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c0098a0 tx=0x7f0a7c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4a9c3ebb8577eca server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:31:18.777 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:17.928+0000 7f0a852ca700 1 -- 172.21.15.12:0/504955104 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0a7c004160 con 0x7f0a80108140 2024-02-18T11:31:18.777 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:17.928+0000 7f0a852ca700 1 -- 172.21.15.12:0/504955104 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0a7c0042c0 con 0x7f0a80108140 2024-02-18T11:31:18.777 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:17.928+0000 7f0a852ca700 1 -- 172.21.15.12:0/504955104 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0a7c011ac0 con 0x7f0a80108140 2024-02-18T11:31:18.777 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:17.929+0000 7f0a88530700 1 -- 172.21.15.12:0/504955104 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a80108140 msgr2=0x7f0a8010a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:31:18.777 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:17.929+0000 7f0a88530700 1 --2- 172.21.15.12:0/504955104 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a80108140 0x7f0a8010a530 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c0098a0 tx=0x7f0a7c009bb0 comp rx=0 tx=0).stop 2024-02-18T11:31:18.777 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:17.929+0000 7f0a88530700 1 -- 172.21.15.12:0/504955104 shutdown_connections 2024-02-18T11:31:18.777 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:17.929+0000 7f0a88530700 1 --2- 172.21.15.12:0/504955104 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a80108140 0x7f0a8010a530 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:31:18.777 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:17.929+0000 7f0a88530700 1 -- 172.21.15.12:0/504955104 >> 172.21.15.12:0/504955104 conn(0x7f0a80101c10 msgr2=0x7f0a80104030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:31:18.777 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:17.929+0000 7f0a88530700 1 -- 172.21.15.12:0/504955104 shutdown_connections 2024-02-18T11:31:18.778 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:17.929+0000 7f0a88530700 1 -- 172.21.15.12:0/504955104 wait complete. 2024-02-18T11:31:18.778 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:17.929+0000 7f0a88530700 1 Processor -- start 2024-02-18T11:31:18.778 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:17.929+0000 7f0a88530700 1 -- start start 2024-02-18T11:31:18.778 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:17.930+0000 7f0a88530700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a80108140 0x7f0a801079c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:31:18.778 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:17.930+0000 7f0a88530700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a80107f00 con 0x7f0a80108140 2024-02-18T11:31:18.778 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:17.930+0000 7f0a862cc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a80108140 0x7f0a801079c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:31:18.778 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:17.930+0000 7f0a862cc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a80108140 0x7f0a801079c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60334/0 (socket says 172.21.15.12:60334) 2024-02-18T11:31:18.778 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:17.930+0000 7f0a862cc700 1 -- 172.21.15.12:0/540089170 learned_addr learned my addr 172.21.15.12:0/540089170 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:31:18.778 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:17.930+0000 7f0a862cc700 1 -- 172.21.15.12:0/540089170 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a7c009550 con 0x7f0a80108140 2024-02-18T11:31:18.778 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:17.930+0000 7f0a862cc700 1 --2- 172.21.15.12:0/540089170 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a80108140 0x7f0a801079c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c011a70 tx=0x7f0a7c011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:31:18.778 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:17.930+0000 7f0a777fe700 1 -- 172.21.15.12:0/540089170 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0a7c0039c0 con 0x7f0a80108140 2024-02-18T11:31:18.778 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:17.930+0000 7f0a777fe700 1 -- 172.21.15.12:0/540089170 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0a7c003b20 con 0x7f0a80108140 2024-02-18T11:31:18.778 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:17.930+0000 7f0a777fe700 1 -- 172.21.15.12:0/540089170 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0a7c024610 con 0x7f0a80108140 2024-02-18T11:31:18.778 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:17.930+0000 7f0a88530700 1 -- 172.21.15.12:0/540089170 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a801060f0 con 0x7f0a80108140 2024-02-18T11:31:18.778 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:17.930+0000 7f0a88530700 1 -- 172.21.15.12:0/540089170 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a80106590 con 0x7f0a80108140 2024-02-18T11:31:18.778 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:17.931+0000 7f0a88530700 1 -- 172.21.15.12:0/540089170 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a8010b8d0 con 0x7f0a80108140 2024-02-18T11:31:18.779 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:17.932+0000 7f0a777fe700 1 -- 172.21.15.12:0/540089170 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 46091+0+0 (secure 0 0 0) 0x7f0a7c01aa10 con 0x7f0a80108140 2024-02-18T11:31:18.779 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:17.932+0000 7f0a777fe700 1 --2- 172.21.15.12:0/540089170 >> [v2:172.21.15.12:6800/3925204723,v1:172.21.15.12:6801/3925204723] conn(0x7f0a6c039160 0x7f0a6c03b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:31:18.779 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:17.932+0000 7f0a777fe700 1 -- 172.21.15.12:0/540089170 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f0a7c04d140 con 0x7f0a80108140 2024-02-18T11:31:18.779 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:17.932+0000 7f0a85acb700 1 -- 172.21.15.12:0/540089170 >> [v2:172.21.15.12:6800/3925204723,v1:172.21.15.12:6801/3925204723] conn(0x7f0a6c039160 msgr2=0x7f0a6c03b620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/3925204723 2024-02-18T11:31:18.779 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:17.932+0000 7f0a85acb700 1 --2- 172.21.15.12:0/540089170 >> [v2:172.21.15.12:6800/3925204723,v1:172.21.15.12:6801/3925204723] conn(0x7f0a6c039160 0x7f0a6c03b620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-18T11:31:18.779 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:17.937+0000 7f0a777fe700 1 -- 172.21.15.12:0/540089170 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0a7c01a430 con 0x7f0a80108140 2024-02-18T11:31:18.779 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:18.133+0000 7f0a85acb700 1 -- 172.21.15.12:0/540089170 >> [v2:172.21.15.12:6800/3925204723,v1:172.21.15.12:6801/3925204723] conn(0x7f0a6c039160 msgr2=0x7f0a6c03b620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/3925204723 2024-02-18T11:31:18.779 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:18.133+0000 7f0a85acb700 1 --2- 172.21.15.12:0/540089170 >> [v2:172.21.15.12:6800/3925204723,v1:172.21.15.12:6801/3925204723] conn(0x7f0a6c039160 0x7f0a6c03b620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-18T11:31:18.779 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:18.202+0000 7f0a88530700 1 -- 172.21.15.12:0/540089170 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f0a80106c80 con 0x7f0a80108140 2024-02-18T11:31:18.779 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:18.202+0000 7f0a777fe700 1 -- 172.21.15.12:0/540089170 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7f0a7c018be0 con 0x7f0a80108140 2024-02-18T11:31:18.779 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:18.205+0000 7f0a88530700 1 -- 172.21.15.12:0/540089170 >> [v2:172.21.15.12:6800/3925204723,v1:172.21.15.12:6801/3925204723] conn(0x7f0a6c039160 msgr2=0x7f0a6c03b620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-18T11:31:18.779 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:18.205+0000 7f0a88530700 1 --2- 172.21.15.12:0/540089170 >> [v2:172.21.15.12:6800/3925204723,v1:172.21.15.12:6801/3925204723] conn(0x7f0a6c039160 0x7f0a6c03b620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:31:18.779 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:18.205+0000 7f0a88530700 1 -- 172.21.15.12:0/540089170 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a80108140 msgr2=0x7f0a801079c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:31:18.779 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:18.205+0000 7f0a88530700 1 --2- 172.21.15.12:0/540089170 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a80108140 0x7f0a801079c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c011a70 tx=0x7f0a7c011b50 comp rx=0 tx=0).stop 2024-02-18T11:31:18.779 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:18.205+0000 7f0a88530700 1 -- 172.21.15.12:0/540089170 shutdown_connections 2024-02-18T11:31:18.780 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:18.205+0000 7f0a88530700 1 --2- 172.21.15.12:0/540089170 >> [v2:172.21.15.12:6800/3925204723,v1:172.21.15.12:6801/3925204723] conn(0x7f0a6c039160 0x7f0a6c03b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:31:18.780 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:18.205+0000 7f0a88530700 1 --2- 172.21.15.12:0/540089170 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a80108140 0x7f0a801079c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:31:18.780 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:18.205+0000 7f0a88530700 1 -- 172.21.15.12:0/540089170 >> 172.21.15.12:0/540089170 conn(0x7f0a80101c10 msgr2=0x7f0a801027f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:31:18.780 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:18.205+0000 7f0a88530700 1 -- 172.21.15.12:0/540089170 shutdown_connections 2024-02-18T11:31:18.780 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:18.205+0000 7f0a88530700 1 -- 172.21.15.12:0/540089170 wait complete. 2024-02-18T11:31:18.780 INFO:teuthology.orchestra.run.smithi012.stdout:Waiting for the mgr to restart... 2024-02-18T11:31:18.780 INFO:teuthology.orchestra.run.smithi012.stdout:Waiting for mgr epoch 5... 2024-02-18T11:31:19.038 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:18 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/540089170' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-18T11:31:19.341 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:19 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:31:19.042+0000 7fb3c146c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-18T11:31:19.341 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:19 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:31:19.158+0000 7fb3c146c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-18T11:31:19.341 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:19 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:31:19.248+0000 7fb3c146c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-18T11:31:19.674 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:19 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:31:19.345+0000 7fb3c146c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-18T11:31:19.674 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:19 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:31:19.523+0000 7fb3c146c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-18T11:31:20.924 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:20 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:31:20.510+0000 7fb3c146c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-18T11:31:20.924 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:20 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:31:20.643+0000 7fb3c146c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-18T11:31:21.273 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:20 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:31:20.986+0000 7fb3c146c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-18T11:31:21.274 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:21 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:31:21.078+0000 7fb3c146c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-18T11:31:21.274 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:21 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:31:21.278+0000 7fb3c146c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-18T11:31:21.674 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:21 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:31:21.468+0000 7fb3c146c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-18T11:31:22.016 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:21 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:31:21.759+0000 7fb3c146c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-18T11:31:22.016 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:22 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:31:22.020+0000 7fb3c146c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-18T11:31:22.424 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:22 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:31:22.262+0000 7fb3c146c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-18T11:31:23.424 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:23 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:31:23.085+0000 7fb3c146c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-18T11:31:23.924 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:23 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:31:23.667+0000 7fb3c146c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-18T11:31:23.924 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:23 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:31:23.768+0000 7fb3c146c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-18T11:31:24.424 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:23 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:31:23.987+0000 7fb3c146c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-18T11:31:24.425 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:24 smithi012 ceph-mon[32460]: Active manager daemon x restarted 2024-02-18T11:31:24.425 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:24 smithi012 ceph-mon[32460]: Activating manager daemon x 2024-02-18T11:31:24.425 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:24 smithi012 ceph-mon[32460]: osdmap e2: 0 total, 0 up, 0 in 2024-02-18T11:31:24.425 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:24 smithi012 ceph-mon[32460]: mgrmap e6: x(active, starting, since 0.00346449s) 2024-02-18T11:31:24.425 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:24 smithi012 ceph-mon[32460]: from='mgr.14120 172.21.15.12:0/2151529525' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-18T11:31:24.425 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:24 smithi012 ceph-mon[32460]: from='mgr.14120 172.21.15.12:0/2151529525' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-18T11:31:24.425 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:24 smithi012 ceph-mon[32460]: from='mgr.14120 172.21.15.12:0/2151529525' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-18T11:31:24.425 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:24 smithi012 ceph-mon[32460]: from='mgr.14120 172.21.15.12:0/2151529525' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-18T11:31:24.425 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:24 smithi012 ceph-mon[32460]: from='mgr.14120 172.21.15.12:0/2151529525' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-18T11:31:24.425 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:24 smithi012 ceph-mon[32460]: Manager daemon x is now available 2024-02-18T11:31:24.425 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:24 smithi012 ceph-mon[32460]: from='mgr.14120 172.21.15.12:0/2151529525' entity='mgr.x' 2024-02-18T11:31:24.425 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:24 smithi012 ceph-mon[32460]: from='mgr.14120 172.21.15.12:0/2151529525' entity='mgr.x' 2024-02-18T11:31:24.425 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:24 smithi012 ceph-mon[32460]: from='mgr.14120 172.21.15.12:0/2151529525' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:31:24.425 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:24 smithi012 ceph-mon[32460]: from='mgr.14120 172.21.15.12:0/2151529525' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:31:25.298 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:25 smithi012 ceph-mon[32460]: Found migration_current of "None". Setting to last migration. 2024-02-18T11:31:25.298 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:25 smithi012 ceph-mon[32460]: from='mgr.14120 172.21.15.12:0/2151529525' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:31:25.298 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:25 smithi012 ceph-mon[32460]: from='mgr.14120 172.21.15.12:0/2151529525' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-02-18T11:31:25.298 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:25 smithi012 ceph-mon[32460]: from='mgr.14120 172.21.15.12:0/2151529525' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-02-18T11:31:25.299 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:25 smithi012 ceph-mon[32460]: from='mgr.14120 172.21.15.12:0/2151529525' entity='mgr.x' 2024-02-18T11:31:25.299 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:25 smithi012 ceph-mon[32460]: from='mgr.14120 172.21.15.12:0/2151529525' entity='mgr.x' 2024-02-18T11:31:25.299 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:25 smithi012 ceph-mon[32460]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-02-18T11:31:25.299 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:25 smithi012 ceph-mon[32460]: mgrmap e7: x(active, since 1.00566s) 2024-02-18T11:31:25.299 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:25 smithi012 ceph-mon[32460]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-02-18T11:31:25.552 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout { 2024-02-18T11:31:25.552 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-02-18T11:31:25.552 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "initialized": true 2024-02-18T11:31:25.552 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout } 2024-02-18T11:31:25.552 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:19.184+0000 7fc5ca988700 1 Processor -- start 2024-02-18T11:31:25.552 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:19.184+0000 7fc5ca988700 1 -- start start 2024-02-18T11:31:25.552 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:19.185+0000 7fc5ca988700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc5c4108180 0x7fc5c410a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:31:25.553 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:19.185+0000 7fc5ca988700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5c407ae80 con 0x7fc5c4108180 2024-02-18T11:31:25.553 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:19.185+0000 7fc5c3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc5c4108180 0x7fc5c410a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:31:25.553 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:19.185+0000 7fc5c3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc5c4108180 0x7fc5c410a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60350/0 (socket says 172.21.15.12:60350) 2024-02-18T11:31:25.553 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:19.185+0000 7fc5c3fff700 1 -- 172.21.15.12:0/907716276 learned_addr learned my addr 172.21.15.12:0/907716276 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:31:25.553 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:19.185+0000 7fc5c3fff700 1 -- 172.21.15.12:0/907716276 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5c410aab0 con 0x7fc5c4108180 2024-02-18T11:31:25.553 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:19.185+0000 7fc5c3fff700 1 --2- 172.21.15.12:0/907716276 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc5c4108180 0x7fc5c410a570 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc5b40098a0 tx=0x7fc5b4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe0056e79895adbe server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:31:25.553 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:19.185+0000 7fc5c2ffd700 1 -- 172.21.15.12:0/907716276 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc5b4004160 con 0x7fc5c4108180 2024-02-18T11:31:25.553 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:19.186+0000 7fc5c2ffd700 1 -- 172.21.15.12:0/907716276 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc5b40042c0 con 0x7fc5c4108180 2024-02-18T11:31:25.553 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:19.186+0000 7fc5c2ffd700 1 -- 172.21.15.12:0/907716276 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc5b4011ac0 con 0x7fc5c4108180 2024-02-18T11:31:25.553 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:19.186+0000 7fc5ca988700 1 -- 172.21.15.12:0/907716276 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc5c4108180 msgr2=0x7fc5c410a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:31:25.553 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:19.186+0000 7fc5ca988700 1 --2- 172.21.15.12:0/907716276 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc5c4108180 0x7fc5c410a570 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc5b40098a0 tx=0x7fc5b4009bb0 comp rx=0 tx=0).stop 2024-02-18T11:31:25.553 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:19.186+0000 7fc5ca988700 1 -- 172.21.15.12:0/907716276 shutdown_connections 2024-02-18T11:31:25.553 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:19.186+0000 7fc5ca988700 1 --2- 172.21.15.12:0/907716276 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc5c4108180 0x7fc5c410a570 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:31:25.554 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:19.186+0000 7fc5ca988700 1 -- 172.21.15.12:0/907716276 >> 172.21.15.12:0/907716276 conn(0x7fc5c4101bf0 msgr2=0x7fc5c4104010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:31:25.554 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:19.186+0000 7fc5ca988700 1 -- 172.21.15.12:0/907716276 shutdown_connections 2024-02-18T11:31:25.554 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:19.186+0000 7fc5ca988700 1 -- 172.21.15.12:0/907716276 wait complete. 2024-02-18T11:31:25.554 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:19.187+0000 7fc5ca988700 1 Processor -- start 2024-02-18T11:31:25.554 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:19.187+0000 7fc5ca988700 1 -- start start 2024-02-18T11:31:25.554 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:19.187+0000 7fc5ca988700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc5c4108180 0x7fc5c419b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:31:25.554 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:19.187+0000 7fc5ca988700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5c419b9d0 con 0x7fc5c4108180 2024-02-18T11:31:25.554 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:19.187+0000 7fc5c3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc5c4108180 0x7fc5c419b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:31:25.554 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:19.187+0000 7fc5c3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc5c4108180 0x7fc5c419b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60356/0 (socket says 172.21.15.12:60356) 2024-02-18T11:31:25.554 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:19.187+0000 7fc5c3fff700 1 -- 172.21.15.12:0/3505987150 learned_addr learned my addr 172.21.15.12:0/3505987150 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:31:25.554 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:19.188+0000 7fc5c3fff700 1 -- 172.21.15.12:0/3505987150 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5b4009550 con 0x7fc5c4108180 2024-02-18T11:31:25.554 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:19.188+0000 7fc5c3fff700 1 --2- 172.21.15.12:0/3505987150 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc5c4108180 0x7fc5c419b490 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc5b4009130 tx=0x7fc5b4011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:31:25.554 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:19.188+0000 7fc5c17fa700 1 -- 172.21.15.12:0/3505987150 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc5b4003aa0 con 0x7fc5c4108180 2024-02-18T11:31:25.555 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:19.188+0000 7fc5c17fa700 1 -- 172.21.15.12:0/3505987150 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc5b4003c00 con 0x7fc5c4108180 2024-02-18T11:31:25.555 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:19.188+0000 7fc5ca988700 1 -- 172.21.15.12:0/3505987150 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5c419bbd0 con 0x7fc5c4108180 2024-02-18T11:31:25.555 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:19.188+0000 7fc5c17fa700 1 -- 172.21.15.12:0/3505987150 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc5b4024800 con 0x7fc5c4108180 2024-02-18T11:31:25.555 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:19.188+0000 7fc5ca988700 1 -- 172.21.15.12:0/3505987150 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5c419c070 con 0x7fc5c4108180 2024-02-18T11:31:25.555 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:19.189+0000 7fc5c17fa700 1 -- 172.21.15.12:0/3505987150 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 46091+0+0 (secure 0 0 0) 0x7fc5b4024ac0 con 0x7fc5c4108180 2024-02-18T11:31:25.555 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:19.189+0000 7fc5c17fa700 1 --2- 172.21.15.12:0/3505987150 >> [v2:172.21.15.12:6800/3925204723,v1:172.21.15.12:6801/3925204723] conn(0x7fc5ac039110 0x7fc5ac03b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:31:25.555 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:19.189+0000 7fc5c17fa700 1 -- 172.21.15.12:0/3505987150 --> [v2:172.21.15.12:6800/3925204723,v1:172.21.15.12:6801/3925204723] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc5ac03bce0 con 0x7fc5ac039110 2024-02-18T11:31:25.555 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:19.189+0000 7fc5c17fa700 1 -- 172.21.15.12:0/3505987150 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc5b404d200 con 0x7fc5c4108180 2024-02-18T11:31:25.555 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:19.189+0000 7fc5c37fe700 1 -- 172.21.15.12:0/3505987150 >> [v2:172.21.15.12:6800/3925204723,v1:172.21.15.12:6801/3925204723] conn(0x7fc5ac039110 msgr2=0x7fc5ac03b5d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/3925204723 2024-02-18T11:31:25.555 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:19.189+0000 7fc5c37fe700 1 --2- 172.21.15.12:0/3505987150 >> [v2:172.21.15.12:6800/3925204723,v1:172.21.15.12:6801/3925204723] conn(0x7fc5ac039110 0x7fc5ac03b5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-18T11:31:25.555 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:19.390+0000 7fc5c37fe700 1 -- 172.21.15.12:0/3505987150 >> [v2:172.21.15.12:6800/3925204723,v1:172.21.15.12:6801/3925204723] conn(0x7fc5ac039110 msgr2=0x7fc5ac03b5d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/3925204723 2024-02-18T11:31:25.555 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:19.390+0000 7fc5c37fe700 1 --2- 172.21.15.12:0/3505987150 >> [v2:172.21.15.12:6800/3925204723,v1:172.21.15.12:6801/3925204723] conn(0x7fc5ac039110 0x7fc5ac03b5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-18T11:31:25.555 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:19.791+0000 7fc5c37fe700 1 -- 172.21.15.12:0/3505987150 >> [v2:172.21.15.12:6800/3925204723,v1:172.21.15.12:6801/3925204723] conn(0x7fc5ac039110 msgr2=0x7fc5ac03b5d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/3925204723 2024-02-18T11:31:25.556 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:19.791+0000 7fc5c37fe700 1 --2- 172.21.15.12:0/3505987150 >> [v2:172.21.15.12:6800/3925204723,v1:172.21.15.12:6801/3925204723] conn(0x7fc5ac039110 0x7fc5ac03b5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-02-18T11:31:25.556 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:20.592+0000 7fc5c37fe700 1 -- 172.21.15.12:0/3505987150 >> [v2:172.21.15.12:6800/3925204723,v1:172.21.15.12:6801/3925204723] conn(0x7fc5ac039110 msgr2=0x7fc5ac03b5d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/3925204723 2024-02-18T11:31:25.556 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:20.592+0000 7fc5c37fe700 1 --2- 172.21.15.12:0/3505987150 >> [v2:172.21.15.12:6800/3925204723,v1:172.21.15.12:6801/3925204723] conn(0x7fc5ac039110 0x7fc5ac03b5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-02-18T11:31:25.556 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:22.193+0000 7fc5c37fe700 1 -- 172.21.15.12:0/3505987150 >> [v2:172.21.15.12:6800/3925204723,v1:172.21.15.12:6801/3925204723] conn(0x7fc5ac039110 msgr2=0x7fc5ac03b5d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/3925204723 2024-02-18T11:31:25.556 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:22.193+0000 7fc5c37fe700 1 --2- 172.21.15.12:0/3505987150 >> [v2:172.21.15.12:6800/3925204723,v1:172.21.15.12:6801/3925204723] conn(0x7fc5ac039110 0x7fc5ac03b5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-02-18T11:31:25.556 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:24.002+0000 7fc5c17fa700 1 -- 172.21.15.12:0/3505987150 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mgrmap(e 6) v1 ==== 45858+0+0 (secure 0 0 0) 0x7fc5b402d3f0 con 0x7fc5c4108180 2024-02-18T11:31:25.556 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:24.002+0000 7fc5c17fa700 1 -- 172.21.15.12:0/3505987150 >> [v2:172.21.15.12:6800/3925204723,v1:172.21.15.12:6801/3925204723] conn(0x7fc5ac039110 msgr2=0x7fc5ac03b5d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-18T11:31:25.556 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:24.002+0000 7fc5c17fa700 1 --2- 172.21.15.12:0/3505987150 >> [v2:172.21.15.12:6800/3925204723,v1:172.21.15.12:6801/3925204723] conn(0x7fc5ac039110 0x7fc5ac03b5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:31:25.556 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:25.003+0000 7fc5c17fa700 1 -- 172.21.15.12:0/3505987150 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mgrmap(e 7) v1 ==== 45985+0+0 (secure 0 0 0) 0x7fc5b404eca0 con 0x7fc5c4108180 2024-02-18T11:31:25.556 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:25.003+0000 7fc5c17fa700 1 --2- 172.21.15.12:0/3505987150 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7fc5ac03ca00 0x7fc5ac03edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:31:25.556 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:25.003+0000 7fc5c17fa700 1 -- 172.21.15.12:0/3505987150 --> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc5ac03bce0 con 0x7fc5ac03ca00 2024-02-18T11:31:25.557 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:25.004+0000 7fc5c37fe700 1 --2- 172.21.15.12:0/3505987150 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7fc5ac03ca00 0x7fc5ac03edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:31:25.557 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:25.004+0000 7fc5c37fe700 1 --2- 172.21.15.12:0/3505987150 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7fc5ac03ca00 0x7fc5ac03edf0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc5b8003a10 tx=0x7fc5b8009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:31:25.557 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:25.005+0000 7fc5c17fa700 1 -- 172.21.15.12:0/3505987150 <== mgr.14120 v2:172.21.15.12:6800/4135274411 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7fc5ac03bce0 con 0x7fc5ac03ca00 2024-02-18T11:31:25.557 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:25.011+0000 7fc5ca988700 1 -- 172.21.15.12:0/3505987150 --> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fc5c410b8b0 con 0x7fc5ac03ca00 2024-02-18T11:31:25.557 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:25.012+0000 7fc5c17fa700 1 -- 172.21.15.12:0/3505987150 <== mgr.14120 v2:172.21.15.12:6800/4135274411 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fc5c410b8b0 con 0x7fc5ac03ca00 2024-02-18T11:31:25.557 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:25.012+0000 7fc5ca988700 1 -- 172.21.15.12:0/3505987150 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7fc5ac03ca00 msgr2=0x7fc5ac03edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:31:25.557 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:25.012+0000 7fc5ca988700 1 --2- 172.21.15.12:0/3505987150 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7fc5ac03ca00 0x7fc5ac03edf0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc5b8003a10 tx=0x7fc5b8009290 comp rx=0 tx=0).stop 2024-02-18T11:31:25.557 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:25.012+0000 7fc5ca988700 1 -- 172.21.15.12:0/3505987150 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc5c4108180 msgr2=0x7fc5c419b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:31:25.557 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:25.012+0000 7fc5ca988700 1 --2- 172.21.15.12:0/3505987150 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc5c4108180 0x7fc5c419b490 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc5b4009130 tx=0x7fc5b4011b40 comp rx=0 tx=0).stop 2024-02-18T11:31:25.557 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:25.012+0000 7fc5ca988700 1 -- 172.21.15.12:0/3505987150 shutdown_connections 2024-02-18T11:31:25.557 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:25.012+0000 7fc5ca988700 1 --2- 172.21.15.12:0/3505987150 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc5c4108180 0x7fc5c419b490 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:31:25.557 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:25.012+0000 7fc5ca988700 1 --2- 172.21.15.12:0/3505987150 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7fc5ac03ca00 0x7fc5ac03edf0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:31:25.557 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:25.012+0000 7fc5ca988700 1 --2- 172.21.15.12:0/3505987150 >> [v2:172.21.15.12:6800/3925204723,v1:172.21.15.12:6801/3925204723] conn(0x7fc5ac039110 0x7fc5ac03b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:31:25.558 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:25.012+0000 7fc5ca988700 1 -- 172.21.15.12:0/3505987150 >> 172.21.15.12:0/3505987150 conn(0x7fc5c4101bf0 msgr2=0x7fc5c41026f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:31:25.558 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:25.012+0000 7fc5ca988700 1 -- 172.21.15.12:0/3505987150 shutdown_connections 2024-02-18T11:31:25.558 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:25.012+0000 7fc5ca988700 1 -- 172.21.15.12:0/3505987150 wait complete. 2024-02-18T11:31:25.558 INFO:teuthology.orchestra.run.smithi012.stdout:mgr epoch 5 is available 2024-02-18T11:31:25.558 INFO:teuthology.orchestra.run.smithi012.stdout:Setting orchestrator backend to cephadm... 2024-02-18T11:31:26.189 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:26 smithi012 ceph-mon[32460]: [18/Feb/2024:11:31:25] ENGINE Bus STARTING 2024-02-18T11:31:26.189 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:26 smithi012 ceph-mon[32460]: [18/Feb/2024:11:31:25] ENGINE Serving on http://172.21.15.12:8765 2024-02-18T11:31:26.189 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:26 smithi012 ceph-mon[32460]: [18/Feb/2024:11:31:25] ENGINE Serving on https://172.21.15.12:7150 2024-02-18T11:31:26.189 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:26 smithi012 ceph-mon[32460]: [18/Feb/2024:11:31:25] ENGINE Bus STARTED 2024-02-18T11:31:26.189 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:26 smithi012 ceph-mon[32460]: from='mgr.14120 172.21.15.12:0/2151529525' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:31:26.689 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:25.941+0000 7f90224d5700 1 Processor -- start 2024-02-18T11:31:26.689 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:25.941+0000 7f90224d5700 1 -- start start 2024-02-18T11:31:26.689 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:25.941+0000 7f90224d5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f901c1076d0 0x7f901c107af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:31:26.689 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:25.941+0000 7f90224d5700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f901c1080c0 con 0x7f901c1076d0 2024-02-18T11:31:26.689 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:25.941+0000 7f901bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f901c1076d0 0x7f901c107af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:31:26.689 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:25.941+0000 7f901bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f901c1076d0 0x7f901c107af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51480/0 (socket says 172.21.15.12:51480) 2024-02-18T11:31:26.689 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:25.941+0000 7f901bfff700 1 -- 172.21.15.12:0/3178284670 learned_addr learned my addr 172.21.15.12:0/3178284670 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:31:26.689 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:25.941+0000 7f901bfff700 1 -- 172.21.15.12:0/3178284670 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f901c108880 con 0x7f901c1076d0 2024-02-18T11:31:26.689 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:25.941+0000 7f901bfff700 1 --2- 172.21.15.12:0/3178284670 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f901c1076d0 0x7f901c107af0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f900c0098a0 tx=0x7f900c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b21d2cc26a6f8b1 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:31:26.689 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:25.941+0000 7f901affd700 1 -- 172.21.15.12:0/3178284670 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f900c004160 con 0x7f901c1076d0 2024-02-18T11:31:26.689 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:25.942+0000 7f901affd700 1 -- 172.21.15.12:0/3178284670 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f900c0042c0 con 0x7f901c1076d0 2024-02-18T11:31:26.689 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:25.942+0000 7f901affd700 1 -- 172.21.15.12:0/3178284670 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f900c011ac0 con 0x7f901c1076d0 2024-02-18T11:31:26.690 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:25.942+0000 7f90224d5700 1 -- 172.21.15.12:0/3178284670 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f901c1076d0 msgr2=0x7f901c107af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:31:26.690 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:25.942+0000 7f90224d5700 1 --2- 172.21.15.12:0/3178284670 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f901c1076d0 0x7f901c107af0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f900c0098a0 tx=0x7f900c009bb0 comp rx=0 tx=0).stop 2024-02-18T11:31:26.690 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:25.942+0000 7f90224d5700 1 -- 172.21.15.12:0/3178284670 shutdown_connections 2024-02-18T11:31:26.690 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:25.942+0000 7f90224d5700 1 --2- 172.21.15.12:0/3178284670 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f901c1076d0 0x7f901c107af0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:31:26.690 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:25.942+0000 7f90224d5700 1 -- 172.21.15.12:0/3178284670 >> 172.21.15.12:0/3178284670 conn(0x7f901c102d90 msgr2=0x7f901c1051b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:31:26.690 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:25.942+0000 7f90224d5700 1 -- 172.21.15.12:0/3178284670 shutdown_connections 2024-02-18T11:31:26.690 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:25.942+0000 7f90224d5700 1 -- 172.21.15.12:0/3178284670 wait complete. 2024-02-18T11:31:26.690 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:25.943+0000 7f90224d5700 1 Processor -- start 2024-02-18T11:31:26.690 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:25.943+0000 7f90224d5700 1 -- start start 2024-02-18T11:31:26.690 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:25.943+0000 7f90224d5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f901c1076d0 0x7f901c19d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:31:26.690 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:25.943+0000 7f90224d5700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f901c19dc00 con 0x7f901c1076d0 2024-02-18T11:31:26.690 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:25.943+0000 7f901bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f901c1076d0 0x7f901c19d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:31:26.690 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:25.943+0000 7f901bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f901c1076d0 0x7f901c19d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51484/0 (socket says 172.21.15.12:51484) 2024-02-18T11:31:26.690 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:25.943+0000 7f901bfff700 1 -- 172.21.15.12:0/1763238928 learned_addr learned my addr 172.21.15.12:0/1763238928 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:31:26.690 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:25.943+0000 7f901bfff700 1 -- 172.21.15.12:0/1763238928 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f900c009550 con 0x7f901c1076d0 2024-02-18T11:31:26.691 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:25.943+0000 7f901bfff700 1 --2- 172.21.15.12:0/1763238928 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f901c1076d0 0x7f901c19d6c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f900c003f70 tx=0x7f900c011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:31:26.691 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:25.944+0000 7f90197fa700 1 -- 172.21.15.12:0/1763238928 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f900c0039f0 con 0x7f901c1076d0 2024-02-18T11:31:26.691 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:25.944+0000 7f90197fa700 1 -- 172.21.15.12:0/1763238928 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f900c003b50 con 0x7f901c1076d0 2024-02-18T11:31:26.691 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:25.944+0000 7f90224d5700 1 -- 172.21.15.12:0/1763238928 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f901c19de00 con 0x7f901c1076d0 2024-02-18T11:31:26.691 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:25.944+0000 7f90197fa700 1 -- 172.21.15.12:0/1763238928 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f900c01a690 con 0x7f901c1076d0 2024-02-18T11:31:26.691 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:25.944+0000 7f90224d5700 1 -- 172.21.15.12:0/1763238928 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f901c19e2a0 con 0x7f901c1076d0 2024-02-18T11:31:26.691 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:25.945+0000 7f90197fa700 1 -- 172.21.15.12:0/1763238928 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 7) v1 ==== 45985+0+0 (secure 0 0 0) 0x7f900c024430 con 0x7f901c1076d0 2024-02-18T11:31:26.691 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:25.945+0000 7f90224d5700 1 -- 172.21.15.12:0/1763238928 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f901c197a90 con 0x7f901c1076d0 2024-02-18T11:31:26.691 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:25.945+0000 7f90197fa700 1 --2- 172.21.15.12:0/1763238928 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7f9004039000 0x7f900403b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:31:26.691 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:25.945+0000 7f90197fa700 1 -- 172.21.15.12:0/1763238928 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f900c04d2f0 con 0x7f901c1076d0 2024-02-18T11:31:26.691 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:25.945+0000 7f901b7fe700 1 --2- 172.21.15.12:0/1763238928 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7f9004039000 0x7f900403b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:31:26.691 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:25.945+0000 7f901b7fe700 1 --2- 172.21.15.12:0/1763238928 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7f9004039000 0x7f900403b4c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9010006fd0 tx=0x7f9010006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:31:26.691 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:25.950+0000 7f90197fa700 1 -- 172.21.15.12:0/1763238928 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f900c018a40 con 0x7f901c1076d0 2024-02-18T11:31:26.691 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:26.054+0000 7f90197fa700 1 -- 172.21.15.12:0/1763238928 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mgrmap(e 8) v1 ==== 46091+0+0 (secure 0 0 0) 0x7f900c02d910 con 0x7f901c1076d0 2024-02-18T11:31:26.692 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:26.171+0000 7f90224d5700 1 -- 172.21.15.12:0/1763238928 --> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f901c02da80 con 0x7f9004039000 2024-02-18T11:31:26.692 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:26.178+0000 7f90197fa700 1 -- 172.21.15.12:0/1763238928 <== mgr.14120 v2:172.21.15.12:6800/4135274411 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f901c02da80 con 0x7f9004039000 2024-02-18T11:31:26.692 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:26.180+0000 7f90224d5700 1 -- 172.21.15.12:0/1763238928 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7f9004039000 msgr2=0x7f900403b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:31:26.692 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:26.180+0000 7f90224d5700 1 --2- 172.21.15.12:0/1763238928 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7f9004039000 0x7f900403b4c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9010006fd0 tx=0x7f9010006e40 comp rx=0 tx=0).stop 2024-02-18T11:31:26.692 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:26.180+0000 7f90224d5700 1 -- 172.21.15.12:0/1763238928 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f901c1076d0 msgr2=0x7f901c19d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:31:26.692 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:26.180+0000 7f90224d5700 1 --2- 172.21.15.12:0/1763238928 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f901c1076d0 0x7f901c19d6c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f900c003f70 tx=0x7f900c011b40 comp rx=0 tx=0).stop 2024-02-18T11:31:26.692 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:26.181+0000 7f90224d5700 1 -- 172.21.15.12:0/1763238928 shutdown_connections 2024-02-18T11:31:26.692 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:26.181+0000 7f90224d5700 1 --2- 172.21.15.12:0/1763238928 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7f9004039000 0x7f900403b4c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:31:26.692 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:26.181+0000 7f90224d5700 1 --2- 172.21.15.12:0/1763238928 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f901c1076d0 0x7f901c19d6c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:31:26.692 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:26.181+0000 7f90224d5700 1 -- 172.21.15.12:0/1763238928 >> 172.21.15.12:0/1763238928 conn(0x7f901c102d90 msgr2=0x7f901c194bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:31:26.692 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:26.181+0000 7f90224d5700 1 -- 172.21.15.12:0/1763238928 shutdown_connections 2024-02-18T11:31:26.692 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:26.181+0000 7f90224d5700 1 -- 172.21.15.12:0/1763238928 wait complete. 2024-02-18T11:31:27.334 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:27 smithi012 ceph-mon[32460]: mgrmap e8: x(active, since 2s) 2024-02-18T11:31:27.335 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:27 smithi012 ceph-mon[32460]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:31:27.335 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:27 smithi012 ceph-mon[32460]: from='mgr.14120 172.21.15.12:0/2151529525' entity='mgr.x' 2024-02-18T11:31:27.335 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:27 smithi012 ceph-mon[32460]: from='mgr.14120 172.21.15.12:0/2151529525' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:31:27.845 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout value unchanged 2024-02-18T11:31:27.845 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:27.126+0000 7f94dbcb3700 1 Processor -- start 2024-02-18T11:31:27.846 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:27.126+0000 7f94dbcb3700 1 -- start start 2024-02-18T11:31:27.846 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:27.126+0000 7f94dbcb3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94d41076d0 0x7f94d4107af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:31:27.846 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:27.126+0000 7f94dbcb3700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d41080c0 con 0x7f94d41076d0 2024-02-18T11:31:27.846 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:27.126+0000 7f94d9a4f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94d41076d0 0x7f94d4107af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:31:27.846 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:27.126+0000 7f94d9a4f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94d41076d0 0x7f94d4107af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51492/0 (socket says 172.21.15.12:51492) 2024-02-18T11:31:27.846 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:27.126+0000 7f94d9a4f700 1 -- 172.21.15.12:0/1479411312 learned_addr learned my addr 172.21.15.12:0/1479411312 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:31:27.846 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:27.127+0000 7f94d9a4f700 1 -- 172.21.15.12:0/1479411312 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94d4108880 con 0x7f94d41076d0 2024-02-18T11:31:27.846 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:27.127+0000 7f94d9a4f700 1 --2- 172.21.15.12:0/1479411312 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94d41076d0 0x7f94d4107af0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f94d00098a0 tx=0x7f94d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37bbcb1ab5384346 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:31:27.846 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:27.127+0000 7f94d8a4d700 1 -- 172.21.15.12:0/1479411312 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f94d0004160 con 0x7f94d41076d0 2024-02-18T11:31:27.846 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:27.127+0000 7f94d8a4d700 1 -- 172.21.15.12:0/1479411312 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f94d00042c0 con 0x7f94d41076d0 2024-02-18T11:31:27.846 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:27.127+0000 7f94d8a4d700 1 -- 172.21.15.12:0/1479411312 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f94d0011ac0 con 0x7f94d41076d0 2024-02-18T11:31:27.846 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:27.127+0000 7f94dbcb3700 1 -- 172.21.15.12:0/1479411312 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94d41076d0 msgr2=0x7f94d4107af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:31:27.846 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:27.127+0000 7f94dbcb3700 1 --2- 172.21.15.12:0/1479411312 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94d41076d0 0x7f94d4107af0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f94d00098a0 tx=0x7f94d0009bb0 comp rx=0 tx=0).stop 2024-02-18T11:31:27.846 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:27.127+0000 7f94dbcb3700 1 -- 172.21.15.12:0/1479411312 shutdown_connections 2024-02-18T11:31:27.847 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:27.128+0000 7f94dbcb3700 1 --2- 172.21.15.12:0/1479411312 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94d41076d0 0x7f94d4107af0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:31:27.847 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:27.128+0000 7f94dbcb3700 1 -- 172.21.15.12:0/1479411312 >> 172.21.15.12:0/1479411312 conn(0x7f94d4102d30 msgr2=0x7f94d4105150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:31:27.847 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:27.128+0000 7f94dbcb3700 1 -- 172.21.15.12:0/1479411312 shutdown_connections 2024-02-18T11:31:27.847 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:27.128+0000 7f94dbcb3700 1 -- 172.21.15.12:0/1479411312 wait complete. 2024-02-18T11:31:27.847 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:27.128+0000 7f94dbcb3700 1 Processor -- start 2024-02-18T11:31:27.847 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:27.128+0000 7f94dbcb3700 1 -- start start 2024-02-18T11:31:27.847 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:27.128+0000 7f94dbcb3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94d41076d0 0x7f94d419b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:31:27.847 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:27.128+0000 7f94dbcb3700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d419ba30 con 0x7f94d41076d0 2024-02-18T11:31:27.847 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:27.129+0000 7f94d9a4f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94d41076d0 0x7f94d419b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:31:27.847 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:27.129+0000 7f94d9a4f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94d41076d0 0x7f94d419b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51496/0 (socket says 172.21.15.12:51496) 2024-02-18T11:31:27.847 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:27.129+0000 7f94d9a4f700 1 -- 172.21.15.12:0/1184393460 learned_addr learned my addr 172.21.15.12:0/1184393460 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:31:27.848 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:27.129+0000 7f94d9a4f700 1 -- 172.21.15.12:0/1184393460 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94d0009550 con 0x7f94d41076d0 2024-02-18T11:31:27.848 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:27.129+0000 7f94d9a4f700 1 --2- 172.21.15.12:0/1184393460 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94d41076d0 0x7f94d419b4f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f94d0009fb0 tx=0x7f94d0011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:31:27.848 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:27.129+0000 7f94caffd700 1 -- 172.21.15.12:0/1184393460 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f94d00038b0 con 0x7f94d41076d0 2024-02-18T11:31:27.848 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:27.129+0000 7f94caffd700 1 -- 172.21.15.12:0/1184393460 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f94d0003a10 con 0x7f94d41076d0 2024-02-18T11:31:27.848 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:27.129+0000 7f94dbcb3700 1 -- 172.21.15.12:0/1184393460 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94d419bc30 con 0x7f94d41076d0 2024-02-18T11:31:27.848 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:27.129+0000 7f94caffd700 1 -- 172.21.15.12:0/1184393460 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f94d00195e0 con 0x7f94d41076d0 2024-02-18T11:31:27.848 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:27.129+0000 7f94dbcb3700 1 -- 172.21.15.12:0/1184393460 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94d419c0d0 con 0x7f94d41076d0 2024-02-18T11:31:27.848 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:27.130+0000 7f94caffd700 1 -- 172.21.15.12:0/1184393460 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 8) v1 ==== 46091+0+0 (secure 0 0 0) 0x7f94d0019850 con 0x7f94d41076d0 2024-02-18T11:31:27.848 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:27.130+0000 7f94dbcb3700 1 -- 172.21.15.12:0/1184393460 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94d410bab0 con 0x7f94d41076d0 2024-02-18T11:31:27.848 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:27.130+0000 7f94caffd700 1 --2- 172.21.15.12:0/1184393460 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7f94c0039110 0x7f94c003b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:31:27.848 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:27.131+0000 7f94caffd700 1 -- 172.21.15.12:0/1184393460 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f94d004d510 con 0x7f94d41076d0 2024-02-18T11:31:27.848 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:27.131+0000 7f94d924e700 1 --2- 172.21.15.12:0/1184393460 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7f94c0039110 0x7f94c003b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:31:27.848 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:27.131+0000 7f94d924e700 1 --2- 172.21.15.12:0/1184393460 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7f94c0039110 0x7f94c003b5d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f94c4006fd0 tx=0x7f94c4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:31:27.848 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:27.136+0000 7f94caffd700 1 -- 172.21.15.12:0/1184393460 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f94d0018530 con 0x7f94d41076d0 2024-02-18T11:31:27.849 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:27.339+0000 7f94dbcb3700 1 -- 172.21.15.12:0/1184393460 --> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f94d4103fd0 con 0x7f94c0039110 2024-02-18T11:31:27.849 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:27.339+0000 7f94caffd700 1 -- 172.21.15.12:0/1184393460 <== mgr.14120 v2:172.21.15.12:6800/4135274411 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f94d4103fd0 con 0x7f94c0039110 2024-02-18T11:31:27.849 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:27.341+0000 7f94dbcb3700 1 -- 172.21.15.12:0/1184393460 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7f94c0039110 msgr2=0x7f94c003b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:31:27.849 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:27.341+0000 7f94dbcb3700 1 --2- 172.21.15.12:0/1184393460 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7f94c0039110 0x7f94c003b5d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f94c4006fd0 tx=0x7f94c4006e40 comp rx=0 tx=0).stop 2024-02-18T11:31:27.849 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:27.341+0000 7f94dbcb3700 1 -- 172.21.15.12:0/1184393460 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94d41076d0 msgr2=0x7f94d419b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:31:27.849 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:27.341+0000 7f94dbcb3700 1 --2- 172.21.15.12:0/1184393460 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94d41076d0 0x7f94d419b4f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f94d0009fb0 tx=0x7f94d0011b20 comp rx=0 tx=0).stop 2024-02-18T11:31:27.849 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:27.341+0000 7f94dbcb3700 1 -- 172.21.15.12:0/1184393460 shutdown_connections 2024-02-18T11:31:27.849 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:27.341+0000 7f94dbcb3700 1 --2- 172.21.15.12:0/1184393460 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7f94c0039110 0x7f94c003b5d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:31:27.849 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:27.341+0000 7f94dbcb3700 1 --2- 172.21.15.12:0/1184393460 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94d41076d0 0x7f94d419b4f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:31:27.849 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:27.341+0000 7f94dbcb3700 1 -- 172.21.15.12:0/1184393460 >> 172.21.15.12:0/1184393460 conn(0x7f94d4102d30 msgr2=0x7f94d41038c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:31:27.849 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:27.342+0000 7f94dbcb3700 1 -- 172.21.15.12:0/1184393460 shutdown_connections 2024-02-18T11:31:27.849 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:27.342+0000 7f94dbcb3700 1 -- 172.21.15.12:0/1184393460 wait complete. 2024-02-18T11:31:27.849 INFO:teuthology.orchestra.run.smithi012.stdout:Generating ssh key... 2024-02-18T11:31:28.208 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:28 smithi012 ceph-mon[32460]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:31:29.308 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:29 smithi012 ceph-mon[32460]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:31:29.308 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:29 smithi012 ceph-mon[32460]: Generating ssh key... 2024-02-18T11:31:29.309 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:29 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: Generating public/private rsa key pair. 2024-02-18T11:31:29.309 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:29 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: Your identification has been saved in /tmp/tmp5vp9tz1b/key. 2024-02-18T11:31:29.309 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:29 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: Your public key has been saved in /tmp/tmp5vp9tz1b/key.pub. 2024-02-18T11:31:29.309 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:29 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: The key fingerprint is: 2024-02-18T11:31:29.309 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:29 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: SHA256:RHjptCJ5zzl6S6giJeLoov0XPVqQxvD3XnaorZvcaGs ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715 2024-02-18T11:31:29.309 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:29 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: The key's randomart image is: 2024-02-18T11:31:29.309 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:29 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: +---[RSA 3072]----+ 2024-02-18T11:31:29.309 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:29 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: | ... | 2024-02-18T11:31:29.309 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:29 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: | . ..+ | 2024-02-18T11:31:29.309 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:29 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: | = =.. | 2024-02-18T11:31:29.310 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:29 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: | o O.+ | 2024-02-18T11:31:29.310 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:29 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: | + BSo . | 2024-02-18T11:31:29.310 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:29 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: |o . ..O . + . | 2024-02-18T11:31:29.310 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:29 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: |+o .=.+ = . | 2024-02-18T11:31:29.310 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:29 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: |+o. .+...E+. | 2024-02-18T11:31:29.310 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:29 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: |=o.oo. ..+B+. | 2024-02-18T11:31:29.310 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:29 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: +----[SHA256]-----+ 2024-02-18T11:31:29.828 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:28.258+0000 7f3d08e87700 1 Processor -- start 2024-02-18T11:31:29.828 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:28.258+0000 7f3d08e87700 1 -- start start 2024-02-18T11:31:29.828 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:28.258+0000 7f3d08e87700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d041076a0 0x7f3d04107ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:31:29.828 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:28.258+0000 7f3d08e87700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d04108090 con 0x7f3d041076a0 2024-02-18T11:31:29.828 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:28.258+0000 7f3d0259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d041076a0 0x7f3d04107ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:31:29.828 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:28.258+0000 7f3d0259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d041076a0 0x7f3d04107ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51500/0 (socket says 172.21.15.12:51500) 2024-02-18T11:31:29.828 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:28.258+0000 7f3d0259c700 1 -- 172.21.15.12:0/1568529332 learned_addr learned my addr 172.21.15.12:0/1568529332 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:31:29.828 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:28.258+0000 7f3d0259c700 1 -- 172.21.15.12:0/1568529332 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d04108850 con 0x7f3d041076a0 2024-02-18T11:31:29.828 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:28.259+0000 7f3d0259c700 1 --2- 172.21.15.12:0/1568529332 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d041076a0 0x7f3d04107ac0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3cf80098a0 tx=0x7f3cf8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca5ab6116966f59d server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:31:29.828 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:28.259+0000 7f3d0159a700 1 -- 172.21.15.12:0/1568529332 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3cf8004160 con 0x7f3d041076a0 2024-02-18T11:31:29.829 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:28.259+0000 7f3d0159a700 1 -- 172.21.15.12:0/1568529332 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3cf80042c0 con 0x7f3d041076a0 2024-02-18T11:31:29.829 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:28.259+0000 7f3d0159a700 1 -- 172.21.15.12:0/1568529332 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3cf8011ac0 con 0x7f3d041076a0 2024-02-18T11:31:29.829 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:28.259+0000 7f3d08e87700 1 -- 172.21.15.12:0/1568529332 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d041076a0 msgr2=0x7f3d04107ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:31:29.829 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:28.259+0000 7f3d08e87700 1 --2- 172.21.15.12:0/1568529332 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d041076a0 0x7f3d04107ac0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3cf80098a0 tx=0x7f3cf8009bb0 comp rx=0 tx=0).stop 2024-02-18T11:31:29.829 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:28.259+0000 7f3d08e87700 1 -- 172.21.15.12:0/1568529332 shutdown_connections 2024-02-18T11:31:29.829 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:28.259+0000 7f3d08e87700 1 --2- 172.21.15.12:0/1568529332 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d041076a0 0x7f3d04107ac0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:31:29.829 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:28.259+0000 7f3d08e87700 1 -- 172.21.15.12:0/1568529332 >> 172.21.15.12:0/1568529332 conn(0x7f3d04102d00 msgr2=0x7f3d04105120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:31:29.829 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:28.259+0000 7f3d08e87700 1 -- 172.21.15.12:0/1568529332 shutdown_connections 2024-02-18T11:31:29.829 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:28.260+0000 7f3d08e87700 1 -- 172.21.15.12:0/1568529332 wait complete. 2024-02-18T11:31:29.829 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:28.260+0000 7f3d08e87700 1 Processor -- start 2024-02-18T11:31:29.829 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:28.260+0000 7f3d08e87700 1 -- start start 2024-02-18T11:31:29.829 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:28.260+0000 7f3d08e87700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d041076a0 0x7f3d0407cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:31:29.829 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:28.260+0000 7f3d08e87700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0407d0c0 con 0x7f3d041076a0 2024-02-18T11:31:29.829 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:28.261+0000 7f3d0259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d041076a0 0x7f3d0407cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:31:29.829 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:28.261+0000 7f3d0259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d041076a0 0x7f3d0407cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51506/0 (socket says 172.21.15.12:51506) 2024-02-18T11:31:29.830 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:28.261+0000 7f3d0259c700 1 -- 172.21.15.12:0/4175009737 learned_addr learned my addr 172.21.15.12:0/4175009737 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:31:29.830 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:28.261+0000 7f3d0259c700 1 -- 172.21.15.12:0/4175009737 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cf8009550 con 0x7f3d041076a0 2024-02-18T11:31:29.830 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:28.261+0000 7f3d0259c700 1 --2- 172.21.15.12:0/4175009737 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d041076a0 0x7f3d0407cb80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8003f70 tx=0x7f3cf8011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:31:29.830 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:28.261+0000 7f3cf37fe700 1 -- 172.21.15.12:0/4175009737 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3cf80039f0 con 0x7f3d041076a0 2024-02-18T11:31:29.830 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:28.261+0000 7f3cf37fe700 1 -- 172.21.15.12:0/4175009737 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3cf8003b50 con 0x7f3d041076a0 2024-02-18T11:31:29.830 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:28.261+0000 7f3d08e87700 1 -- 172.21.15.12:0/4175009737 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d0407b2b0 con 0x7f3d041076a0 2024-02-18T11:31:29.830 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:28.261+0000 7f3cf37fe700 1 -- 172.21.15.12:0/4175009737 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3cf801a690 con 0x7f3d041076a0 2024-02-18T11:31:29.830 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:28.261+0000 7f3d08e87700 1 -- 172.21.15.12:0/4175009737 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d0407b750 con 0x7f3d041076a0 2024-02-18T11:31:29.830 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:28.262+0000 7f3cf37fe700 1 -- 172.21.15.12:0/4175009737 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 8) v1 ==== 46091+0+0 (secure 0 0 0) 0x7f3cf801a970 con 0x7f3d041076a0 2024-02-18T11:31:29.830 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:28.262+0000 7f3cf37fe700 1 --2- 172.21.15.12:0/4175009737 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7f3cec0390c0 0x7f3cec03b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:31:29.830 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:28.262+0000 7f3cf37fe700 1 -- 172.21.15.12:0/4175009737 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f3cf804d4f0 con 0x7f3d041076a0 2024-02-18T11:31:29.830 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:28.262+0000 7f3d08e87700 1 -- 172.21.15.12:0/4175009737 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d0410ba80 con 0x7f3d041076a0 2024-02-18T11:31:29.830 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:28.262+0000 7f3d01d9b700 1 --2- 172.21.15.12:0/4175009737 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7f3cec0390c0 0x7f3cec03b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:31:29.830 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:28.263+0000 7f3d01d9b700 1 --2- 172.21.15.12:0/4175009737 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7f3cec0390c0 0x7f3cec03b580 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3cf4006fd0 tx=0x7f3cf4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:31:29.831 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:28.267+0000 7f3cf37fe700 1 -- 172.21.15.12:0/4175009737 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3cf801ae00 con 0x7f3d041076a0 2024-02-18T11:31:29.831 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:28.469+0000 7f3d08e87700 1 -- 172.21.15.12:0/4175009737 --> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f3d0407bd00 con 0x7f3cec0390c0 2024-02-18T11:31:29.831 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:29.300+0000 7f3cf37fe700 1 -- 172.21.15.12:0/4175009737 <== mgr.14120 v2:172.21.15.12:6800/4135274411 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f3d0407bd00 con 0x7f3cec0390c0 2024-02-18T11:31:29.831 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:29.302+0000 7f3d08e87700 1 -- 172.21.15.12:0/4175009737 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7f3cec0390c0 msgr2=0x7f3cec03b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:31:29.831 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:29.302+0000 7f3d08e87700 1 --2- 172.21.15.12:0/4175009737 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7f3cec0390c0 0x7f3cec03b580 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3cf4006fd0 tx=0x7f3cf4006e40 comp rx=0 tx=0).stop 2024-02-18T11:31:29.831 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:29.302+0000 7f3d08e87700 1 -- 172.21.15.12:0/4175009737 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d041076a0 msgr2=0x7f3d0407cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:31:29.831 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:29.302+0000 7f3d08e87700 1 --2- 172.21.15.12:0/4175009737 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d041076a0 0x7f3d0407cb80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8003f70 tx=0x7f3cf8011b40 comp rx=0 tx=0).stop 2024-02-18T11:31:29.831 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:29.302+0000 7f3d08e87700 1 -- 172.21.15.12:0/4175009737 shutdown_connections 2024-02-18T11:31:29.831 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:29.302+0000 7f3d08e87700 1 --2- 172.21.15.12:0/4175009737 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7f3cec0390c0 0x7f3cec03b580 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:31:29.831 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:29.302+0000 7f3d08e87700 1 --2- 172.21.15.12:0/4175009737 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d041076a0 0x7f3d0407cb80 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:31:29.831 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:29.302+0000 7f3d08e87700 1 -- 172.21.15.12:0/4175009737 >> 172.21.15.12:0/4175009737 conn(0x7f3d04102d00 msgr2=0x7f3d04103850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:31:29.831 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:29.302+0000 7f3d08e87700 1 -- 172.21.15.12:0/4175009737 shutdown_connections 2024-02-18T11:31:29.831 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:29.302+0000 7f3d08e87700 1 -- 172.21.15.12:0/4175009737 wait complete. 2024-02-18T11:31:30.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:30 smithi012 ceph-mon[32460]: from='mgr.14120 172.21.15.12:0/2151529525' entity='mgr.x' 2024-02-18T11:31:30.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:30 smithi012 ceph-mon[32460]: from='mgr.14120 172.21.15.12:0/2151529525' entity='mgr.x' 2024-02-18T11:31:30.936 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCsOxk1tzTDdhxzE92jO/raSNrXiPGUOfiYVJqSk5YsD8hBS9Bx0f4NSwPvwXjzQkKXjuyMfFbsnTg/fJ+tB/wANj52SWtSK2/wUEhWe8Vk+uXVvGLAxVGO+mQER9iiQtPJp1zIrI8amzHbgn5rJ4wV5DgPizEIHv+EpCX24Ig7PrRM96G2Ym0tex6URkhEvLX78LEH0tVvwAVWuC69EcATwiIZYSpIQSs5SpLyj89KxUQB8wM5gUV5jSe3/I1f3FuoU5u+i0prXNwkpUFE2owpD3YkZ/2ffUxS55z0WDL1lUKfgkCM8dCKuoTOShPstAbu4i6+kjqEDiOO2/l1IV657k60uLKrHX5BqW2kFPeO7Be3uSoA2pEVME+MJ87XWlW7N3Ut+MwXuOR95MAquLieGs2WqRNZc116ESTEt7Izqq0H4P/73NJmpY9VWP4rXUFxJNNQUQUkkyyFfMIRKtu3Wj1UKOO1gx+LNOV7Z7gMvBBLdmjPbsde+7pdILDIxwk= ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715 2024-02-18T11:31:30.937 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:30.220+0000 7f7a69860700 1 Processor -- start 2024-02-18T11:31:30.937 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:30.220+0000 7f7a69860700 1 -- start start 2024-02-18T11:31:30.937 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:30.220+0000 7f7a69860700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a64107730 0x7f7a64107b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:31:30.937 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:30.220+0000 7f7a69860700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a64108120 con 0x7f7a64107730 2024-02-18T11:31:30.937 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:30.221+0000 7f7a62ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a64107730 0x7f7a64107b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:31:30.937 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:30.221+0000 7f7a62ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a64107730 0x7f7a64107b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51512/0 (socket says 172.21.15.12:51512) 2024-02-18T11:31:30.937 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:30.221+0000 7f7a62ffd700 1 -- 172.21.15.12:0/4190478926 learned_addr learned my addr 172.21.15.12:0/4190478926 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:31:30.937 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:30.221+0000 7f7a62ffd700 1 -- 172.21.15.12:0/4190478926 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a64108930 con 0x7f7a64107730 2024-02-18T11:31:30.937 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:30.221+0000 7f7a62ffd700 1 --2- 172.21.15.12:0/4190478926 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a64107730 0x7f7a64107b50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c009e80 tx=0x7f7a5c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30a7c88b6c201008 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:31:30.937 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:30.221+0000 7f7a61ffb700 1 -- 172.21.15.12:0/4190478926 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7a5c00e070 con 0x7f7a64107730 2024-02-18T11:31:30.937 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:30.221+0000 7f7a61ffb700 1 -- 172.21.15.12:0/4190478926 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7a5c00b8c0 con 0x7f7a64107730 2024-02-18T11:31:30.937 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:30.221+0000 7f7a61ffb700 1 -- 172.21.15.12:0/4190478926 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7a5c004550 con 0x7f7a64107730 2024-02-18T11:31:30.938 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:30.222+0000 7f7a69860700 1 -- 172.21.15.12:0/4190478926 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a64107730 msgr2=0x7f7a64107b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:31:30.938 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:30.222+0000 7f7a69860700 1 --2- 172.21.15.12:0/4190478926 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a64107730 0x7f7a64107b50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c009e80 tx=0x7f7a5c00b330 comp rx=0 tx=0).stop 2024-02-18T11:31:30.938 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:30.222+0000 7f7a69860700 1 -- 172.21.15.12:0/4190478926 shutdown_connections 2024-02-18T11:31:30.938 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:30.222+0000 7f7a69860700 1 --2- 172.21.15.12:0/4190478926 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a64107730 0x7f7a64107b50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:31:30.938 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:30.222+0000 7f7a69860700 1 -- 172.21.15.12:0/4190478926 >> 172.21.15.12:0/4190478926 conn(0x7f7a64102d50 msgr2=0x7f7a64105170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:31:30.938 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:30.222+0000 7f7a69860700 1 -- 172.21.15.12:0/4190478926 shutdown_connections 2024-02-18T11:31:30.938 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:30.222+0000 7f7a69860700 1 -- 172.21.15.12:0/4190478926 wait complete. 2024-02-18T11:31:30.938 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:30.223+0000 7f7a69860700 1 Processor -- start 2024-02-18T11:31:30.938 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:30.223+0000 7f7a69860700 1 -- start start 2024-02-18T11:31:30.938 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:30.223+0000 7f7a69860700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a64107730 0x7f7a6419b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:31:30.938 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:30.223+0000 7f7a69860700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6419b950 con 0x7f7a64107730 2024-02-18T11:31:30.938 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:30.223+0000 7f7a62ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a64107730 0x7f7a6419b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:31:30.938 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:30.223+0000 7f7a62ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a64107730 0x7f7a6419b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51526/0 (socket says 172.21.15.12:51526) 2024-02-18T11:31:30.938 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:30.223+0000 7f7a62ffd700 1 -- 172.21.15.12:0/1594484166 learned_addr learned my addr 172.21.15.12:0/1594484166 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:31:30.938 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:30.223+0000 7f7a62ffd700 1 -- 172.21.15.12:0/1594484166 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a5c009b30 con 0x7f7a64107730 2024-02-18T11:31:30.939 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:30.224+0000 7f7a62ffd700 1 --2- 172.21.15.12:0/1594484166 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a64107730 0x7f7a6419b410 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c009fd0 tx=0x7f7a5c003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:31:30.939 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:30.224+0000 7f7a6885e700 1 -- 172.21.15.12:0/1594484166 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7a5c00e040 con 0x7f7a64107730 2024-02-18T11:31:30.939 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:30.224+0000 7f7a6885e700 1 -- 172.21.15.12:0/1594484166 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7a5c0128f0 con 0x7f7a64107730 2024-02-18T11:31:30.939 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:30.224+0000 7f7a6885e700 1 -- 172.21.15.12:0/1594484166 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7a5c01c770 con 0x7f7a64107730 2024-02-18T11:31:30.939 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:30.224+0000 7f7a69860700 1 -- 172.21.15.12:0/1594484166 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a6419bb50 con 0x7f7a64107730 2024-02-18T11:31:30.939 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:30.224+0000 7f7a69860700 1 -- 172.21.15.12:0/1594484166 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a6419bfd0 con 0x7f7a64107730 2024-02-18T11:31:30.939 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:30.225+0000 7f7a6885e700 1 -- 172.21.15.12:0/1594484166 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 8) v1 ==== 46091+0+0 (secure 0 0 0) 0x7f7a5c01c9e0 con 0x7f7a64107730 2024-02-18T11:31:30.939 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:30.225+0000 7f7a69860700 1 -- 172.21.15.12:0/1594484166 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a64068da0 con 0x7f7a64107730 2024-02-18T11:31:30.939 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:30.225+0000 7f7a6885e700 1 --2- 172.21.15.12:0/1594484166 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7f7a50039160 0x7f7a5003b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:31:30.939 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:30.225+0000 7f7a6885e700 1 -- 172.21.15.12:0/1594484166 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f7a5c04d3b0 con 0x7f7a64107730 2024-02-18T11:31:30.939 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:30.225+0000 7f7a627fc700 1 --2- 172.21.15.12:0/1594484166 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7f7a50039160 0x7f7a5003b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:31:30.939 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:30.225+0000 7f7a627fc700 1 --2- 172.21.15.12:0/1594484166 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7f7a50039160 0x7f7a5003b620 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c006fd0 tx=0x7f7a4c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:31:30.939 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:30.230+0000 7f7a6885e700 1 -- 172.21.15.12:0/1594484166 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7a5c011960 con 0x7f7a64107730 2024-02-18T11:31:30.939 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:30.432+0000 7f7a69860700 1 -- 172.21.15.12:0/1594484166 --> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f7a64103f50 con 0x7f7a50039160 2024-02-18T11:31:30.940 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:30.433+0000 7f7a6885e700 1 -- 172.21.15.12:0/1594484166 <== mgr.14120 v2:172.21.15.12:6800/4135274411 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f7a64103f50 con 0x7f7a50039160 2024-02-18T11:31:30.940 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:30.435+0000 7f7a69860700 1 -- 172.21.15.12:0/1594484166 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7f7a50039160 msgr2=0x7f7a5003b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:31:30.940 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:30.435+0000 7f7a69860700 1 --2- 172.21.15.12:0/1594484166 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7f7a50039160 0x7f7a5003b620 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c006fd0 tx=0x7f7a4c006e40 comp rx=0 tx=0).stop 2024-02-18T11:31:30.940 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:30.435+0000 7f7a69860700 1 -- 172.21.15.12:0/1594484166 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a64107730 msgr2=0x7f7a6419b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:31:30.940 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:30.435+0000 7f7a69860700 1 --2- 172.21.15.12:0/1594484166 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a64107730 0x7f7a6419b410 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c009fd0 tx=0x7f7a5c003d00 comp rx=0 tx=0).stop 2024-02-18T11:31:30.940 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:30.435+0000 7f7a69860700 1 -- 172.21.15.12:0/1594484166 shutdown_connections 2024-02-18T11:31:30.940 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:30.435+0000 7f7a69860700 1 --2- 172.21.15.12:0/1594484166 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7f7a50039160 0x7f7a5003b620 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:31:30.940 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:30.435+0000 7f7a69860700 1 --2- 172.21.15.12:0/1594484166 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a64107730 0x7f7a6419b410 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:31:30.940 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:30.435+0000 7f7a69860700 1 -- 172.21.15.12:0/1594484166 >> 172.21.15.12:0/1594484166 conn(0x7f7a64102d50 msgr2=0x7f7a64103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:31:30.940 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:30.435+0000 7f7a69860700 1 -- 172.21.15.12:0/1594484166 shutdown_connections 2024-02-18T11:31:30.940 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:30.435+0000 7f7a69860700 1 -- 172.21.15.12:0/1594484166 wait complete. 2024-02-18T11:31:30.940 INFO:teuthology.orchestra.run.smithi012.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-02-18T11:31:30.940 INFO:teuthology.orchestra.run.smithi012.stdout:Adding key to root@localhost authorized_keys... 2024-02-18T11:31:30.940 INFO:teuthology.orchestra.run.smithi012.stdout:Adding host smithi012... 2024-02-18T11:31:31.296 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:31 smithi012 ceph-mon[32460]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:31:32.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:32 smithi012 ceph-mon[32460]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi012", "addr": "172.21.15.12", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:31:33.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:33 smithi012 ceph-mon[32460]: Deploying cephadm binary to smithi012 2024-02-18T11:31:36.526 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout Added host 'smithi012' with addr '172.21.15.12' 2024-02-18T11:31:36.526 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:31.330+0000 7f1bb3abc700 1 Processor -- start 2024-02-18T11:31:36.526 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:31.330+0000 7f1bb3abc700 1 -- start start 2024-02-18T11:31:36.526 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:31.330+0000 7f1bb3abc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1bac108120 0x7f1bac10a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:31:36.526 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:31.330+0000 7f1bb3abc700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bac07aec0 con 0x7f1bac108120 2024-02-18T11:31:36.526 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:31.330+0000 7f1bb1858700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1bac108120 0x7f1bac10a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:31:36.526 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:31.330+0000 7f1bb1858700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1bac108120 0x7f1bac10a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51538/0 (socket says 172.21.15.12:51538) 2024-02-18T11:31:36.526 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:31.330+0000 7f1bb1858700 1 -- 172.21.15.12:0/4056097993 learned_addr learned my addr 172.21.15.12:0/4056097993 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:31:36.526 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:31.331+0000 7f1bb1858700 1 -- 172.21.15.12:0/4056097993 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bac10aa50 con 0x7f1bac108120 2024-02-18T11:31:36.526 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:31.331+0000 7f1bb1858700 1 --2- 172.21.15.12:0/4056097993 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1bac108120 0x7f1bac10a510 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1ba80098a0 tx=0x7f1ba8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c39ad78b4711c84f server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:31:36.527 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:31.331+0000 7f1bb0856700 1 -- 172.21.15.12:0/4056097993 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1ba8004160 con 0x7f1bac108120 2024-02-18T11:31:36.527 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:31.331+0000 7f1bb0856700 1 -- 172.21.15.12:0/4056097993 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1ba80042c0 con 0x7f1bac108120 2024-02-18T11:31:36.527 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:31.331+0000 7f1bb0856700 1 -- 172.21.15.12:0/4056097993 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1ba8011ac0 con 0x7f1bac108120 2024-02-18T11:31:36.527 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:31.331+0000 7f1bb3abc700 1 -- 172.21.15.12:0/4056097993 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1bac108120 msgr2=0x7f1bac10a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:31:36.527 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:31.331+0000 7f1bb3abc700 1 --2- 172.21.15.12:0/4056097993 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1bac108120 0x7f1bac10a510 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1ba80098a0 tx=0x7f1ba8009bb0 comp rx=0 tx=0).stop 2024-02-18T11:31:36.527 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:31.332+0000 7f1bb3abc700 1 -- 172.21.15.12:0/4056097993 shutdown_connections 2024-02-18T11:31:36.527 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:31.332+0000 7f1bb3abc700 1 --2- 172.21.15.12:0/4056097993 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1bac108120 0x7f1bac10a510 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:31:36.527 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:31.332+0000 7f1bb3abc700 1 -- 172.21.15.12:0/4056097993 >> 172.21.15.12:0/4056097993 conn(0x7f1bac101bd0 msgr2=0x7f1bac103ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:31:36.527 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:31.332+0000 7f1bb3abc700 1 -- 172.21.15.12:0/4056097993 shutdown_connections 2024-02-18T11:31:36.528 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:31.332+0000 7f1bb3abc700 1 -- 172.21.15.12:0/4056097993 wait complete. 2024-02-18T11:31:36.528 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:31.332+0000 7f1bb3abc700 1 Processor -- start 2024-02-18T11:31:36.528 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:31.332+0000 7f1bb3abc700 1 -- start start 2024-02-18T11:31:36.528 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:31.332+0000 7f1bb3abc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1bac108120 0x7f1bac19b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:31:36.528 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:31.332+0000 7f1bb3abc700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bac19b9a0 con 0x7f1bac108120 2024-02-18T11:31:36.528 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:31.332+0000 7f1bb1858700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1bac108120 0x7f1bac19b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:31:36.528 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:31.332+0000 7f1bb1858700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1bac108120 0x7f1bac19b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51548/0 (socket says 172.21.15.12:51548) 2024-02-18T11:31:36.528 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:31.332+0000 7f1bb1858700 1 -- 172.21.15.12:0/1692493632 learned_addr learned my addr 172.21.15.12:0/1692493632 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:31:36.528 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:31.333+0000 7f1bb1858700 1 -- 172.21.15.12:0/1692493632 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ba8009550 con 0x7f1bac108120 2024-02-18T11:31:36.528 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:31.333+0000 7f1bb1858700 1 --2- 172.21.15.12:0/1692493632 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1bac108120 0x7f1bac19b460 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1ba8009130 tx=0x7f1ba8011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:31:36.528 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:31.333+0000 7f1ba2ffd700 1 -- 172.21.15.12:0/1692493632 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1ba8003aa0 con 0x7f1bac108120 2024-02-18T11:31:36.528 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:31.333+0000 7f1bb3abc700 1 -- 172.21.15.12:0/1692493632 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bac19bba0 con 0x7f1bac108120 2024-02-18T11:31:36.528 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:31.333+0000 7f1ba2ffd700 1 -- 172.21.15.12:0/1692493632 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1ba8003c00 con 0x7f1bac108120 2024-02-18T11:31:36.529 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:31.333+0000 7f1ba2ffd700 1 -- 172.21.15.12:0/1692493632 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1ba8024800 con 0x7f1bac108120 2024-02-18T11:31:36.529 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:31.333+0000 7f1bb3abc700 1 -- 172.21.15.12:0/1692493632 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bac19c040 con 0x7f1bac108120 2024-02-18T11:31:36.529 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:31.334+0000 7f1ba2ffd700 1 -- 172.21.15.12:0/1692493632 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 8) v1 ==== 46091+0+0 (secure 0 0 0) 0x7f1ba801a430 con 0x7f1bac108120 2024-02-18T11:31:36.529 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:31.334+0000 7f1bb3abc700 1 -- 172.21.15.12:0/1692493632 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bac10b8b0 con 0x7f1bac108120 2024-02-18T11:31:36.529 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:31.334+0000 7f1ba2ffd700 1 --2- 172.21.15.12:0/1692493632 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7f1b98039110 0x7f1b9803b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:31:36.529 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:31.334+0000 7f1ba2ffd700 1 -- 172.21.15.12:0/1692493632 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f1ba804d3c0 con 0x7f1bac108120 2024-02-18T11:31:36.529 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:31.334+0000 7f1bb1057700 1 --2- 172.21.15.12:0/1692493632 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7f1b98039110 0x7f1b9803b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:31:36.529 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:31.335+0000 7f1bb1057700 1 --2- 172.21.15.12:0/1692493632 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7f1b98039110 0x7f1b9803b5d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c006fd0 tx=0x7f1b9c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:31:36.529 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:31.339+0000 7f1ba2ffd700 1 -- 172.21.15.12:0/1692493632 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1ba801abd0 con 0x7f1bac108120 2024-02-18T11:31:36.529 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:31.547+0000 7f1bb3abc700 1 -- 172.21.15.12:0/1692493632 --> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi012", "addr": "172.21.15.12", "target": ["mon-mgr", ""]}) v1 -- 0x7f1bac102e60 con 0x7f1b98039110 2024-02-18T11:31:36.529 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:35.940+0000 7f1ba2ffd700 1 -- 172.21.15.12:0/1692493632 <== mgr.14120 v2:172.21.15.12:6800/4135274411 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f1bac102e60 con 0x7f1b98039110 2024-02-18T11:31:36.529 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:35.942+0000 7f1bb3abc700 1 -- 172.21.15.12:0/1692493632 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7f1b98039110 msgr2=0x7f1b9803b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:31:36.529 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:35.942+0000 7f1bb3abc700 1 --2- 172.21.15.12:0/1692493632 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7f1b98039110 0x7f1b9803b5d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c006fd0 tx=0x7f1b9c006e40 comp rx=0 tx=0).stop 2024-02-18T11:31:36.529 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:35.942+0000 7f1bb3abc700 1 -- 172.21.15.12:0/1692493632 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1bac108120 msgr2=0x7f1bac19b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:31:36.529 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:35.942+0000 7f1bb3abc700 1 --2- 172.21.15.12:0/1692493632 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1bac108120 0x7f1bac19b460 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1ba8009130 tx=0x7f1ba8011b40 comp rx=0 tx=0).stop 2024-02-18T11:31:36.530 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:35.942+0000 7f1bb3abc700 1 -- 172.21.15.12:0/1692493632 shutdown_connections 2024-02-18T11:31:36.530 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:35.942+0000 7f1bb3abc700 1 --2- 172.21.15.12:0/1692493632 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7f1b98039110 0x7f1b9803b5d0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:31:36.530 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:35.942+0000 7f1bb3abc700 1 --2- 172.21.15.12:0/1692493632 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1bac108120 0x7f1bac19b460 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:31:36.530 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:35.942+0000 7f1bb3abc700 1 -- 172.21.15.12:0/1692493632 >> 172.21.15.12:0/1692493632 conn(0x7f1bac101bd0 msgr2=0x7f1bac102750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:31:36.530 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:35.942+0000 7f1bb3abc700 1 -- 172.21.15.12:0/1692493632 shutdown_connections 2024-02-18T11:31:36.530 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:35.943+0000 7f1bb3abc700 1 -- 172.21.15.12:0/1692493632 wait complete. 2024-02-18T11:31:36.530 INFO:teuthology.orchestra.run.smithi012.stdout:Deploying unmanaged mon service... 2024-02-18T11:31:37.019 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:36 smithi012 ceph-mon[32460]: from='mgr.14120 172.21.15.12:0/2151529525' entity='mgr.x' 2024-02-18T11:31:37.019 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:36 smithi012 ceph-mon[32460]: Added host smithi012 2024-02-18T11:31:37.020 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:36 smithi012 ceph-mon[32460]: from='mgr.14120 172.21.15.12:0/2151529525' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:31:37.929 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-02-18T11:31:37.929 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:37.022+0000 7fd955b63700 1 Processor -- start 2024-02-18T11:31:37.929 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:37.022+0000 7fd955b63700 1 -- start start 2024-02-18T11:31:37.929 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:37.022+0000 7fd955b63700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9501065c0 0x7fd9501069e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:31:37.929 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:37.022+0000 7fd955b63700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd950106fb0 con 0x7fd9501065c0 2024-02-18T11:31:37.929 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:37.022+0000 7fd94f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9501065c0 0x7fd9501069e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:31:37.930 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:37.022+0000 7fd94f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9501065c0 0x7fd9501069e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33112/0 (socket says 172.21.15.12:33112) 2024-02-18T11:31:37.930 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:37.022+0000 7fd94f7fe700 1 -- 172.21.15.12:0/2352784372 learned_addr learned my addr 172.21.15.12:0/2352784372 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:31:37.930 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:37.023+0000 7fd94f7fe700 1 -- 172.21.15.12:0/2352784372 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9501077c0 con 0x7fd9501065c0 2024-02-18T11:31:37.930 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:37.023+0000 7fd94f7fe700 1 --2- 172.21.15.12:0/2352784372 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9501065c0 0x7fd9501069e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd940009e80 tx=0x7fd94000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=339c248dd1d7f349 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:31:37.930 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:37.023+0000 7fd94e7fc700 1 -- 172.21.15.12:0/2352784372 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd94000e070 con 0x7fd9501065c0 2024-02-18T11:31:37.930 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:37.023+0000 7fd94e7fc700 1 -- 172.21.15.12:0/2352784372 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd94000b8c0 con 0x7fd9501065c0 2024-02-18T11:31:37.930 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:37.023+0000 7fd94e7fc700 1 -- 172.21.15.12:0/2352784372 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd940004550 con 0x7fd9501065c0 2024-02-18T11:31:37.930 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:37.023+0000 7fd955b63700 1 -- 172.21.15.12:0/2352784372 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9501065c0 msgr2=0x7fd9501069e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:31:37.930 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:37.023+0000 7fd955b63700 1 --2- 172.21.15.12:0/2352784372 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9501065c0 0x7fd9501069e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd940009e80 tx=0x7fd94000b330 comp rx=0 tx=0).stop 2024-02-18T11:31:37.930 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:37.024+0000 7fd955b63700 1 -- 172.21.15.12:0/2352784372 shutdown_connections 2024-02-18T11:31:37.930 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:37.024+0000 7fd955b63700 1 --2- 172.21.15.12:0/2352784372 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9501065c0 0x7fd9501069e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:31:37.931 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:37.024+0000 7fd955b63700 1 -- 172.21.15.12:0/2352784372 >> 172.21.15.12:0/2352784372 conn(0x7fd950101bc0 msgr2=0x7fd950103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:31:37.931 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:37.024+0000 7fd955b63700 1 -- 172.21.15.12:0/2352784372 shutdown_connections 2024-02-18T11:31:37.931 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:37.024+0000 7fd955b63700 1 -- 172.21.15.12:0/2352784372 wait complete. 2024-02-18T11:31:37.931 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:37.024+0000 7fd955b63700 1 Processor -- start 2024-02-18T11:31:37.931 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:37.024+0000 7fd955b63700 1 -- start start 2024-02-18T11:31:37.931 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:37.025+0000 7fd955b63700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9501065c0 0x7fd95019b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:31:37.931 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:37.025+0000 7fd955b63700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd95019b980 con 0x7fd9501065c0 2024-02-18T11:31:37.931 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:37.025+0000 7fd94f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9501065c0 0x7fd95019b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:31:37.931 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:37.025+0000 7fd94f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9501065c0 0x7fd95019b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33126/0 (socket says 172.21.15.12:33126) 2024-02-18T11:31:37.931 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:37.025+0000 7fd94f7fe700 1 -- 172.21.15.12:0/2655419055 learned_addr learned my addr 172.21.15.12:0/2655419055 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:31:37.931 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:37.025+0000 7fd94f7fe700 1 -- 172.21.15.12:0/2655419055 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd940009b30 con 0x7fd9501065c0 2024-02-18T11:31:37.931 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:37.025+0000 7fd94f7fe700 1 --2- 172.21.15.12:0/2655419055 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9501065c0 0x7fd95019b440 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd940009fd0 tx=0x7fd940003d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:31:37.931 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:37.025+0000 7fd94cff9700 1 -- 172.21.15.12:0/2655419055 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd94000e040 con 0x7fd9501065c0 2024-02-18T11:31:37.931 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:37.025+0000 7fd955b63700 1 -- 172.21.15.12:0/2655419055 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd95019bb80 con 0x7fd9501065c0 2024-02-18T11:31:37.932 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:37.025+0000 7fd94cff9700 1 -- 172.21.15.12:0/2655419055 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd9400128f0 con 0x7fd9501065c0 2024-02-18T11:31:37.932 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:37.025+0000 7fd94cff9700 1 -- 172.21.15.12:0/2655419055 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd94001c890 con 0x7fd9501065c0 2024-02-18T11:31:37.932 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:37.025+0000 7fd955b63700 1 -- 172.21.15.12:0/2655419055 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd95019c000 con 0x7fd9501065c0 2024-02-18T11:31:37.932 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:37.026+0000 7fd94cff9700 1 -- 172.21.15.12:0/2655419055 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 8) v1 ==== 46091+0+0 (secure 0 0 0) 0x7fd94001cb00 con 0x7fd9501065c0 2024-02-18T11:31:37.932 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:37.026+0000 7fd955b63700 1 -- 172.21.15.12:0/2655419055 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd950195660 con 0x7fd9501065c0 2024-02-18T11:31:37.932 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:37.026+0000 7fd94cff9700 1 --2- 172.21.15.12:0/2655419055 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7fd93c039110 0x7fd93c03b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:31:37.932 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:37.026+0000 7fd94cff9700 1 -- 172.21.15.12:0/2655419055 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fd94004d270 con 0x7fd9501065c0 2024-02-18T11:31:37.932 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:37.026+0000 7fd94effd700 1 --2- 172.21.15.12:0/2655419055 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7fd93c039110 0x7fd93c03b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:31:37.932 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:37.027+0000 7fd94effd700 1 --2- 172.21.15.12:0/2655419055 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7fd93c039110 0x7fd93c03b5d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd938006fd0 tx=0x7fd938006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:31:37.932 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:37.031+0000 7fd94cff9700 1 -- 172.21.15.12:0/2655419055 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd940011990 con 0x7fd9501065c0 2024-02-18T11:31:37.932 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:37.261+0000 7fd955b63700 1 -- 172.21.15.12:0/2655419055 --> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fd95002da40 con 0x7fd93c039110 2024-02-18T11:31:37.932 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:37.269+0000 7fd94cff9700 1 -- 172.21.15.12:0/2655419055 <== mgr.14120 v2:172.21.15.12:6800/4135274411 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fd95002da40 con 0x7fd93c039110 2024-02-18T11:31:37.932 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:37.271+0000 7fd955b63700 1 -- 172.21.15.12:0/2655419055 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7fd93c039110 msgr2=0x7fd93c03b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:31:37.933 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:37.271+0000 7fd955b63700 1 --2- 172.21.15.12:0/2655419055 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7fd93c039110 0x7fd93c03b5d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd938006fd0 tx=0x7fd938006e40 comp rx=0 tx=0).stop 2024-02-18T11:31:37.933 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:37.271+0000 7fd955b63700 1 -- 172.21.15.12:0/2655419055 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9501065c0 msgr2=0x7fd95019b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:31:37.933 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:37.271+0000 7fd955b63700 1 --2- 172.21.15.12:0/2655419055 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9501065c0 0x7fd95019b440 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd940009fd0 tx=0x7fd940003d70 comp rx=0 tx=0).stop 2024-02-18T11:31:37.933 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:37.271+0000 7fd955b63700 1 -- 172.21.15.12:0/2655419055 shutdown_connections 2024-02-18T11:31:37.933 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:37.271+0000 7fd955b63700 1 --2- 172.21.15.12:0/2655419055 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7fd93c039110 0x7fd93c03b5d0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:31:37.933 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:37.271+0000 7fd955b63700 1 --2- 172.21.15.12:0/2655419055 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9501065c0 0x7fd95019b440 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:31:37.933 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:37.271+0000 7fd955b63700 1 -- 172.21.15.12:0/2655419055 >> 172.21.15.12:0/2655419055 conn(0x7fd950101bc0 msgr2=0x7fd950102980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:31:37.933 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:37.271+0000 7fd955b63700 1 -- 172.21.15.12:0/2655419055 shutdown_connections 2024-02-18T11:31:37.933 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:37.271+0000 7fd955b63700 1 -- 172.21.15.12:0/2655419055 wait complete. 2024-02-18T11:31:37.933 INFO:teuthology.orchestra.run.smithi012.stdout:Deploying unmanaged mgr service... 2024-02-18T11:31:38.542 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:38 smithi012 ceph-mon[32460]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:31:38.543 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:38 smithi012 ceph-mon[32460]: Saving service mon spec with placement count:5 2024-02-18T11:31:38.543 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:38 smithi012 ceph-mon[32460]: from='mgr.14120 172.21.15.12:0/2151529525' entity='mgr.x' 2024-02-18T11:31:39.665 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-02-18T11:31:39.665 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:38.761+0000 7f257615e700 1 Processor -- start 2024-02-18T11:31:39.665 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:38.761+0000 7f257615e700 1 -- start start 2024-02-18T11:31:39.665 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:38.761+0000 7f257615e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f25701065b0 0x7f25701069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:31:39.665 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:38.761+0000 7f257615e700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2570106fa0 con 0x7f25701065b0 2024-02-18T11:31:39.665 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:38.762+0000 7f256f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f25701065b0 0x7f25701069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:31:39.666 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:38.762+0000 7f256f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f25701065b0 0x7f25701069d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33130/0 (socket says 172.21.15.12:33130) 2024-02-18T11:31:39.666 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:38.762+0000 7f256f7fe700 1 -- 172.21.15.12:0/751720664 learned_addr learned my addr 172.21.15.12:0/751720664 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:31:39.666 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:38.762+0000 7f256f7fe700 1 -- 172.21.15.12:0/751720664 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25701077b0 con 0x7f25701065b0 2024-02-18T11:31:39.666 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:38.762+0000 7f256f7fe700 1 --2- 172.21.15.12:0/751720664 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f25701065b0 0x7f25701069d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f25600098a0 tx=0x7f2560009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f97935482351527e server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:31:39.666 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:38.762+0000 7f256e7fc700 1 -- 172.21.15.12:0/751720664 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2560004160 con 0x7f25701065b0 2024-02-18T11:31:39.666 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:38.762+0000 7f256e7fc700 1 -- 172.21.15.12:0/751720664 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f25600042c0 con 0x7f25701065b0 2024-02-18T11:31:39.666 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:38.762+0000 7f256e7fc700 1 -- 172.21.15.12:0/751720664 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2560011ac0 con 0x7f25701065b0 2024-02-18T11:31:39.666 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:38.763+0000 7f257615e700 1 -- 172.21.15.12:0/751720664 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f25701065b0 msgr2=0x7f25701069d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:31:39.666 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:38.763+0000 7f257615e700 1 --2- 172.21.15.12:0/751720664 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f25701065b0 0x7f25701069d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f25600098a0 tx=0x7f2560009bb0 comp rx=0 tx=0).stop 2024-02-18T11:31:39.666 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:38.763+0000 7f257615e700 1 -- 172.21.15.12:0/751720664 shutdown_connections 2024-02-18T11:31:39.666 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:38.763+0000 7f257615e700 1 --2- 172.21.15.12:0/751720664 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f25701065b0 0x7f25701069d0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:31:39.666 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:38.763+0000 7f257615e700 1 -- 172.21.15.12:0/751720664 >> 172.21.15.12:0/751720664 conn(0x7f2570101b90 msgr2=0x7f2570103fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:31:39.666 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:38.763+0000 7f257615e700 1 -- 172.21.15.12:0/751720664 shutdown_connections 2024-02-18T11:31:39.666 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:38.763+0000 7f257615e700 1 -- 172.21.15.12:0/751720664 wait complete. 2024-02-18T11:31:39.667 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:38.763+0000 7f257615e700 1 Processor -- start 2024-02-18T11:31:39.667 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:38.764+0000 7f257615e700 1 -- start start 2024-02-18T11:31:39.667 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:38.764+0000 7f257615e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f25701065b0 0x7f2570191720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:31:39.667 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:38.764+0000 7f257615e700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2570191c60 con 0x7f25701065b0 2024-02-18T11:31:39.667 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:38.764+0000 7f256f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f25701065b0 0x7f2570191720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:31:39.667 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:38.764+0000 7f256f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f25701065b0 0x7f2570191720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33132/0 (socket says 172.21.15.12:33132) 2024-02-18T11:31:39.667 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:38.764+0000 7f256f7fe700 1 -- 172.21.15.12:0/1646645353 learned_addr learned my addr 172.21.15.12:0/1646645353 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:31:39.667 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:38.764+0000 7f256f7fe700 1 -- 172.21.15.12:0/1646645353 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2560009550 con 0x7f25701065b0 2024-02-18T11:31:39.667 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:38.764+0000 7f256f7fe700 1 --2- 172.21.15.12:0/1646645353 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f25701065b0 0x7f2570191720 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2560003f70 tx=0x7f2560011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:31:39.667 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:38.765+0000 7f256cff9700 1 -- 172.21.15.12:0/1646645353 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f25600039f0 con 0x7f25701065b0 2024-02-18T11:31:39.667 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:38.765+0000 7f256cff9700 1 -- 172.21.15.12:0/1646645353 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2560003b50 con 0x7f25701065b0 2024-02-18T11:31:39.667 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:38.765+0000 7f257615e700 1 -- 172.21.15.12:0/1646645353 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f257018fe50 con 0x7f25701065b0 2024-02-18T11:31:39.667 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:38.765+0000 7f256cff9700 1 -- 172.21.15.12:0/1646645353 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f256001a690 con 0x7f25701065b0 2024-02-18T11:31:39.667 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:38.765+0000 7f257615e700 1 -- 172.21.15.12:0/1646645353 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25701902f0 con 0x7f25701065b0 2024-02-18T11:31:39.668 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:38.765+0000 7f256cff9700 1 -- 172.21.15.12:0/1646645353 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 8) v1 ==== 46091+0+0 (secure 0 0 0) 0x7f256001a970 con 0x7f25701065b0 2024-02-18T11:31:39.668 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:38.766+0000 7f257615e700 1 -- 172.21.15.12:0/1646645353 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f257010a9e0 con 0x7f25701065b0 2024-02-18T11:31:39.668 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:38.766+0000 7f256cff9700 1 --2- 172.21.15.12:0/1646645353 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7f25580390c0 0x7f255803b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:31:39.668 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:38.766+0000 7f256cff9700 1 -- 172.21.15.12:0/1646645353 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f256004d6e0 con 0x7f25701065b0 2024-02-18T11:31:39.668 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:38.766+0000 7f256effd700 1 --2- 172.21.15.12:0/1646645353 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7f25580390c0 0x7f255803b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:31:39.668 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:38.766+0000 7f256effd700 1 --2- 172.21.15.12:0/1646645353 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7f25580390c0 0x7f255803b580 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2564006fd0 tx=0x7f2564006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:31:39.668 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:38.770+0000 7f256cff9700 1 -- 172.21.15.12:0/1646645353 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f256001ac20 con 0x7f25701065b0 2024-02-18T11:31:39.668 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:38.991+0000 7f257615e700 1 -- 172.21.15.12:0/1646645353 --> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f2570195310 con 0x7f25580390c0 2024-02-18T11:31:39.668 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:38.995+0000 7f256cff9700 1 -- 172.21.15.12:0/1646645353 <== mgr.14120 v2:172.21.15.12:6800/4135274411 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f2570195310 con 0x7f25580390c0 2024-02-18T11:31:39.668 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:38.997+0000 7f257615e700 1 -- 172.21.15.12:0/1646645353 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7f25580390c0 msgr2=0x7f255803b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:31:39.668 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:38.997+0000 7f257615e700 1 --2- 172.21.15.12:0/1646645353 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7f25580390c0 0x7f255803b580 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2564006fd0 tx=0x7f2564006e40 comp rx=0 tx=0).stop 2024-02-18T11:31:39.669 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:38.997+0000 7f257615e700 1 -- 172.21.15.12:0/1646645353 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f25701065b0 msgr2=0x7f2570191720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:31:39.669 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:38.997+0000 7f257615e700 1 --2- 172.21.15.12:0/1646645353 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f25701065b0 0x7f2570191720 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2560003f70 tx=0x7f2560011b40 comp rx=0 tx=0).stop 2024-02-18T11:31:39.669 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:38.997+0000 7f257615e700 1 -- 172.21.15.12:0/1646645353 shutdown_connections 2024-02-18T11:31:39.669 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:38.997+0000 7f257615e700 1 --2- 172.21.15.12:0/1646645353 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7f25580390c0 0x7f255803b580 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:31:39.669 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:38.997+0000 7f257615e700 1 --2- 172.21.15.12:0/1646645353 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f25701065b0 0x7f2570191720 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:31:39.669 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:38.997+0000 7f257615e700 1 -- 172.21.15.12:0/1646645353 >> 172.21.15.12:0/1646645353 conn(0x7f2570101b90 msgr2=0x7f2570102a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:31:39.669 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:38.997+0000 7f257615e700 1 -- 172.21.15.12:0/1646645353 shutdown_connections 2024-02-18T11:31:39.669 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:38.998+0000 7f257615e700 1 -- 172.21.15.12:0/1646645353 wait complete. 2024-02-18T11:31:40.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:39 smithi012 ceph-mon[32460]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:31:40.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:39 smithi012 ceph-mon[32460]: Saving service mgr spec with placement count:2 2024-02-18T11:31:40.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:39 smithi012 ceph-mon[32460]: from='mgr.14120 172.21.15.12:0/2151529525' entity='mgr.x' 2024-02-18T11:31:40.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:39 smithi012 ceph-mon[32460]: from='mgr.14120 172.21.15.12:0/2151529525' entity='mgr.x' 2024-02-18T11:31:40.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:39 smithi012 ceph-mon[32460]: from='mgr.14120 172.21.15.12:0/2151529525' entity='mgr.x' 2024-02-18T11:31:41.132 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:40.119+0000 7fd158436700 1 Processor -- start 2024-02-18T11:31:41.132 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:40.119+0000 7fd158436700 1 -- start start 2024-02-18T11:31:41.132 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:40.119+0000 7fd158436700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1501076a0 0x7fd150107ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:31:41.132 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:40.119+0000 7fd158436700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd150108090 con 0x7fd1501076a0 2024-02-18T11:31:41.132 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:40.120+0000 7fd1561d2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1501076a0 0x7fd150107ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:31:41.132 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:40.120+0000 7fd1561d2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1501076a0 0x7fd150107ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33136/0 (socket says 172.21.15.12:33136) 2024-02-18T11:31:41.133 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:40.120+0000 7fd1561d2700 1 -- 172.21.15.12:0/3757465331 learned_addr learned my addr 172.21.15.12:0/3757465331 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:31:41.133 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:40.120+0000 7fd1561d2700 1 -- 172.21.15.12:0/3757465331 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd150108850 con 0x7fd1501076a0 2024-02-18T11:31:41.133 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:40.120+0000 7fd1561d2700 1 --2- 172.21.15.12:0/3757465331 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1501076a0 0x7fd150107ac0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd1400098a0 tx=0x7fd140009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed9b6a5b365d152a server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:31:41.133 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:40.120+0000 7fd1551d0700 1 -- 172.21.15.12:0/3757465331 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd140004160 con 0x7fd1501076a0 2024-02-18T11:31:41.133 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:40.120+0000 7fd1551d0700 1 -- 172.21.15.12:0/3757465331 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd1400042c0 con 0x7fd1501076a0 2024-02-18T11:31:41.133 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:40.120+0000 7fd1551d0700 1 -- 172.21.15.12:0/3757465331 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd140011a80 con 0x7fd1501076a0 2024-02-18T11:31:41.133 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:40.121+0000 7fd158436700 1 -- 172.21.15.12:0/3757465331 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1501076a0 msgr2=0x7fd150107ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:31:41.133 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:40.121+0000 7fd158436700 1 --2- 172.21.15.12:0/3757465331 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1501076a0 0x7fd150107ac0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd1400098a0 tx=0x7fd140009bb0 comp rx=0 tx=0).stop 2024-02-18T11:31:41.133 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:40.121+0000 7fd158436700 1 -- 172.21.15.12:0/3757465331 shutdown_connections 2024-02-18T11:31:41.133 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:40.121+0000 7fd158436700 1 --2- 172.21.15.12:0/3757465331 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1501076a0 0x7fd150107ac0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:31:41.133 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:40.121+0000 7fd158436700 1 -- 172.21.15.12:0/3757465331 >> 172.21.15.12:0/3757465331 conn(0x7fd150102d00 msgr2=0x7fd150105120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:31:41.133 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:40.121+0000 7fd158436700 1 -- 172.21.15.12:0/3757465331 shutdown_connections 2024-02-18T11:31:41.133 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:40.121+0000 7fd158436700 1 -- 172.21.15.12:0/3757465331 wait complete. 2024-02-18T11:31:41.133 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:40.122+0000 7fd158436700 1 Processor -- start 2024-02-18T11:31:41.134 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:40.122+0000 7fd158436700 1 -- start start 2024-02-18T11:31:41.134 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:40.122+0000 7fd158436700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1501076a0 0x7fd15019b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:31:41.134 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:40.122+0000 7fd158436700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd15019b950 con 0x7fd1501076a0 2024-02-18T11:31:41.134 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:40.122+0000 7fd1561d2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1501076a0 0x7fd15019b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:31:41.134 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:40.122+0000 7fd1561d2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1501076a0 0x7fd15019b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33148/0 (socket says 172.21.15.12:33148) 2024-02-18T11:31:41.134 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:40.122+0000 7fd1561d2700 1 -- 172.21.15.12:0/2194899228 learned_addr learned my addr 172.21.15.12:0/2194899228 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:31:41.134 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:40.122+0000 7fd1561d2700 1 -- 172.21.15.12:0/2194899228 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd140009550 con 0x7fd1501076a0 2024-02-18T11:31:41.134 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:40.122+0000 7fd1561d2700 1 --2- 172.21.15.12:0/2194899228 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1501076a0 0x7fd15019b410 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd140011740 tx=0x7fd140011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:31:41.134 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:40.123+0000 7fd1477fe700 1 -- 172.21.15.12:0/2194899228 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd1400039e0 con 0x7fd1501076a0 2024-02-18T11:31:41.134 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:40.123+0000 7fd1477fe700 1 -- 172.21.15.12:0/2194899228 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd140003b40 con 0x7fd1501076a0 2024-02-18T11:31:41.134 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:40.123+0000 7fd158436700 1 -- 172.21.15.12:0/2194899228 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd15019bb50 con 0x7fd1501076a0 2024-02-18T11:31:41.134 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:40.123+0000 7fd1477fe700 1 -- 172.21.15.12:0/2194899228 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd140024740 con 0x7fd1501076a0 2024-02-18T11:31:41.135 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:40.123+0000 7fd158436700 1 -- 172.21.15.12:0/2194899228 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd15019bff0 con 0x7fd1501076a0 2024-02-18T11:31:41.135 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:40.124+0000 7fd1477fe700 1 -- 172.21.15.12:0/2194899228 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 8) v1 ==== 46091+0+0 (secure 0 0 0) 0x7fd14002d3f0 con 0x7fd1501076a0 2024-02-18T11:31:41.135 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:40.124+0000 7fd158436700 1 -- 172.21.15.12:0/2194899228 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd15010ba80 con 0x7fd1501076a0 2024-02-18T11:31:41.135 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:40.124+0000 7fd1477fe700 1 --2- 172.21.15.12:0/2194899228 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7fd13c039110 0x7fd13c03b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:31:41.135 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:40.124+0000 7fd1477fe700 1 -- 172.21.15.12:0/2194899228 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fd14004d5c0 con 0x7fd1501076a0 2024-02-18T11:31:41.135 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:40.124+0000 7fd1559d1700 1 --2- 172.21.15.12:0/2194899228 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7fd13c039110 0x7fd13c03b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:31:41.135 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:40.124+0000 7fd1559d1700 1 --2- 172.21.15.12:0/2194899228 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7fd13c039110 0x7fd13c03b5d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd14c006fd0 tx=0x7fd14c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:31:41.135 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:40.133+0000 7fd1477fe700 1 -- 172.21.15.12:0/2194899228 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd140010970 con 0x7fd1501076a0 2024-02-18T11:31:41.135 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:40.341+0000 7fd158436700 1 -- 172.21.15.12:0/2194899228 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7fd15019e930 con 0x7fd1501076a0 2024-02-18T11:31:41.135 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:40.347+0000 7fd1477fe700 1 -- 172.21.15.12:0/2194899228 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7fd140010380 con 0x7fd1501076a0 2024-02-18T11:31:41.135 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:40.349+0000 7fd158436700 1 -- 172.21.15.12:0/2194899228 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7fd13c039110 msgr2=0x7fd13c03b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:31:41.135 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:40.349+0000 7fd158436700 1 --2- 172.21.15.12:0/2194899228 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7fd13c039110 0x7fd13c03b5d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd14c006fd0 tx=0x7fd14c006e40 comp rx=0 tx=0).stop 2024-02-18T11:31:41.136 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:40.349+0000 7fd158436700 1 -- 172.21.15.12:0/2194899228 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1501076a0 msgr2=0x7fd15019b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:31:41.136 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:40.349+0000 7fd158436700 1 --2- 172.21.15.12:0/2194899228 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1501076a0 0x7fd15019b410 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd140011740 tx=0x7fd140011b00 comp rx=0 tx=0).stop 2024-02-18T11:31:41.136 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:40.349+0000 7fd158436700 1 -- 172.21.15.12:0/2194899228 shutdown_connections 2024-02-18T11:31:41.136 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:40.349+0000 7fd158436700 1 --2- 172.21.15.12:0/2194899228 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7fd13c039110 0x7fd13c03b5d0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:31:41.136 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:40.349+0000 7fd158436700 1 --2- 172.21.15.12:0/2194899228 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1501076a0 0x7fd15019b410 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:31:41.136 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:40.349+0000 7fd158436700 1 -- 172.21.15.12:0/2194899228 >> 172.21.15.12:0/2194899228 conn(0x7fd150102d00 msgr2=0x7fd1501037b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:31:41.136 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:40.350+0000 7fd158436700 1 -- 172.21.15.12:0/2194899228 shutdown_connections 2024-02-18T11:31:41.136 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:40.350+0000 7fd158436700 1 -- 172.21.15.12:0/2194899228 wait complete. 2024-02-18T11:31:41.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:41 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/2194899228' entity='client.admin' 2024-02-18T11:31:42.280 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:41.543+0000 7f795ff0f700 1 Processor -- start 2024-02-18T11:31:42.280 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:41.543+0000 7f795ff0f700 1 -- start start 2024-02-18T11:31:42.281 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:41.543+0000 7f795ff0f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f79581065c0 0x7f79581069e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:31:42.281 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:41.543+0000 7f795ff0f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7958106fb0 con 0x7f79581065c0 2024-02-18T11:31:42.281 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:41.543+0000 7f795dcab700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f79581065c0 0x7f79581069e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:31:42.281 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:41.543+0000 7f795dcab700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f79581065c0 0x7f79581069e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53504/0 (socket says 172.21.15.12:53504) 2024-02-18T11:31:42.281 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:41.543+0000 7f795dcab700 1 -- 172.21.15.12:0/459358690 learned_addr learned my addr 172.21.15.12:0/459358690 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:31:42.281 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:41.544+0000 7f795dcab700 1 -- 172.21.15.12:0/459358690 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79581077c0 con 0x7f79581065c0 2024-02-18T11:31:42.281 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:41.544+0000 7f795dcab700 1 --2- 172.21.15.12:0/459358690 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f79581065c0 0x7f79581069e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f79540098a0 tx=0x7f7954009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da43cb3fc89df0e0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:31:42.281 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:41.544+0000 7f795cca9700 1 -- 172.21.15.12:0/459358690 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7954004160 con 0x7f79581065c0 2024-02-18T11:31:42.281 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:41.544+0000 7f795cca9700 1 -- 172.21.15.12:0/459358690 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f79540042c0 con 0x7f79581065c0 2024-02-18T11:31:42.281 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:41.544+0000 7f795cca9700 1 -- 172.21.15.12:0/459358690 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7954011a80 con 0x7f79581065c0 2024-02-18T11:31:42.281 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:41.544+0000 7f795ff0f700 1 -- 172.21.15.12:0/459358690 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f79581065c0 msgr2=0x7f79581069e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:31:42.281 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:41.544+0000 7f795ff0f700 1 --2- 172.21.15.12:0/459358690 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f79581065c0 0x7f79581069e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f79540098a0 tx=0x7f7954009bb0 comp rx=0 tx=0).stop 2024-02-18T11:31:42.281 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:41.544+0000 7f795ff0f700 1 -- 172.21.15.12:0/459358690 shutdown_connections 2024-02-18T11:31:42.281 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:41.545+0000 7f795ff0f700 1 --2- 172.21.15.12:0/459358690 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f79581065c0 0x7f79581069e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:31:42.282 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:41.545+0000 7f795ff0f700 1 -- 172.21.15.12:0/459358690 >> 172.21.15.12:0/459358690 conn(0x7f7958101bc0 msgr2=0x7f7958103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:31:42.282 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:41.545+0000 7f795ff0f700 1 -- 172.21.15.12:0/459358690 shutdown_connections 2024-02-18T11:31:42.282 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:41.545+0000 7f795ff0f700 1 -- 172.21.15.12:0/459358690 wait complete. 2024-02-18T11:31:42.282 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:41.545+0000 7f795ff0f700 1 Processor -- start 2024-02-18T11:31:42.282 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:41.545+0000 7f795ff0f700 1 -- start start 2024-02-18T11:31:42.282 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:41.545+0000 7f795ff0f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f79581065c0 0x7f795819b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:31:42.282 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:41.545+0000 7f795ff0f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f795819bab0 con 0x7f79581065c0 2024-02-18T11:31:42.282 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:41.545+0000 7f795dcab700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f79581065c0 0x7f795819b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:31:42.282 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:41.545+0000 7f795dcab700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f79581065c0 0x7f795819b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53514/0 (socket says 172.21.15.12:53514) 2024-02-18T11:31:42.282 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:41.545+0000 7f795dcab700 1 -- 172.21.15.12:0/3637241209 learned_addr learned my addr 172.21.15.12:0/3637241209 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:31:42.282 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:41.546+0000 7f795dcab700 1 -- 172.21.15.12:0/3637241209 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7954009550 con 0x7f79581065c0 2024-02-18T11:31:42.282 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:41.546+0000 7f795dcab700 1 --2- 172.21.15.12:0/3637241209 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f79581065c0 0x7f795819b570 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7954003f20 tx=0x7f7954011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:31:42.282 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:41.546+0000 7f794effd700 1 -- 172.21.15.12:0/3637241209 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f79540038b0 con 0x7f79581065c0 2024-02-18T11:31:42.282 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:41.546+0000 7f794effd700 1 -- 172.21.15.12:0/3637241209 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7954003a10 con 0x7f79581065c0 2024-02-18T11:31:42.282 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:41.546+0000 7f795ff0f700 1 -- 172.21.15.12:0/3637241209 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f795819bcb0 con 0x7f79581065c0 2024-02-18T11:31:42.283 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:41.546+0000 7f794effd700 1 -- 172.21.15.12:0/3637241209 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f795401a5c0 con 0x7f79581065c0 2024-02-18T11:31:42.283 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:41.546+0000 7f795ff0f700 1 -- 172.21.15.12:0/3637241209 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f795819c150 con 0x7f79581065c0 2024-02-18T11:31:42.283 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:41.547+0000 7f794effd700 1 -- 172.21.15.12:0/3637241209 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 8) v1 ==== 46091+0+0 (secure 0 0 0) 0x7f795401a830 con 0x7f79581065c0 2024-02-18T11:31:42.283 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:41.547+0000 7f795ff0f700 1 -- 172.21.15.12:0/3637241209 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7958195660 con 0x7f79581065c0 2024-02-18T11:31:42.283 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:41.547+0000 7f794effd700 1 --2- 172.21.15.12:0/3637241209 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7f79440390c0 0x7f794403b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:31:42.283 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:41.547+0000 7f794effd700 1 -- 172.21.15.12:0/3637241209 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f795404d440 con 0x7f79581065c0 2024-02-18T11:31:42.283 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:41.547+0000 7f795d4aa700 1 --2- 172.21.15.12:0/3637241209 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7f79440390c0 0x7f794403b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:31:42.283 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:41.548+0000 7f795d4aa700 1 --2- 172.21.15.12:0/3637241209 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7f79440390c0 0x7f794403b580 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7948006fd0 tx=0x7f7948006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:31:42.283 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:41.552+0000 7f794effd700 1 -- 172.21.15.12:0/3637241209 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7954018a40 con 0x7f79581065c0 2024-02-18T11:31:42.283 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:41.755+0000 7f795ff0f700 1 -- 172.21.15.12:0/3637241209 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f795810a9f0 con 0x7f79581065c0 2024-02-18T11:31:42.283 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:41.759+0000 7f794effd700 1 -- 172.21.15.12:0/3637241209 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f795404cdb0 con 0x7f79581065c0 2024-02-18T11:31:42.283 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:41.761+0000 7f795ff0f700 1 -- 172.21.15.12:0/3637241209 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7f79440390c0 msgr2=0x7f794403b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:31:42.283 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:41.761+0000 7f795ff0f700 1 --2- 172.21.15.12:0/3637241209 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7f79440390c0 0x7f794403b580 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7948006fd0 tx=0x7f7948006e40 comp rx=0 tx=0).stop 2024-02-18T11:31:42.283 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:41.761+0000 7f795ff0f700 1 -- 172.21.15.12:0/3637241209 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f79581065c0 msgr2=0x7f795819b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:31:42.283 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:41.761+0000 7f795ff0f700 1 --2- 172.21.15.12:0/3637241209 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f79581065c0 0x7f795819b570 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7954003f20 tx=0x7f7954011b00 comp rx=0 tx=0).stop 2024-02-18T11:31:42.284 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:41.761+0000 7f795ff0f700 1 -- 172.21.15.12:0/3637241209 shutdown_connections 2024-02-18T11:31:42.284 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:41.761+0000 7f795ff0f700 1 --2- 172.21.15.12:0/3637241209 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7f79440390c0 0x7f794403b580 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:31:42.284 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:41.761+0000 7f795ff0f700 1 --2- 172.21.15.12:0/3637241209 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f79581065c0 0x7f795819b570 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:31:42.284 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:41.761+0000 7f795ff0f700 1 -- 172.21.15.12:0/3637241209 >> 172.21.15.12:0/3637241209 conn(0x7f7958101bc0 msgr2=0x7f7958102ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:31:42.284 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:41.761+0000 7f795ff0f700 1 -- 172.21.15.12:0/3637241209 shutdown_connections 2024-02-18T11:31:42.284 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:41.761+0000 7f795ff0f700 1 -- 172.21.15.12:0/3637241209 wait complete. 2024-02-18T11:31:42.284 INFO:teuthology.orchestra.run.smithi012.stdout:Enabling the dashboard module... 2024-02-18T11:31:42.983 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:42 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/3637241209' entity='client.admin' 2024-02-18T11:31:44.105 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:43 smithi012 ceph-mon[32460]: from='mgr.14120 172.21.15.12:0/2151529525' entity='mgr.x' 2024-02-18T11:31:44.105 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:43 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/3552225915' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-02-18T11:31:44.105 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:43 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: ignoring --setuser ceph since I am not root 2024-02-18T11:31:44.105 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:43 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: ignoring --setgroup ceph since I am not root 2024-02-18T11:31:44.388 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:42.713+0000 7f4e74934700 1 Processor -- start 2024-02-18T11:31:44.388 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:42.713+0000 7f4e74934700 1 -- start start 2024-02-18T11:31:44.389 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:42.713+0000 7f4e74934700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e701076b0 0x7f4e70107ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:31:44.389 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:42.713+0000 7f4e74934700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e701080a0 con 0x7f4e701076b0 2024-02-18T11:31:44.389 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:42.714+0000 7f4e6e59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e701076b0 0x7f4e70107ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:31:44.389 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:42.714+0000 7f4e6e59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e701076b0 0x7f4e70107ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53524/0 (socket says 172.21.15.12:53524) 2024-02-18T11:31:44.389 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:42.714+0000 7f4e6e59c700 1 -- 172.21.15.12:0/4139735895 learned_addr learned my addr 172.21.15.12:0/4139735895 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:31:44.389 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:42.714+0000 7f4e6e59c700 1 -- 172.21.15.12:0/4139735895 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e70108860 con 0x7f4e701076b0 2024-02-18T11:31:44.389 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:42.714+0000 7f4e6e59c700 1 --2- 172.21.15.12:0/4139735895 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e701076b0 0x7f4e70107ad0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4e68009e80 tx=0x7f4e6800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40bf4ba01b000543 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:31:44.389 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:42.714+0000 7f4e6d59a700 1 -- 172.21.15.12:0/4139735895 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4e6800e070 con 0x7f4e701076b0 2024-02-18T11:31:44.389 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:42.714+0000 7f4e6d59a700 1 -- 172.21.15.12:0/4139735895 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4e6800b8c0 con 0x7f4e701076b0 2024-02-18T11:31:44.389 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:42.714+0000 7f4e6d59a700 1 -- 172.21.15.12:0/4139735895 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4e680044a0 con 0x7f4e701076b0 2024-02-18T11:31:44.389 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:42.715+0000 7f4e74934700 1 -- 172.21.15.12:0/4139735895 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e701076b0 msgr2=0x7f4e70107ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:31:44.389 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:42.715+0000 7f4e74934700 1 --2- 172.21.15.12:0/4139735895 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e701076b0 0x7f4e70107ad0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4e68009e80 tx=0x7f4e6800b330 comp rx=0 tx=0).stop 2024-02-18T11:31:44.389 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:42.715+0000 7f4e74934700 1 -- 172.21.15.12:0/4139735895 shutdown_connections 2024-02-18T11:31:44.389 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:42.715+0000 7f4e74934700 1 --2- 172.21.15.12:0/4139735895 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e701076b0 0x7f4e70107ad0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:31:44.390 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:42.715+0000 7f4e74934700 1 -- 172.21.15.12:0/4139735895 >> 172.21.15.12:0/4139735895 conn(0x7f4e70102d30 msgr2=0x7f4e70105150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:31:44.390 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:42.715+0000 7f4e74934700 1 -- 172.21.15.12:0/4139735895 shutdown_connections 2024-02-18T11:31:44.390 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:42.715+0000 7f4e74934700 1 -- 172.21.15.12:0/4139735895 wait complete. 2024-02-18T11:31:44.390 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:42.716+0000 7f4e74934700 1 Processor -- start 2024-02-18T11:31:44.390 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:42.716+0000 7f4e74934700 1 -- start start 2024-02-18T11:31:44.390 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:42.716+0000 7f4e74934700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e701076b0 0x7f4e7019b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:31:44.390 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:42.716+0000 7f4e74934700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e7019b8f0 con 0x7f4e701076b0 2024-02-18T11:31:44.390 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:42.716+0000 7f4e6e59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e701076b0 0x7f4e7019b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:31:44.390 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:42.716+0000 7f4e6e59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e701076b0 0x7f4e7019b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53530/0 (socket says 172.21.15.12:53530) 2024-02-18T11:31:44.390 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:42.716+0000 7f4e6e59c700 1 -- 172.21.15.12:0/3552225915 learned_addr learned my addr 172.21.15.12:0/3552225915 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:31:44.390 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:42.716+0000 7f4e6e59c700 1 -- 172.21.15.12:0/3552225915 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e68009b30 con 0x7f4e701076b0 2024-02-18T11:31:44.390 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:42.717+0000 7f4e6e59c700 1 --2- 172.21.15.12:0/3552225915 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e701076b0 0x7f4e7019b3b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4e68004070 tx=0x7f4e6800bd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:31:44.390 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:42.717+0000 7f4e677fe700 1 -- 172.21.15.12:0/3552225915 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4e6800e040 con 0x7f4e701076b0 2024-02-18T11:31:44.390 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:42.717+0000 7f4e677fe700 1 -- 172.21.15.12:0/3552225915 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4e68003d60 con 0x7f4e701076b0 2024-02-18T11:31:44.390 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:42.717+0000 7f4e74934700 1 -- 172.21.15.12:0/3552225915 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e7019baf0 con 0x7f4e701076b0 2024-02-18T11:31:44.391 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:42.717+0000 7f4e677fe700 1 -- 172.21.15.12:0/3552225915 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4e6801c780 con 0x7f4e701076b0 2024-02-18T11:31:44.391 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:42.717+0000 7f4e74934700 1 -- 172.21.15.12:0/3552225915 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e7019bff0 con 0x7f4e701076b0 2024-02-18T11:31:44.391 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:42.718+0000 7f4e677fe700 1 -- 172.21.15.12:0/3552225915 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 8) v1 ==== 46091+0+0 (secure 0 0 0) 0x7f4e6801c9b0 con 0x7f4e701076b0 2024-02-18T11:31:44.391 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:42.718+0000 7f4e677fe700 1 --2- 172.21.15.12:0/3552225915 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7f4e5c039110 0x7f4e5c03b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:31:44.391 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:42.718+0000 7f4e677fe700 1 -- 172.21.15.12:0/3552225915 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f4e6804d160 con 0x7f4e701076b0 2024-02-18T11:31:44.391 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:42.718+0000 7f4e74934700 1 -- 172.21.15.12:0/3552225915 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e7010ba90 con 0x7f4e701076b0 2024-02-18T11:31:44.391 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:42.718+0000 7f4e6dd9b700 1 --2- 172.21.15.12:0/3552225915 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7f4e5c039110 0x7f4e5c03b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:31:44.391 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:42.718+0000 7f4e6dd9b700 1 --2- 172.21.15.12:0/3552225915 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7f4e5c039110 0x7f4e5c03b5d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4e58006fd0 tx=0x7f4e58006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:31:44.391 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:42.723+0000 7f4e677fe700 1 -- 172.21.15.12:0/3552225915 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4e68017070 con 0x7f4e701076b0 2024-02-18T11:31:44.391 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:42.987+0000 7f4e74934700 1 -- 172.21.15.12:0/3552225915 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f4e7010bc70 con 0x7f4e701076b0 2024-02-18T11:31:44.391 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:43.798+0000 7f4e677fe700 1 -- 172.21.15.12:0/3552225915 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f4e68020070 con 0x7f4e701076b0 2024-02-18T11:31:44.391 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:43.800+0000 7f4e677fe700 1 -- 172.21.15.12:0/3552225915 <== mon.0 v2:172.21.15.12:3300/0 8 ==== mgrmap(e 9) v1 ==== 46104+0+0 (secure 0 0 0) 0x7f4e6801ccb0 con 0x7f4e701076b0 2024-02-18T11:31:44.391 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:43.801+0000 7f4e74934700 1 -- 172.21.15.12:0/3552225915 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7f4e5c039110 msgr2=0x7f4e5c03b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:31:44.391 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:43.801+0000 7f4e74934700 1 --2- 172.21.15.12:0/3552225915 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7f4e5c039110 0x7f4e5c03b5d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4e58006fd0 tx=0x7f4e58006e40 comp rx=0 tx=0).stop 2024-02-18T11:31:44.391 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:43.801+0000 7f4e74934700 1 -- 172.21.15.12:0/3552225915 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e701076b0 msgr2=0x7f4e7019b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:31:44.392 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:43.801+0000 7f4e74934700 1 --2- 172.21.15.12:0/3552225915 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e701076b0 0x7f4e7019b3b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4e68004070 tx=0x7f4e6800bd10 comp rx=0 tx=0).stop 2024-02-18T11:31:44.392 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:43.801+0000 7f4e74934700 1 -- 172.21.15.12:0/3552225915 shutdown_connections 2024-02-18T11:31:44.392 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:43.801+0000 7f4e74934700 1 --2- 172.21.15.12:0/3552225915 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7f4e5c039110 0x7f4e5c03b5d0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:31:44.392 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:43.801+0000 7f4e74934700 1 --2- 172.21.15.12:0/3552225915 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e701076b0 0x7f4e7019b3b0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:31:44.392 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:43.801+0000 7f4e74934700 1 -- 172.21.15.12:0/3552225915 >> 172.21.15.12:0/3552225915 conn(0x7f4e70102d30 msgr2=0x7f4e701037b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:31:44.392 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:43.801+0000 7f4e74934700 1 -- 172.21.15.12:0/3552225915 shutdown_connections 2024-02-18T11:31:44.392 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:43.802+0000 7f4e74934700 1 -- 172.21.15.12:0/3552225915 wait complete. 2024-02-18T11:31:44.668 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:44 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:31:44.389+0000 7fb3e7e52200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-18T11:31:44.668 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:44 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:31:44.507+0000 7fb3e7e52200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-18T11:31:44.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:44 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/3552225915' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-02-18T11:31:44.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:44 smithi012 ceph-mon[32460]: mgrmap e9: x(active, since 19s) 2024-02-18T11:31:45.412 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:45 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:31:45.176+0000 7fb3e7e52200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-18T11:31:45.412 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:45 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:31:45.271+0000 7fb3e7e52200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-18T11:31:45.412 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:45 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:31:45.416+0000 7fb3e7e52200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-18T11:31:45.674 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:45 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:31:45.515+0000 7fb3e7e52200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-18T11:31:45.674 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:45 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:31:45.600+0000 7fb3e7e52200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-18T11:31:45.695 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout { 2024-02-18T11:31:45.695 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-02-18T11:31:45.695 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-18T11:31:45.695 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-02-18T11:31:45.695 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-02-18T11:31:45.695 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout } 2024-02-18T11:31:45.695 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:44.872+0000 7f37704de700 1 Processor -- start 2024-02-18T11:31:45.695 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:44.872+0000 7f37704de700 1 -- start start 2024-02-18T11:31:45.696 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:44.872+0000 7f37704de700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3768107730 0x7f3768107b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:31:45.696 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:44.872+0000 7f37704de700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3768108120 con 0x7f3768107730 2024-02-18T11:31:45.696 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:44.873+0000 7f376e27a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3768107730 0x7f3768107b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:31:45.696 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:44.873+0000 7f376e27a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3768107730 0x7f3768107b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53554/0 (socket says 172.21.15.12:53554) 2024-02-18T11:31:45.696 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:44.873+0000 7f376e27a700 1 -- 172.21.15.12:0/1875937254 learned_addr learned my addr 172.21.15.12:0/1875937254 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:31:45.696 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:44.873+0000 7f376e27a700 1 -- 172.21.15.12:0/1875937254 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37681088e0 con 0x7f3768107730 2024-02-18T11:31:45.696 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:44.873+0000 7f376e27a700 1 --2- 172.21.15.12:0/1875937254 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3768107730 0x7f3768107b50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f375c0098a0 tx=0x7f375c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd965ff400950c50 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:31:45.696 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:44.873+0000 7f376d278700 1 -- 172.21.15.12:0/1875937254 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f375c004160 con 0x7f3768107730 2024-02-18T11:31:45.696 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:44.873+0000 7f376d278700 1 -- 172.21.15.12:0/1875937254 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f375c0042c0 con 0x7f3768107730 2024-02-18T11:31:45.696 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:44.873+0000 7f376d278700 1 -- 172.21.15.12:0/1875937254 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f375c011a80 con 0x7f3768107730 2024-02-18T11:31:45.696 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:44.874+0000 7f37704de700 1 -- 172.21.15.12:0/1875937254 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3768107730 msgr2=0x7f3768107b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:31:45.696 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:44.874+0000 7f37704de700 1 --2- 172.21.15.12:0/1875937254 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3768107730 0x7f3768107b50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f375c0098a0 tx=0x7f375c009bb0 comp rx=0 tx=0).stop 2024-02-18T11:31:45.696 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:44.874+0000 7f37704de700 1 -- 172.21.15.12:0/1875937254 shutdown_connections 2024-02-18T11:31:45.696 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:44.874+0000 7f37704de700 1 --2- 172.21.15.12:0/1875937254 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3768107730 0x7f3768107b50 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:31:45.697 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:44.874+0000 7f37704de700 1 -- 172.21.15.12:0/1875937254 >> 172.21.15.12:0/1875937254 conn(0x7f3768102d90 msgr2=0x7f37681051b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:31:45.697 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:44.874+0000 7f37704de700 1 -- 172.21.15.12:0/1875937254 shutdown_connections 2024-02-18T11:31:45.697 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:44.874+0000 7f37704de700 1 -- 172.21.15.12:0/1875937254 wait complete. 2024-02-18T11:31:45.697 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:44.875+0000 7f37704de700 1 Processor -- start 2024-02-18T11:31:45.697 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:44.875+0000 7f37704de700 1 -- start start 2024-02-18T11:31:45.697 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:44.875+0000 7f37704de700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3768107730 0x7f376807cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:31:45.697 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:44.875+0000 7f37704de700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f376807d0c0 con 0x7f3768107730 2024-02-18T11:31:45.697 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:44.875+0000 7f376e27a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3768107730 0x7f376807cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:31:45.697 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:44.875+0000 7f376e27a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3768107730 0x7f376807cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53570/0 (socket says 172.21.15.12:53570) 2024-02-18T11:31:45.697 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:44.875+0000 7f376e27a700 1 -- 172.21.15.12:0/33829598 learned_addr learned my addr 172.21.15.12:0/33829598 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:31:45.697 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:44.875+0000 7f376e27a700 1 -- 172.21.15.12:0/33829598 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f375c009550 con 0x7f3768107730 2024-02-18T11:31:45.697 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:44.876+0000 7f376e27a700 1 --2- 172.21.15.12:0/33829598 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3768107730 0x7f376807cb80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f375c000c00 tx=0x7f375c011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:31:45.697 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:44.876+0000 7f375b7fe700 1 -- 172.21.15.12:0/33829598 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f375c003aa0 con 0x7f3768107730 2024-02-18T11:31:45.698 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:44.876+0000 7f37704de700 1 -- 172.21.15.12:0/33829598 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f376807b2b0 con 0x7f3768107730 2024-02-18T11:31:45.698 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:44.876+0000 7f375b7fe700 1 -- 172.21.15.12:0/33829598 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f375c003c00 con 0x7f3768107730 2024-02-18T11:31:45.698 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:44.876+0000 7f375b7fe700 1 -- 172.21.15.12:0/33829598 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f375c024930 con 0x7f3768107730 2024-02-18T11:31:45.698 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:44.876+0000 7f37704de700 1 -- 172.21.15.12:0/33829598 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f376807b750 con 0x7f3768107730 2024-02-18T11:31:45.698 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:44.877+0000 7f37704de700 1 -- 172.21.15.12:0/33829598 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f376810bb10 con 0x7f3768107730 2024-02-18T11:31:45.698 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:44.877+0000 7f375b7fe700 1 -- 172.21.15.12:0/33829598 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 9) v1 ==== 46104+0+0 (secure 0 0 0) 0x7f375c02d3f0 con 0x7f3768107730 2024-02-18T11:31:45.698 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:44.877+0000 7f375b7fe700 1 --2- 172.21.15.12:0/33829598 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7f37540391c0 0x7f375403b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:31:45.698 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:44.877+0000 7f375b7fe700 1 -- 172.21.15.12:0/33829598 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f375c04d3e0 con 0x7f3768107730 2024-02-18T11:31:45.698 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:44.877+0000 7f376da79700 1 -- 172.21.15.12:0/33829598 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7f37540391c0 msgr2=0x7f375403b680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/4135274411 2024-02-18T11:31:45.698 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:44.877+0000 7f376da79700 1 --2- 172.21.15.12:0/33829598 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7f37540391c0 0x7f375403b680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-18T11:31:45.698 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:44.882+0000 7f375b7fe700 1 -- 172.21.15.12:0/33829598 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f375c010740 con 0x7f3768107730 2024-02-18T11:31:45.698 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:45.078+0000 7f376da79700 1 -- 172.21.15.12:0/33829598 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7f37540391c0 msgr2=0x7f375403b680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/4135274411 2024-02-18T11:31:45.698 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:45.078+0000 7f376da79700 1 --2- 172.21.15.12:0/33829598 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7f37540391c0 0x7f375403b680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-18T11:31:45.698 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:45.142+0000 7f37704de700 1 -- 172.21.15.12:0/33829598 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f376807bc90 con 0x7f3768107730 2024-02-18T11:31:45.698 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:45.142+0000 7f375b7fe700 1 -- 172.21.15.12:0/33829598 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7f375c010380 con 0x7f3768107730 2024-02-18T11:31:45.699 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:45.144+0000 7f37704de700 1 -- 172.21.15.12:0/33829598 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7f37540391c0 msgr2=0x7f375403b680 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-18T11:31:45.699 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:45.144+0000 7f37704de700 1 --2- 172.21.15.12:0/33829598 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7f37540391c0 0x7f375403b680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:31:45.699 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:45.144+0000 7f37704de700 1 -- 172.21.15.12:0/33829598 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3768107730 msgr2=0x7f376807cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:31:45.699 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:45.144+0000 7f37704de700 1 --2- 172.21.15.12:0/33829598 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3768107730 0x7f376807cb80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f375c000c00 tx=0x7f375c011b00 comp rx=0 tx=0).stop 2024-02-18T11:31:45.699 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:45.144+0000 7f37704de700 1 -- 172.21.15.12:0/33829598 shutdown_connections 2024-02-18T11:31:45.699 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:45.144+0000 7f37704de700 1 --2- 172.21.15.12:0/33829598 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7f37540391c0 0x7f375403b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:31:45.699 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:45.144+0000 7f37704de700 1 --2- 172.21.15.12:0/33829598 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3768107730 0x7f376807cb80 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:31:45.699 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:45.144+0000 7f37704de700 1 -- 172.21.15.12:0/33829598 >> 172.21.15.12:0/33829598 conn(0x7f3768102d90 msgr2=0x7f3768103880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:31:45.699 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:45.144+0000 7f37704de700 1 -- 172.21.15.12:0/33829598 shutdown_connections 2024-02-18T11:31:45.699 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:45.144+0000 7f37704de700 1 -- 172.21.15.12:0/33829598 wait complete. 2024-02-18T11:31:45.699 INFO:teuthology.orchestra.run.smithi012.stdout:Waiting for the mgr to restart... 2024-02-18T11:31:45.699 INFO:teuthology.orchestra.run.smithi012.stdout:Waiting for mgr epoch 9... 2024-02-18T11:31:45.988 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:45 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/33829598' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-18T11:31:45.988 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:45 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:31:45.878+0000 7fb3e7e52200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-18T11:31:45.988 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:45 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:31:45.992+0000 7fb3e7e52200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-18T11:31:46.297 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:46 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:31:46.083+0000 7fb3e7e52200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-18T11:31:46.298 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:46 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:31:46.181+0000 7fb3e7e52200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-18T11:31:46.674 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:46 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:31:46.361+0000 7fb3e7e52200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-18T11:31:47.674 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:47 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:31:47.348+0000 7fb3e7e52200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-18T11:31:47.674 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:47 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:31:47.482+0000 7fb3e7e52200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-18T11:31:48.117 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:47 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:31:47.821+0000 7fb3e7e52200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-18T11:31:48.117 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:47 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:31:47.913+0000 7fb3e7e52200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-18T11:31:48.117 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:48 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:31:48.121+0000 7fb3e7e52200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-18T11:31:48.424 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:48 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:31:48.317+0000 7fb3e7e52200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-18T11:31:48.891 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:48 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:31:48.616+0000 7fb3e7e52200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-18T11:31:48.891 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:48 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:31:48.895+0000 7fb3e7e52200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-18T11:31:49.174 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:49 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:31:49.141+0000 7fb3e7e52200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-18T11:31:50.424 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:49 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:31:49.980+0000 7fb3e7e52200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-18T11:31:50.919 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:50 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:31:50.597+0000 7fb3e7e52200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-18T11:31:50.919 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:50 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:31:50.698+0000 7fb3e7e52200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-18T11:31:50.919 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:31:50 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:31:50.923+0000 7fb3e7e52200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-18T11:31:51.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:50 smithi012 ceph-mon[32460]: Active manager daemon x restarted 2024-02-18T11:31:51.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:50 smithi012 ceph-mon[32460]: Activating manager daemon x 2024-02-18T11:31:51.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:50 smithi012 ceph-mon[32460]: osdmap e3: 0 total, 0 up, 0 in 2024-02-18T11:31:51.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:50 smithi012 ceph-mon[32460]: mgrmap e10: x(active, starting, since 0.00353883s) 2024-02-18T11:31:51.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:50 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-18T11:31:51.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:50 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-18T11:31:51.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:50 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-18T11:31:51.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:50 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-18T11:31:51.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:50 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-18T11:31:51.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:50 smithi012 ceph-mon[32460]: Manager daemon x is now available 2024-02-18T11:31:51.176 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:50 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:31:51.176 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:50 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:31:52.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:51 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-02-18T11:31:52.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:51 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-02-18T11:31:52.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:51 smithi012 ceph-mon[32460]: [18/Feb/2024:11:31:51] ENGINE Bus STARTING 2024-02-18T11:31:52.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:51 smithi012 ceph-mon[32460]: [18/Feb/2024:11:31:51] ENGINE Serving on https://172.21.15.12:7150 2024-02-18T11:31:52.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:51 smithi012 ceph-mon[32460]: [18/Feb/2024:11:31:51] ENGINE Serving on http://172.21.15.12:8765 2024-02-18T11:31:52.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:51 smithi012 ceph-mon[32460]: [18/Feb/2024:11:31:51] ENGINE Bus STARTED 2024-02-18T11:31:52.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:51 smithi012 ceph-mon[32460]: mgrmap e11: x(active, since 1.00603s) 2024-02-18T11:31:52.782 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout { 2024-02-18T11:31:52.783 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-02-18T11:31:52.783 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "initialized": true 2024-02-18T11:31:52.783 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout } 2024-02-18T11:31:52.783 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:46.096+0000 7f3e5301c700 1 Processor -- start 2024-02-18T11:31:52.783 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:46.096+0000 7f3e5301c700 1 -- start start 2024-02-18T11:31:52.783 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:46.096+0000 7f3e5301c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e4c107b00 0x7f3e4c107f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:31:52.783 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:46.096+0000 7f3e5301c700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4c1084f0 con 0x7f3e4c107b00 2024-02-18T11:31:52.783 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:46.096+0000 7f3e50db8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e4c107b00 0x7f3e4c107f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:31:52.783 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:46.096+0000 7f3e50db8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e4c107b00 0x7f3e4c107f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53580/0 (socket says 172.21.15.12:53580) 2024-02-18T11:31:52.783 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:46.096+0000 7f3e50db8700 1 -- 172.21.15.12:0/3466508717 learned_addr learned my addr 172.21.15.12:0/3466508717 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:31:52.783 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:46.096+0000 7f3e50db8700 1 -- 172.21.15.12:0/3466508717 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e4c106760 con 0x7f3e4c107b00 2024-02-18T11:31:52.783 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:46.097+0000 7f3e50db8700 1 --2- 172.21.15.12:0/3466508717 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e4c107b00 0x7f3e4c107f20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3e400098a0 tx=0x7f3e40009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c675b7891af584a9 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:31:52.783 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:46.097+0000 7f3e4b7fe700 1 -- 172.21.15.12:0/3466508717 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3e40004160 con 0x7f3e4c107b00 2024-02-18T11:31:52.784 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:46.097+0000 7f3e4b7fe700 1 -- 172.21.15.12:0/3466508717 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3e400042c0 con 0x7f3e4c107b00 2024-02-18T11:31:52.784 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:46.097+0000 7f3e4b7fe700 1 -- 172.21.15.12:0/3466508717 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3e40011a80 con 0x7f3e4c107b00 2024-02-18T11:31:52.784 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:46.097+0000 7f3e5301c700 1 -- 172.21.15.12:0/3466508717 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e4c107b00 msgr2=0x7f3e4c107f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:31:52.784 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:46.097+0000 7f3e5301c700 1 --2- 172.21.15.12:0/3466508717 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e4c107b00 0x7f3e4c107f20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3e400098a0 tx=0x7f3e40009bb0 comp rx=0 tx=0).stop 2024-02-18T11:31:52.784 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:46.098+0000 7f3e5301c700 1 -- 172.21.15.12:0/3466508717 shutdown_connections 2024-02-18T11:31:52.784 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:46.098+0000 7f3e5301c700 1 --2- 172.21.15.12:0/3466508717 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e4c107b00 0x7f3e4c107f20 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:31:52.784 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:46.098+0000 7f3e5301c700 1 -- 172.21.15.12:0/3466508717 >> 172.21.15.12:0/3466508717 conn(0x7f3e4c101330 msgr2=0x7f3e4c103750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:31:52.784 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:46.098+0000 7f3e5301c700 1 -- 172.21.15.12:0/3466508717 shutdown_connections 2024-02-18T11:31:52.784 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:46.098+0000 7f3e5301c700 1 -- 172.21.15.12:0/3466508717 wait complete. 2024-02-18T11:31:52.784 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:46.098+0000 7f3e5301c700 1 Processor -- start 2024-02-18T11:31:52.784 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:46.098+0000 7f3e5301c700 1 -- start start 2024-02-18T11:31:52.784 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:46.098+0000 7f3e5301c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e4c107b00 0x7f3e4c19b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:31:52.784 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:46.098+0000 7f3e5301c700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4c19b900 con 0x7f3e4c107b00 2024-02-18T11:31:52.785 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:46.099+0000 7f3e50db8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e4c107b00 0x7f3e4c19b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:31:52.785 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:46.099+0000 7f3e50db8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e4c107b00 0x7f3e4c19b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53586/0 (socket says 172.21.15.12:53586) 2024-02-18T11:31:52.785 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:46.099+0000 7f3e50db8700 1 -- 172.21.15.12:0/954820696 learned_addr learned my addr 172.21.15.12:0/954820696 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:31:52.785 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:46.099+0000 7f3e50db8700 1 -- 172.21.15.12:0/954820696 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e40009550 con 0x7f3e4c107b00 2024-02-18T11:31:52.785 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:46.099+0000 7f3e50db8700 1 --2- 172.21.15.12:0/954820696 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e4c107b00 0x7f3e4c19b3c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3e40011740 tx=0x7f3e40011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:31:52.785 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:46.099+0000 7f3e49ffb700 1 -- 172.21.15.12:0/954820696 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3e40003b20 con 0x7f3e4c107b00 2024-02-18T11:31:52.785 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:46.099+0000 7f3e49ffb700 1 -- 172.21.15.12:0/954820696 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3e4001a910 con 0x7f3e4c107b00 2024-02-18T11:31:52.785 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:46.099+0000 7f3e5301c700 1 -- 172.21.15.12:0/954820696 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e4c19bb00 con 0x7f3e4c107b00 2024-02-18T11:31:52.785 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:46.099+0000 7f3e49ffb700 1 -- 172.21.15.12:0/954820696 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3e40024900 con 0x7f3e4c107b00 2024-02-18T11:31:52.785 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:46.099+0000 7f3e5301c700 1 -- 172.21.15.12:0/954820696 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e4c19bfa0 con 0x7f3e4c107b00 2024-02-18T11:31:52.785 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:46.100+0000 7f3e49ffb700 1 -- 172.21.15.12:0/954820696 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 9) v1 ==== 46104+0+0 (secure 0 0 0) 0x7f3e40024a60 con 0x7f3e4c107b00 2024-02-18T11:31:52.785 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:46.100+0000 7f3e49ffb700 1 --2- 172.21.15.12:0/954820696 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7f3e3c039170 0x7f3e3c03b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:31:52.785 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:46.100+0000 7f3e377fe700 1 -- 172.21.15.12:0/954820696 --> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3e4c10a9a0 con 0x7f3e3c039170 2024-02-18T11:31:52.785 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:46.100+0000 7f3e49ffb700 1 -- 172.21.15.12:0/954820696 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f3e4004e650 con 0x7f3e4c107b00 2024-02-18T11:31:52.786 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:46.100+0000 7f3e4bfff700 1 -- 172.21.15.12:0/954820696 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7f3e3c039170 msgr2=0x7f3e3c03b630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/4135274411 2024-02-18T11:31:52.786 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:46.100+0000 7f3e4bfff700 1 --2- 172.21.15.12:0/954820696 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7f3e3c039170 0x7f3e3c03b630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-18T11:31:52.786 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:46.302+0000 7f3e4bfff700 1 -- 172.21.15.12:0/954820696 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7f3e3c039170 msgr2=0x7f3e3c03b630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/4135274411 2024-02-18T11:31:52.786 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:46.302+0000 7f3e4bfff700 1 --2- 172.21.15.12:0/954820696 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7f3e3c039170 0x7f3e3c03b630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-18T11:31:52.786 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:46.703+0000 7f3e4bfff700 1 -- 172.21.15.12:0/954820696 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7f3e3c039170 msgr2=0x7f3e3c03b630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/4135274411 2024-02-18T11:31:52.786 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:46.703+0000 7f3e4bfff700 1 --2- 172.21.15.12:0/954820696 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7f3e3c039170 0x7f3e3c03b630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-02-18T11:31:52.786 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:47.504+0000 7f3e4bfff700 1 -- 172.21.15.12:0/954820696 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7f3e3c039170 msgr2=0x7f3e3c03b630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/4135274411 2024-02-18T11:31:52.786 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:47.504+0000 7f3e4bfff700 1 --2- 172.21.15.12:0/954820696 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7f3e3c039170 0x7f3e3c03b630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-02-18T11:31:52.786 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:49.105+0000 7f3e4bfff700 1 -- 172.21.15.12:0/954820696 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7f3e3c039170 msgr2=0x7f3e3c03b630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/4135274411 2024-02-18T11:31:52.786 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:49.105+0000 7f3e4bfff700 1 --2- 172.21.15.12:0/954820696 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7f3e3c039170 0x7f3e3c03b630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-02-18T11:31:52.786 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:50.938+0000 7f3e49ffb700 1 -- 172.21.15.12:0/954820696 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mgrmap(e 10) v1 ==== 45871+0+0 (secure 0 0 0) 0x7f3e4002dc90 con 0x7f3e4c107b00 2024-02-18T11:31:52.786 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:50.938+0000 7f3e49ffb700 1 -- 172.21.15.12:0/954820696 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7f3e3c039170 msgr2=0x7f3e3c03b630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-18T11:31:52.786 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:50.938+0000 7f3e49ffb700 1 --2- 172.21.15.12:0/954820696 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7f3e3c039170 0x7f3e3c03b630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:31:52.786 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:51.939+0000 7f3e49ffb700 1 -- 172.21.15.12:0/954820696 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mgrmap(e 11) v1 ==== 45998+0+0 (secure 0 0 0) 0x7f3e4004f240 con 0x7f3e4c107b00 2024-02-18T11:31:52.786 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:51.940+0000 7f3e49ffb700 1 --2- 172.21.15.12:0/954820696 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f3e3c03ca70 0x7f3e3c03ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:31:52.787 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:51.940+0000 7f3e49ffb700 1 -- 172.21.15.12:0/954820696 --> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3e4c10a9a0 con 0x7f3e3c03ca70 2024-02-18T11:31:52.787 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:51.940+0000 7f3e4bfff700 1 --2- 172.21.15.12:0/954820696 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f3e3c03ca70 0x7f3e3c03ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:31:52.787 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:51.941+0000 7f3e4bfff700 1 --2- 172.21.15.12:0/954820696 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f3e3c03ca70 0x7f3e3c03ee60 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f3e38003a10 tx=0x7f3e38009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:31:52.787 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:51.941+0000 7f3e49ffb700 1 -- 172.21.15.12:0/954820696 <== mgr.14152 v2:172.21.15.12:6800/1153736685 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7f3e4c10a9a0 con 0x7f3e3c03ca70 2024-02-18T11:31:52.787 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:51.948+0000 7f3e377fe700 1 -- 172.21.15.12:0/954820696 --> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f3e4c10ab10 con 0x7f3e3c03ca70 2024-02-18T11:31:52.787 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:51.948+0000 7f3e49ffb700 1 -- 172.21.15.12:0/954820696 <== mgr.14152 v2:172.21.15.12:6800/1153736685 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f3e4c10ab10 con 0x7f3e3c03ca70 2024-02-18T11:31:52.787 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:51.949+0000 7f3e377fe700 1 -- 172.21.15.12:0/954820696 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f3e3c03ca70 msgr2=0x7f3e3c03ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:31:52.787 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:51.949+0000 7f3e377fe700 1 --2- 172.21.15.12:0/954820696 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f3e3c03ca70 0x7f3e3c03ee60 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f3e38003a10 tx=0x7f3e38009270 comp rx=0 tx=0).stop 2024-02-18T11:31:52.787 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:51.949+0000 7f3e377fe700 1 -- 172.21.15.12:0/954820696 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e4c107b00 msgr2=0x7f3e4c19b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:31:52.787 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:51.949+0000 7f3e377fe700 1 --2- 172.21.15.12:0/954820696 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e4c107b00 0x7f3e4c19b3c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3e40011740 tx=0x7f3e40011a50 comp rx=0 tx=0).stop 2024-02-18T11:31:52.787 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:51.949+0000 7f3e377fe700 1 -- 172.21.15.12:0/954820696 shutdown_connections 2024-02-18T11:31:52.788 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:51.949+0000 7f3e377fe700 1 --2- 172.21.15.12:0/954820696 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f3e3c03ca70 0x7f3e3c03ee60 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:31:52.788 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:51.949+0000 7f3e377fe700 1 --2- 172.21.15.12:0/954820696 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e4c107b00 0x7f3e4c19b3c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:31:52.788 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:51.949+0000 7f3e377fe700 1 --2- 172.21.15.12:0/954820696 >> [v2:172.21.15.12:6800/4135274411,v1:172.21.15.12:6801/4135274411] conn(0x7f3e3c039170 0x7f3e3c03b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:31:52.788 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:51.949+0000 7f3e377fe700 1 -- 172.21.15.12:0/954820696 >> 172.21.15.12:0/954820696 conn(0x7f3e4c101330 msgr2=0x7f3e4c102130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:31:52.788 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:51.949+0000 7f3e377fe700 1 -- 172.21.15.12:0/954820696 shutdown_connections 2024-02-18T11:31:52.788 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:51.949+0000 7f3e377fe700 1 -- 172.21.15.12:0/954820696 wait complete. 2024-02-18T11:31:52.788 INFO:teuthology.orchestra.run.smithi012.stdout:mgr epoch 9 is available 2024-02-18T11:31:52.788 INFO:teuthology.orchestra.run.smithi012.stdout:Generating a dashboard self-signed certificate... 2024-02-18T11:31:53.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:52 smithi012 ceph-mon[32460]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-02-18T11:31:53.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:52 smithi012 ceph-mon[32460]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-02-18T11:31:54.111 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-02-18T11:31:54.112 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:53.222+0000 7f9c7d3e2700 1 Processor -- start 2024-02-18T11:31:54.112 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:53.222+0000 7f9c7d3e2700 1 -- start start 2024-02-18T11:31:54.112 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:53.222+0000 7f9c7d3e2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c78108180 0x7f9c7810a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:31:54.112 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:53.222+0000 7f9c7d3e2700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c78075730 con 0x7f9c78108180 2024-02-18T11:31:54.112 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:53.222+0000 7f9c76ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c78108180 0x7f9c7810a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:31:54.112 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:53.222+0000 7f9c76ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c78108180 0x7f9c7810a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52652/0 (socket says 172.21.15.12:52652) 2024-02-18T11:31:54.112 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:53.222+0000 7f9c76ffd700 1 -- 172.21.15.12:0/3253062395 learned_addr learned my addr 172.21.15.12:0/3253062395 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:31:54.112 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:53.223+0000 7f9c76ffd700 1 -- 172.21.15.12:0/3253062395 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c7810aab0 con 0x7f9c78108180 2024-02-18T11:31:54.112 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:53.223+0000 7f9c76ffd700 1 --2- 172.21.15.12:0/3253062395 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c78108180 0x7f9c7810a570 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9c680098a0 tx=0x7f9c68009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=785f9566351666d9 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:31:54.112 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:53.223+0000 7f9c767fc700 1 -- 172.21.15.12:0/3253062395 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9c68004160 con 0x7f9c78108180 2024-02-18T11:31:54.112 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:53.223+0000 7f9c767fc700 1 -- 172.21.15.12:0/3253062395 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9c680042c0 con 0x7f9c78108180 2024-02-18T11:31:54.112 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:53.223+0000 7f9c767fc700 1 -- 172.21.15.12:0/3253062395 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9c68011a80 con 0x7f9c78108180 2024-02-18T11:31:54.112 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:53.224+0000 7f9c7d3e2700 1 -- 172.21.15.12:0/3253062395 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c78108180 msgr2=0x7f9c7810a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:31:54.113 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:53.224+0000 7f9c7d3e2700 1 --2- 172.21.15.12:0/3253062395 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c78108180 0x7f9c7810a570 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9c680098a0 tx=0x7f9c68009bb0 comp rx=0 tx=0).stop 2024-02-18T11:31:54.113 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:53.224+0000 7f9c7d3e2700 1 -- 172.21.15.12:0/3253062395 shutdown_connections 2024-02-18T11:31:54.113 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:53.224+0000 7f9c7d3e2700 1 --2- 172.21.15.12:0/3253062395 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c78108180 0x7f9c7810a570 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:31:54.113 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:53.224+0000 7f9c7d3e2700 1 -- 172.21.15.12:0/3253062395 >> 172.21.15.12:0/3253062395 conn(0x7f9c78101c10 msgr2=0x7f9c78104030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:31:54.113 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:53.224+0000 7f9c7d3e2700 1 -- 172.21.15.12:0/3253062395 shutdown_connections 2024-02-18T11:31:54.113 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:53.224+0000 7f9c7d3e2700 1 -- 172.21.15.12:0/3253062395 wait complete. 2024-02-18T11:31:54.113 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:53.224+0000 7f9c7d3e2700 1 Processor -- start 2024-02-18T11:31:54.113 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:53.225+0000 7f9c7d3e2700 1 -- start start 2024-02-18T11:31:54.113 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:53.225+0000 7f9c7d3e2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c78108180 0x7f9c7819d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:31:54.113 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:53.225+0000 7f9c7d3e2700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c7819dbf0 con 0x7f9c78108180 2024-02-18T11:31:54.113 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:53.225+0000 7f9c76ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c78108180 0x7f9c7819d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:31:54.113 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:53.225+0000 7f9c76ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c78108180 0x7f9c7819d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52658/0 (socket says 172.21.15.12:52658) 2024-02-18T11:31:54.113 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:53.225+0000 7f9c76ffd700 1 -- 172.21.15.12:0/58535223 learned_addr learned my addr 172.21.15.12:0/58535223 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:31:54.113 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:53.225+0000 7f9c76ffd700 1 -- 172.21.15.12:0/58535223 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c68009550 con 0x7f9c78108180 2024-02-18T11:31:54.114 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:53.225+0000 7f9c76ffd700 1 --2- 172.21.15.12:0/58535223 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c78108180 0x7f9c7819d6b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9c68003f20 tx=0x7f9c68011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:31:54.114 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:53.225+0000 7f9c74ff9700 1 -- 172.21.15.12:0/58535223 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9c680039f0 con 0x7f9c78108180 2024-02-18T11:31:54.114 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:53.225+0000 7f9c74ff9700 1 -- 172.21.15.12:0/58535223 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9c68003b50 con 0x7f9c78108180 2024-02-18T11:31:54.114 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:53.225+0000 7f9c74ff9700 1 -- 172.21.15.12:0/58535223 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9c6801a6d0 con 0x7f9c78108180 2024-02-18T11:31:54.114 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:53.225+0000 7f9c7d3e2700 1 -- 172.21.15.12:0/58535223 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c7819ddf0 con 0x7f9c78108180 2024-02-18T11:31:54.114 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:53.226+0000 7f9c7d3e2700 1 -- 172.21.15.12:0/58535223 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c7819e290 con 0x7f9c78108180 2024-02-18T11:31:54.114 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:53.226+0000 7f9c74ff9700 1 -- 172.21.15.12:0/58535223 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 12) v1 ==== 46104+0+0 (secure 0 0 0) 0x7f9c6801a940 con 0x7f9c78108180 2024-02-18T11:31:54.114 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:53.226+0000 7f9c7d3e2700 1 -- 172.21.15.12:0/58535223 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c7810b8f0 con 0x7f9c78108180 2024-02-18T11:31:54.114 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:53.227+0000 7f9c74ff9700 1 --2- 172.21.15.12:0/58535223 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f9c64039170 0x7f9c6403b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:31:54.114 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:53.227+0000 7f9c74ff9700 1 -- 172.21.15.12:0/58535223 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f9c6804d7a0 con 0x7f9c78108180 2024-02-18T11:31:54.114 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:53.227+0000 7f9c6e7fc700 1 --2- 172.21.15.12:0/58535223 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f9c64039170 0x7f9c6403b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:31:54.114 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:53.227+0000 7f9c6e7fc700 1 --2- 172.21.15.12:0/58535223 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f9c64039170 0x7f9c6403b630 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9c60006fd0 tx=0x7f9c60006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:31:54.114 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:53.232+0000 7f9c74ff9700 1 -- 172.21.15.12:0/58535223 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9c68018dc0 con 0x7f9c78108180 2024-02-18T11:31:54.115 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:53.446+0000 7f9c7d3e2700 1 -- 172.21.15.12:0/58535223 --> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f9c781a0bd0 con 0x7f9c64039170 2024-02-18T11:31:54.115 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:53.560+0000 7f9c74ff9700 1 -- 172.21.15.12:0/58535223 <== mgr.14152 v2:172.21.15.12:6800/1153736685 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f9c781a0bd0 con 0x7f9c64039170 2024-02-18T11:31:54.115 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:53.562+0000 7f9c7d3e2700 1 -- 172.21.15.12:0/58535223 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f9c64039170 msgr2=0x7f9c6403b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:31:54.115 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:53.562+0000 7f9c7d3e2700 1 --2- 172.21.15.12:0/58535223 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f9c64039170 0x7f9c6403b630 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9c60006fd0 tx=0x7f9c60006e40 comp rx=0 tx=0).stop 2024-02-18T11:31:54.115 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:53.562+0000 7f9c7d3e2700 1 -- 172.21.15.12:0/58535223 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c78108180 msgr2=0x7f9c7819d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:31:54.115 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:53.562+0000 7f9c7d3e2700 1 --2- 172.21.15.12:0/58535223 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c78108180 0x7f9c7819d6b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9c68003f20 tx=0x7f9c68011b00 comp rx=0 tx=0).stop 2024-02-18T11:31:54.115 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:53.562+0000 7f9c7d3e2700 1 -- 172.21.15.12:0/58535223 shutdown_connections 2024-02-18T11:31:54.115 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:53.562+0000 7f9c7d3e2700 1 --2- 172.21.15.12:0/58535223 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f9c64039170 0x7f9c6403b630 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:31:54.115 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:53.562+0000 7f9c7d3e2700 1 --2- 172.21.15.12:0/58535223 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c78108180 0x7f9c7819d6b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:31:54.115 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:53.562+0000 7f9c7d3e2700 1 -- 172.21.15.12:0/58535223 >> 172.21.15.12:0/58535223 conn(0x7f9c78101c10 msgr2=0x7f9c78102680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:31:54.115 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:53.562+0000 7f9c7d3e2700 1 -- 172.21.15.12:0/58535223 shutdown_connections 2024-02-18T11:31:54.115 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:53.562+0000 7f9c7d3e2700 1 -- 172.21.15.12:0/58535223 wait complete. 2024-02-18T11:31:54.115 INFO:teuthology.orchestra.run.smithi012.stdout:Creating initial admin user... 2024-02-18T11:31:54.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:53 smithi012 ceph-mon[32460]: mgrmap e12: x(active, since 2s) 2024-02-18T11:31:54.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:53 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:31:54.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:53 smithi012 ceph-mon[32460]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:31:54.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:53 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:31:54.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:53 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:31:55.640 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:55 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:31:55.640 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:55 smithi012 ceph-mon[32460]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:31:55.640 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:55 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:31:55.648 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$UgQzSy46W2Xdbb2MebQItuBltQVtpkimVOaW/LRfmNuy8hW5FgMem", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1708255915, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-02-18T11:31:55.648 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:54.576+0000 7fd723cef700 1 Processor -- start 2024-02-18T11:31:55.648 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:54.576+0000 7fd723cef700 1 -- start start 2024-02-18T11:31:55.648 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:54.576+0000 7fd723cef700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd71c1065d0 0x7fd71c1069f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:31:55.648 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:54.576+0000 7fd723cef700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd71c106fc0 con 0x7fd71c1065d0 2024-02-18T11:31:55.649 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:54.577+0000 7fd721a8b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd71c1065d0 0x7fd71c1069f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:31:55.649 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:54.577+0000 7fd721a8b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd71c1065d0 0x7fd71c1069f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52664/0 (socket says 172.21.15.12:52664) 2024-02-18T11:31:55.649 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:54.577+0000 7fd721a8b700 1 -- 172.21.15.12:0/3623134307 learned_addr learned my addr 172.21.15.12:0/3623134307 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:31:55.649 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:54.577+0000 7fd721a8b700 1 -- 172.21.15.12:0/3623134307 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd71c1077d0 con 0x7fd71c1065d0 2024-02-18T11:31:55.649 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:54.577+0000 7fd721a8b700 1 --2- 172.21.15.12:0/3623134307 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd71c1065d0 0x7fd71c1069f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd70c0098a0 tx=0x7fd70c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2ca64603fe23093 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:31:55.649 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:54.577+0000 7fd720a89700 1 -- 172.21.15.12:0/3623134307 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd70c004160 con 0x7fd71c1065d0 2024-02-18T11:31:55.649 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:54.577+0000 7fd720a89700 1 -- 172.21.15.12:0/3623134307 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd70c0042c0 con 0x7fd71c1065d0 2024-02-18T11:31:55.649 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:54.577+0000 7fd720a89700 1 -- 172.21.15.12:0/3623134307 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd70c011a80 con 0x7fd71c1065d0 2024-02-18T11:31:55.649 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:54.578+0000 7fd723cef700 1 -- 172.21.15.12:0/3623134307 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd71c1065d0 msgr2=0x7fd71c1069f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:31:55.649 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:54.578+0000 7fd723cef700 1 --2- 172.21.15.12:0/3623134307 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd71c1065d0 0x7fd71c1069f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd70c0098a0 tx=0x7fd70c009bb0 comp rx=0 tx=0).stop 2024-02-18T11:31:55.649 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:54.578+0000 7fd723cef700 1 -- 172.21.15.12:0/3623134307 shutdown_connections 2024-02-18T11:31:55.649 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:54.578+0000 7fd723cef700 1 --2- 172.21.15.12:0/3623134307 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd71c1065d0 0x7fd71c1069f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:31:55.649 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:54.578+0000 7fd723cef700 1 -- 172.21.15.12:0/3623134307 >> 172.21.15.12:0/3623134307 conn(0x7fd71c101c10 msgr2=0x7fd71c104030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:31:55.649 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:54.578+0000 7fd723cef700 1 -- 172.21.15.12:0/3623134307 shutdown_connections 2024-02-18T11:31:55.650 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:54.578+0000 7fd723cef700 1 -- 172.21.15.12:0/3623134307 wait complete. 2024-02-18T11:31:55.650 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:54.578+0000 7fd723cef700 1 Processor -- start 2024-02-18T11:31:55.650 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:54.579+0000 7fd723cef700 1 -- start start 2024-02-18T11:31:55.650 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:54.579+0000 7fd723cef700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd71c1065d0 0x7fd71c1992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:31:55.650 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:54.579+0000 7fd723cef700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd71c1997e0 con 0x7fd71c1065d0 2024-02-18T11:31:55.650 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:54.579+0000 7fd721a8b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd71c1065d0 0x7fd71c1992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:31:55.650 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:54.579+0000 7fd721a8b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd71c1065d0 0x7fd71c1992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52672/0 (socket says 172.21.15.12:52672) 2024-02-18T11:31:55.650 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:54.579+0000 7fd721a8b700 1 -- 172.21.15.12:0/803571519 learned_addr learned my addr 172.21.15.12:0/803571519 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:31:55.650 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:54.579+0000 7fd721a8b700 1 -- 172.21.15.12:0/803571519 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd70c009550 con 0x7fd71c1065d0 2024-02-18T11:31:55.650 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:54.579+0000 7fd721a8b700 1 --2- 172.21.15.12:0/803571519 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd71c1065d0 0x7fd71c1992a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd70c003f20 tx=0x7fd70c011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:31:55.650 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:54.579+0000 7fd712ffd700 1 -- 172.21.15.12:0/803571519 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd70c0039f0 con 0x7fd71c1065d0 2024-02-18T11:31:55.650 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:54.579+0000 7fd723cef700 1 -- 172.21.15.12:0/803571519 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd71c1999e0 con 0x7fd71c1065d0 2024-02-18T11:31:55.650 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:54.579+0000 7fd712ffd700 1 -- 172.21.15.12:0/803571519 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd70c003b50 con 0x7fd71c1065d0 2024-02-18T11:31:55.650 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:54.579+0000 7fd712ffd700 1 -- 172.21.15.12:0/803571519 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd70c01a690 con 0x7fd71c1065d0 2024-02-18T11:31:55.650 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:54.580+0000 7fd723cef700 1 -- 172.21.15.12:0/803571519 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd71c199e80 con 0x7fd71c1065d0 2024-02-18T11:31:55.651 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:54.580+0000 7fd712ffd700 1 -- 172.21.15.12:0/803571519 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 12) v1 ==== 46104+0+0 (secure 0 0 0) 0x7fd70c01a950 con 0x7fd71c1065d0 2024-02-18T11:31:55.651 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:54.581+0000 7fd712ffd700 1 --2- 172.21.15.12:0/803571519 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fd70803b340 0x7fd70803d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:31:55.651 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:54.581+0000 7fd712ffd700 1 -- 172.21.15.12:0/803571519 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd70c04d630 con 0x7fd71c1065d0 2024-02-18T11:31:55.651 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:54.581+0000 7fd72128a700 1 --2- 172.21.15.12:0/803571519 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fd70803b340 0x7fd70803d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:31:55.651 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:54.581+0000 7fd723cef700 1 -- 172.21.15.12:0/803571519 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd71c068da0 con 0x7fd71c1065d0 2024-02-18T11:31:55.651 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:54.581+0000 7fd72128a700 1 --2- 172.21.15.12:0/803571519 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fd70803b340 0x7fd70803d800 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd718006fd0 tx=0x7fd718006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:31:55.651 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:54.586+0000 7fd712ffd700 1 -- 172.21.15.12:0/803571519 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd70c022330 con 0x7fd71c1065d0 2024-02-18T11:31:55.651 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:54.803+0000 7fd723cef700 1 -- 172.21.15.12:0/803571519 --> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fd71c063100 con 0x7fd70803b340 2024-02-18T11:31:55.651 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:55.050+0000 7fd712ffd700 1 -- 172.21.15.12:0/803571519 <== mgr.14152 v2:172.21.15.12:6800/1153736685 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7fd71c063100 con 0x7fd70803b340 2024-02-18T11:31:55.652 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:55.052+0000 7fd723cef700 1 -- 172.21.15.12:0/803571519 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fd70803b340 msgr2=0x7fd70803d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:31:55.652 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:55.052+0000 7fd723cef700 1 --2- 172.21.15.12:0/803571519 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fd70803b340 0x7fd70803d800 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd718006fd0 tx=0x7fd718006e40 comp rx=0 tx=0).stop 2024-02-18T11:31:55.652 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:55.052+0000 7fd723cef700 1 -- 172.21.15.12:0/803571519 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd71c1065d0 msgr2=0x7fd71c1992a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:31:55.652 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:55.052+0000 7fd723cef700 1 --2- 172.21.15.12:0/803571519 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd71c1065d0 0x7fd71c1992a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd70c003f20 tx=0x7fd70c011b00 comp rx=0 tx=0).stop 2024-02-18T11:31:55.652 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:55.052+0000 7fd723cef700 1 -- 172.21.15.12:0/803571519 shutdown_connections 2024-02-18T11:31:55.652 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:55.052+0000 7fd723cef700 1 --2- 172.21.15.12:0/803571519 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fd70803b340 0x7fd70803d800 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:31:55.652 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:55.052+0000 7fd723cef700 1 --2- 172.21.15.12:0/803571519 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd71c1065d0 0x7fd71c1992a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:31:55.652 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:55.052+0000 7fd723cef700 1 -- 172.21.15.12:0/803571519 >> 172.21.15.12:0/803571519 conn(0x7fd71c101c10 msgr2=0x7fd71c18faf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:31:55.652 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:55.053+0000 7fd723cef700 1 -- 172.21.15.12:0/803571519 shutdown_connections 2024-02-18T11:31:55.652 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:55.053+0000 7fd723cef700 1 -- 172.21.15.12:0/803571519 wait complete. 2024-02-18T11:31:55.652 INFO:teuthology.orchestra.run.smithi012.stdout:Fetching dashboard port number... 2024-02-18T11:31:57.709 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 8443 2024-02-18T11:31:57.709 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:56.824+0000 7f6eb09f4700 1 Processor -- start 2024-02-18T11:31:57.709 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:56.824+0000 7f6eb09f4700 1 -- start start 2024-02-18T11:31:57.710 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:56.824+0000 7f6eb09f4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6eac1076b0 0x7f6eac107ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:31:57.710 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:56.824+0000 7f6eb09f4700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eac1080a0 con 0x7f6eac1076b0 2024-02-18T11:31:57.710 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:56.825+0000 7f6eaa59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6eac1076b0 0x7f6eac107ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:31:57.710 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:56.825+0000 7f6eaa59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6eac1076b0 0x7f6eac107ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52688/0 (socket says 172.21.15.12:52688) 2024-02-18T11:31:57.710 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:56.825+0000 7f6eaa59c700 1 -- 172.21.15.12:0/3076659488 learned_addr learned my addr 172.21.15.12:0/3076659488 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:31:57.710 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:56.825+0000 7f6eaa59c700 1 -- 172.21.15.12:0/3076659488 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eac108860 con 0x7f6eac1076b0 2024-02-18T11:31:57.710 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:56.825+0000 7f6eaa59c700 1 --2- 172.21.15.12:0/3076659488 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6eac1076b0 0x7f6eac107ad0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c0098a0 tx=0x7f6e9c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38a8d07243779604 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:31:57.710 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:56.825+0000 7f6ea959a700 1 -- 172.21.15.12:0/3076659488 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6e9c004160 con 0x7f6eac1076b0 2024-02-18T11:31:57.710 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:56.825+0000 7f6ea959a700 1 -- 172.21.15.12:0/3076659488 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6e9c0042c0 con 0x7f6eac1076b0 2024-02-18T11:31:57.710 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:56.825+0000 7f6ea959a700 1 -- 172.21.15.12:0/3076659488 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6e9c011a80 con 0x7f6eac1076b0 2024-02-18T11:31:57.710 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:56.826+0000 7f6eb09f4700 1 -- 172.21.15.12:0/3076659488 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6eac1076b0 msgr2=0x7f6eac107ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:31:57.710 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:56.826+0000 7f6eb09f4700 1 --2- 172.21.15.12:0/3076659488 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6eac1076b0 0x7f6eac107ad0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c0098a0 tx=0x7f6e9c009bb0 comp rx=0 tx=0).stop 2024-02-18T11:31:57.710 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:56.826+0000 7f6eb09f4700 1 -- 172.21.15.12:0/3076659488 shutdown_connections 2024-02-18T11:31:57.710 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:56.826+0000 7f6eb09f4700 1 --2- 172.21.15.12:0/3076659488 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6eac1076b0 0x7f6eac107ad0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:31:57.711 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:56.826+0000 7f6eb09f4700 1 -- 172.21.15.12:0/3076659488 >> 172.21.15.12:0/3076659488 conn(0x7f6eac102d30 msgr2=0x7f6eac105150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:31:57.711 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:56.826+0000 7f6eb09f4700 1 -- 172.21.15.12:0/3076659488 shutdown_connections 2024-02-18T11:31:57.711 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:56.826+0000 7f6eb09f4700 1 -- 172.21.15.12:0/3076659488 wait complete. 2024-02-18T11:31:57.711 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:56.827+0000 7f6eb09f4700 1 Processor -- start 2024-02-18T11:31:57.711 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:56.827+0000 7f6eb09f4700 1 -- start start 2024-02-18T11:31:57.711 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:56.827+0000 7f6eb09f4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6eac1076b0 0x7f6eac19b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:31:57.711 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:56.827+0000 7f6eb09f4700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eac19b960 con 0x7f6eac1076b0 2024-02-18T11:31:57.711 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:56.827+0000 7f6eaa59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6eac1076b0 0x7f6eac19b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:31:57.711 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:56.827+0000 7f6eaa59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6eac1076b0 0x7f6eac19b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52698/0 (socket says 172.21.15.12:52698) 2024-02-18T11:31:57.711 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:56.827+0000 7f6eaa59c700 1 -- 172.21.15.12:0/961707613 learned_addr learned my addr 172.21.15.12:0/961707613 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:31:57.711 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:56.827+0000 7f6eaa59c700 1 -- 172.21.15.12:0/961707613 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e9c009550 con 0x7f6eac1076b0 2024-02-18T11:31:57.711 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:56.827+0000 7f6eaa59c700 1 --2- 172.21.15.12:0/961707613 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6eac1076b0 0x7f6eac19b420 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c011740 tx=0x7f6e9c011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:31:57.711 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:56.828+0000 7f6e9b7fe700 1 -- 172.21.15.12:0/961707613 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6e9c0039e0 con 0x7f6eac1076b0 2024-02-18T11:31:57.711 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:56.828+0000 7f6eb09f4700 1 -- 172.21.15.12:0/961707613 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6eac19bb60 con 0x7f6eac1076b0 2024-02-18T11:31:57.712 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:56.828+0000 7f6e9b7fe700 1 -- 172.21.15.12:0/961707613 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6e9c003b40 con 0x7f6eac1076b0 2024-02-18T11:31:57.712 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:56.828+0000 7f6e9b7fe700 1 -- 172.21.15.12:0/961707613 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6e9c024740 con 0x7f6eac1076b0 2024-02-18T11:31:57.712 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:56.828+0000 7f6eb09f4700 1 -- 172.21.15.12:0/961707613 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6eac19c000 con 0x7f6eac1076b0 2024-02-18T11:31:57.712 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:56.829+0000 7f6e9b7fe700 1 -- 172.21.15.12:0/961707613 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 12) v1 ==== 46104+0+0 (secure 0 0 0) 0x7f6e9c02d3f0 con 0x7f6eac1076b0 2024-02-18T11:31:57.712 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:56.829+0000 7f6eb09f4700 1 -- 172.21.15.12:0/961707613 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6eac10ba90 con 0x7f6eac1076b0 2024-02-18T11:31:57.712 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:56.829+0000 7f6e9b7fe700 1 --2- 172.21.15.12:0/961707613 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f6e940391c0 0x7f6e9403b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:31:57.712 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:56.829+0000 7f6e9b7fe700 1 -- 172.21.15.12:0/961707613 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f6e9c04d670 con 0x7f6eac1076b0 2024-02-18T11:31:57.712 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:56.829+0000 7f6ea9d9b700 1 --2- 172.21.15.12:0/961707613 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f6e940391c0 0x7f6e9403b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:31:57.712 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:56.829+0000 7f6ea9d9b700 1 --2- 172.21.15.12:0/961707613 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f6e940391c0 0x7f6e9403b680 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6ea0006fd0 tx=0x7f6ea0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:31:57.712 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:56.834+0000 7f6e9b7fe700 1 -- 172.21.15.12:0/961707613 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6e9c010380 con 0x7f6eac1076b0 2024-02-18T11:31:57.712 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:57.025+0000 7f6e9b7fe700 1 -- 172.21.15.12:0/961707613 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mgrmap(e 13) v1 ==== 46147+0+0 (secure 0 0 0) 0x7f6e9c01a480 con 0x7f6eac1076b0 2024-02-18T11:31:57.712 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:57.037+0000 7f6eb09f4700 1 -- 172.21.15.12:0/961707613 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f6eac19e8e0 con 0x7f6eac1076b0 2024-02-18T11:31:57.712 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:57.037+0000 7f6e9b7fe700 1 -- 172.21.15.12:0/961707613 <== mon.0 v2:172.21.15.12:3300/0 8 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f6e9c018dc0 con 0x7f6eac1076b0 2024-02-18T11:31:57.712 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:57.039+0000 7f6eb09f4700 1 -- 172.21.15.12:0/961707613 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f6e940391c0 msgr2=0x7f6e9403b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:31:57.713 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:57.039+0000 7f6eb09f4700 1 --2- 172.21.15.12:0/961707613 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f6e940391c0 0x7f6e9403b680 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6ea0006fd0 tx=0x7f6ea0006e40 comp rx=0 tx=0).stop 2024-02-18T11:31:57.713 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:57.039+0000 7f6eb09f4700 1 -- 172.21.15.12:0/961707613 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6eac1076b0 msgr2=0x7f6eac19b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:31:57.713 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:57.039+0000 7f6eb09f4700 1 --2- 172.21.15.12:0/961707613 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6eac1076b0 0x7f6eac19b420 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c011740 tx=0x7f6e9c011b00 comp rx=0 tx=0).stop 2024-02-18T11:31:57.713 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:57.039+0000 7f6eb09f4700 1 -- 172.21.15.12:0/961707613 shutdown_connections 2024-02-18T11:31:57.713 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:57.039+0000 7f6eb09f4700 1 --2- 172.21.15.12:0/961707613 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f6e940391c0 0x7f6e9403b680 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:31:57.713 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:57.039+0000 7f6eb09f4700 1 --2- 172.21.15.12:0/961707613 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6eac1076b0 0x7f6eac19b420 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:31:57.713 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:57.039+0000 7f6eb09f4700 1 -- 172.21.15.12:0/961707613 >> 172.21.15.12:0/961707613 conn(0x7f6eac102d30 msgr2=0x7f6eac103820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:31:57.713 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:57.039+0000 7f6eb09f4700 1 -- 172.21.15.12:0/961707613 shutdown_connections 2024-02-18T11:31:57.713 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:57.039+0000 7f6eb09f4700 1 -- 172.21.15.12:0/961707613 wait complete. 2024-02-18T11:31:57.717 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-18T11:31:57.717 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout disabled 2024-02-18T11:31:57.725 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-18T11:31:57.725 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout inactive 2024-02-18T11:31:57.725 INFO:teuthology.orchestra.run.smithi012.stdout:firewalld.service is not enabled 2024-02-18T11:31:57.725 INFO:teuthology.orchestra.run.smithi012.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-02-18T11:31:57.727 INFO:teuthology.orchestra.run.smithi012.stdout:Ceph Dashboard is now available at: 2024-02-18T11:31:57.727 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:31:57.727 INFO:teuthology.orchestra.run.smithi012.stdout: URL: https://smithi012.front.sepia.ceph.com:8443/ 2024-02-18T11:31:57.727 INFO:teuthology.orchestra.run.smithi012.stdout: User: admin 2024-02-18T11:31:57.727 INFO:teuthology.orchestra.run.smithi012.stdout: Password: 8pzf039jma 2024-02-18T11:31:57.727 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:31:57.728 INFO:teuthology.orchestra.run.smithi012.stdout:Saving cluster configuration to /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/config directory 2024-02-18T11:31:57.728 INFO:teuthology.orchestra.run.smithi012.stdout:Enabling autotune for osd_memory_target 2024-02-18T11:31:58.119 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:58 smithi012 ceph-mon[32460]: mgrmap e13: x(active, since 6s) 2024-02-18T11:31:58.120 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:31:58 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/961707613' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-02-18T11:31:59.015 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:58.123+0000 7fc562774700 1 Processor -- start 2024-02-18T11:31:59.016 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:58.123+0000 7fc562774700 1 -- start start 2024-02-18T11:31:59.016 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:58.124+0000 7fc562774700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc55c07ced0 0x7fc55c07b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:31:59.016 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:58.124+0000 7fc562774700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc55c07b900 con 0x7fc55c07ced0 2024-02-18T11:31:59.016 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:58.124+0000 7fc55bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc55c07ced0 0x7fc55c07b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:31:59.016 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:58.124+0000 7fc55bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc55c07ced0 0x7fc55c07b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52700/0 (socket says 172.21.15.12:52700) 2024-02-18T11:31:59.016 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:58.124+0000 7fc55bfff700 1 -- 172.21.15.12:0/2653259978 learned_addr learned my addr 172.21.15.12:0/2653259978 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:31:59.016 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:58.124+0000 7fc55bfff700 1 -- 172.21.15.12:0/2653259978 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc55c07ba40 con 0x7fc55c07ced0 2024-02-18T11:31:59.016 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:58.124+0000 7fc55bfff700 1 --2- 172.21.15.12:0/2653259978 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc55c07ced0 0x7fc55c07b330 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc54c0098a0 tx=0x7fc54c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6767c626fe66ee96 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:31:59.016 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:58.125+0000 7fc55affd700 1 -- 172.21.15.12:0/2653259978 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc54c004160 con 0x7fc55c07ced0 2024-02-18T11:31:59.016 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:58.125+0000 7fc55affd700 1 -- 172.21.15.12:0/2653259978 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc54c0042c0 con 0x7fc55c07ced0 2024-02-18T11:31:59.016 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:58.125+0000 7fc55affd700 1 -- 172.21.15.12:0/2653259978 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc54c011a80 con 0x7fc55c07ced0 2024-02-18T11:31:59.016 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:58.125+0000 7fc562774700 1 -- 172.21.15.12:0/2653259978 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc55c07ced0 msgr2=0x7fc55c07b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:31:59.017 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:58.125+0000 7fc562774700 1 --2- 172.21.15.12:0/2653259978 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc55c07ced0 0x7fc55c07b330 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc54c0098a0 tx=0x7fc54c009bb0 comp rx=0 tx=0).stop 2024-02-18T11:31:59.017 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:58.125+0000 7fc562774700 1 -- 172.21.15.12:0/2653259978 shutdown_connections 2024-02-18T11:31:59.017 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:58.125+0000 7fc562774700 1 --2- 172.21.15.12:0/2653259978 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc55c07ced0 0x7fc55c07b330 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:31:59.017 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:58.125+0000 7fc562774700 1 -- 172.21.15.12:0/2653259978 >> 172.21.15.12:0/2653259978 conn(0x7fc55c102d80 msgr2=0x7fc55c1051a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:31:59.017 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:58.125+0000 7fc562774700 1 -- 172.21.15.12:0/2653259978 shutdown_connections 2024-02-18T11:31:59.017 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:58.125+0000 7fc562774700 1 -- 172.21.15.12:0/2653259978 wait complete. 2024-02-18T11:31:59.017 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:58.126+0000 7fc562774700 1 Processor -- start 2024-02-18T11:31:59.017 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:58.126+0000 7fc562774700 1 -- start start 2024-02-18T11:31:59.017 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:58.126+0000 7fc562774700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc55c07ced0 0x7fc55c19d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:31:59.017 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:58.126+0000 7fc562774700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc55c19dbb0 con 0x7fc55c07ced0 2024-02-18T11:31:59.017 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:58.127+0000 7fc55bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc55c07ced0 0x7fc55c19d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:31:59.017 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:58.127+0000 7fc55bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc55c07ced0 0x7fc55c19d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52702/0 (socket says 172.21.15.12:52702) 2024-02-18T11:31:59.017 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:58.127+0000 7fc55bfff700 1 -- 172.21.15.12:0/2742467212 learned_addr learned my addr 172.21.15.12:0/2742467212 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:31:59.018 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:58.127+0000 7fc55bfff700 1 -- 172.21.15.12:0/2742467212 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc54c009550 con 0x7fc55c07ced0 2024-02-18T11:31:59.018 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:58.127+0000 7fc55bfff700 1 --2- 172.21.15.12:0/2742467212 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc55c07ced0 0x7fc55c19d670 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc54c000c00 tx=0x7fc54c011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:31:59.018 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:58.127+0000 7fc5597fa700 1 -- 172.21.15.12:0/2742467212 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc54c003aa0 con 0x7fc55c07ced0 2024-02-18T11:31:59.018 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:58.127+0000 7fc5597fa700 1 -- 172.21.15.12:0/2742467212 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc54c003c00 con 0x7fc55c07ced0 2024-02-18T11:31:59.018 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:58.127+0000 7fc562774700 1 -- 172.21.15.12:0/2742467212 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc55c19ddb0 con 0x7fc55c07ced0 2024-02-18T11:31:59.018 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:58.127+0000 7fc5597fa700 1 -- 172.21.15.12:0/2742467212 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc54c024930 con 0x7fc55c07ced0 2024-02-18T11:31:59.018 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:58.127+0000 7fc562774700 1 -- 172.21.15.12:0/2742467212 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc55c19e250 con 0x7fc55c07ced0 2024-02-18T11:31:59.018 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:58.128+0000 7fc5597fa700 1 -- 172.21.15.12:0/2742467212 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) v1 ==== 46147+0+0 (secure 0 0 0) 0x7fc54c02d3f0 con 0x7fc55c07ced0 2024-02-18T11:31:59.018 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:58.128+0000 7fc5597fa700 1 --2- 172.21.15.12:0/2742467212 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fc544039260 0x7fc54403b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:31:59.018 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:58.128+0000 7fc562774700 1 -- 172.21.15.12:0/2742467212 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc55c10db60 con 0x7fc55c07ced0 2024-02-18T11:31:59.018 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:58.128+0000 7fc5597fa700 1 -- 172.21.15.12:0/2742467212 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc54c04d760 con 0x7fc55c07ced0 2024-02-18T11:31:59.018 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:58.128+0000 7fc55b7fe700 1 --2- 172.21.15.12:0/2742467212 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fc544039260 0x7fc54403b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:31:59.018 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:58.129+0000 7fc55b7fe700 1 --2- 172.21.15.12:0/2742467212 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fc544039260 0x7fc54403b720 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc550006fd0 tx=0x7fc550006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:31:59.018 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:58.133+0000 7fc5597fa700 1 -- 172.21.15.12:0/2742467212 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc54c010a70 con 0x7fc55c07ced0 2024-02-18T11:31:59.019 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:58.335+0000 7fc562774700 1 -- 172.21.15.12:0/2742467212 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7fc55c1a0b30 con 0x7fc55c07ced0 2024-02-18T11:31:59.019 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:58.336+0000 7fc5597fa700 1 -- 172.21.15.12:0/2742467212 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7fc54c010ed0 con 0x7fc55c07ced0 2024-02-18T11:31:59.019 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:58.337+0000 7fc562774700 1 -- 172.21.15.12:0/2742467212 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fc544039260 msgr2=0x7fc54403b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:31:59.019 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:58.337+0000 7fc562774700 1 --2- 172.21.15.12:0/2742467212 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fc544039260 0x7fc54403b720 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc550006fd0 tx=0x7fc550006e40 comp rx=0 tx=0).stop 2024-02-18T11:31:59.019 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:58.337+0000 7fc562774700 1 -- 172.21.15.12:0/2742467212 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc55c07ced0 msgr2=0x7fc55c19d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:31:59.019 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:58.337+0000 7fc562774700 1 --2- 172.21.15.12:0/2742467212 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc55c07ced0 0x7fc55c19d670 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc54c000c00 tx=0x7fc54c011b00 comp rx=0 tx=0).stop 2024-02-18T11:31:59.019 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:58.338+0000 7fc562774700 1 -- 172.21.15.12:0/2742467212 shutdown_connections 2024-02-18T11:31:59.019 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:58.338+0000 7fc562774700 1 --2- 172.21.15.12:0/2742467212 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fc544039260 0x7fc54403b720 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:31:59.019 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:58.338+0000 7fc562774700 1 --2- 172.21.15.12:0/2742467212 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc55c07ced0 0x7fc55c19d670 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:31:59.019 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:58.338+0000 7fc562774700 1 -- 172.21.15.12:0/2742467212 >> 172.21.15.12:0/2742467212 conn(0x7fc55c102d80 msgr2=0x7fc55c1038f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:31:59.019 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:58.338+0000 7fc562774700 1 -- 172.21.15.12:0/2742467212 shutdown_connections 2024-02-18T11:31:59.019 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:58.338+0000 7fc562774700 1 -- 172.21.15.12:0/2742467212 wait complete. 2024-02-18T11:32:00.298 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:59.452+0000 7f68555f2700 1 Processor -- start 2024-02-18T11:32:00.298 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:59.452+0000 7f68555f2700 1 -- start start 2024-02-18T11:32:00.298 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:59.452+0000 7f68555f2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68501076b0 0x7f6850107ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:32:00.298 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:59.452+0000 7f68555f2700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68501080a0 con 0x7f68501076b0 2024-02-18T11:32:00.298 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:59.452+0000 7f684effd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68501076b0 0x7f6850107ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:32:00.298 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:59.452+0000 7f684effd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68501076b0 0x7f6850107ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52716/0 (socket says 172.21.15.12:52716) 2024-02-18T11:32:00.298 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:59.452+0000 7f684effd700 1 -- 172.21.15.12:0/1152733518 learned_addr learned my addr 172.21.15.12:0/1152733518 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:32:00.298 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:59.452+0000 7f684effd700 1 -- 172.21.15.12:0/1152733518 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6850108860 con 0x7f68501076b0 2024-02-18T11:32:00.299 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:59.452+0000 7f684effd700 1 --2- 172.21.15.12:0/1152733518 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68501076b0 0x7f6850107ad0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f68400098a0 tx=0x7f6840009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=840d558efea53b55 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:32:00.299 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:59.453+0000 7f684dffb700 1 -- 172.21.15.12:0/1152733518 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6840004160 con 0x7f68501076b0 2024-02-18T11:32:00.299 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:59.453+0000 7f684dffb700 1 -- 172.21.15.12:0/1152733518 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f68400042c0 con 0x7f68501076b0 2024-02-18T11:32:00.299 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:59.453+0000 7f684dffb700 1 -- 172.21.15.12:0/1152733518 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6840011a80 con 0x7f68501076b0 2024-02-18T11:32:00.299 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:59.453+0000 7f68555f2700 1 -- 172.21.15.12:0/1152733518 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68501076b0 msgr2=0x7f6850107ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:32:00.299 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:59.453+0000 7f68555f2700 1 --2- 172.21.15.12:0/1152733518 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68501076b0 0x7f6850107ad0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f68400098a0 tx=0x7f6840009bb0 comp rx=0 tx=0).stop 2024-02-18T11:32:00.299 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:59.453+0000 7f68555f2700 1 -- 172.21.15.12:0/1152733518 shutdown_connections 2024-02-18T11:32:00.299 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:59.453+0000 7f68555f2700 1 --2- 172.21.15.12:0/1152733518 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68501076b0 0x7f6850107ad0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:32:00.299 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:59.453+0000 7f68555f2700 1 -- 172.21.15.12:0/1152733518 >> 172.21.15.12:0/1152733518 conn(0x7f6850102d90 msgr2=0x7f68501051b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:32:00.299 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:59.453+0000 7f68555f2700 1 -- 172.21.15.12:0/1152733518 shutdown_connections 2024-02-18T11:32:00.299 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:59.453+0000 7f68555f2700 1 -- 172.21.15.12:0/1152733518 wait complete. 2024-02-18T11:32:00.299 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:59.454+0000 7f68555f2700 1 Processor -- start 2024-02-18T11:32:00.299 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:59.454+0000 7f68555f2700 1 -- start start 2024-02-18T11:32:00.299 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:59.454+0000 7f68555f2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68501076b0 0x7f685019b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:32:00.300 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:59.454+0000 7f68555f2700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f685019b9a0 con 0x7f68501076b0 2024-02-18T11:32:00.300 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:59.454+0000 7f684effd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68501076b0 0x7f685019b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:32:00.300 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:59.454+0000 7f684effd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68501076b0 0x7f685019b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52722/0 (socket says 172.21.15.12:52722) 2024-02-18T11:32:00.300 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:59.454+0000 7f684effd700 1 -- 172.21.15.12:0/2113482284 learned_addr learned my addr 172.21.15.12:0/2113482284 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:32:00.300 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:59.455+0000 7f684effd700 1 -- 172.21.15.12:0/2113482284 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6840009550 con 0x7f68501076b0 2024-02-18T11:32:00.300 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:59.455+0000 7f684effd700 1 --2- 172.21.15.12:0/2113482284 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68501076b0 0x7f685019b460 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6840011740 tx=0x7f6840011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:32:00.300 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:59.455+0000 7f6837fff700 1 -- 172.21.15.12:0/2113482284 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f68400039e0 con 0x7f68501076b0 2024-02-18T11:32:00.300 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:59.455+0000 7f6837fff700 1 -- 172.21.15.12:0/2113482284 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6840003b40 con 0x7f68501076b0 2024-02-18T11:32:00.300 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:59.455+0000 7f68555f2700 1 -- 172.21.15.12:0/2113482284 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f685019bba0 con 0x7f68501076b0 2024-02-18T11:32:00.300 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:59.455+0000 7f6837fff700 1 -- 172.21.15.12:0/2113482284 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6840024710 con 0x7f68501076b0 2024-02-18T11:32:00.300 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:59.455+0000 7f68555f2700 1 -- 172.21.15.12:0/2113482284 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f685019c040 con 0x7f68501076b0 2024-02-18T11:32:00.301 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:59.456+0000 7f6837fff700 1 -- 172.21.15.12:0/2113482284 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) v1 ==== 46147+0+0 (secure 0 0 0) 0x7f68400249d0 con 0x7f68501076b0 2024-02-18T11:32:00.301 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:59.456+0000 7f6837fff700 1 --2- 172.21.15.12:0/2113482284 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f68380391b0 0x7f683803b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:32:00.301 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:59.456+0000 7f68555f2700 1 -- 172.21.15.12:0/2113482284 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f685010ba90 con 0x7f68501076b0 2024-02-18T11:32:00.301 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:59.456+0000 7f6837fff700 1 -- 172.21.15.12:0/2113482284 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f684004d890 con 0x7f68501076b0 2024-02-18T11:32:00.301 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:59.456+0000 7f684e7fc700 1 --2- 172.21.15.12:0/2113482284 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f68380391b0 0x7f683803b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:32:00.301 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:59.457+0000 7f684e7fc700 1 --2- 172.21.15.12:0/2113482284 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f68380391b0 0x7f683803b670 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6844006fd0 tx=0x7f6844006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:32:00.301 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:59.461+0000 7f6837fff700 1 -- 172.21.15.12:0/2113482284 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6840022310 con 0x7f68501076b0 2024-02-18T11:32:00.301 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:59.779+0000 7f68555f2700 1 -- 172.21.15.12:0/2113482284 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f685019e920 con 0x7f68501076b0 2024-02-18T11:32:00.301 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:59.782+0000 7f6837fff700 1 -- 172.21.15.12:0/2113482284 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) v1 ==== 153+0+0 (secure 0 0 0) 0x7f6840018ba0 con 0x7f68501076b0 2024-02-18T11:32:00.301 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:59.784+0000 7f68555f2700 1 -- 172.21.15.12:0/2113482284 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f68380391b0 msgr2=0x7f683803b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:32:00.301 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:59.784+0000 7f68555f2700 1 --2- 172.21.15.12:0/2113482284 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f68380391b0 0x7f683803b670 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6844006fd0 tx=0x7f6844006e40 comp rx=0 tx=0).stop 2024-02-18T11:32:00.301 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:59.784+0000 7f68555f2700 1 -- 172.21.15.12:0/2113482284 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68501076b0 msgr2=0x7f685019b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:32:00.301 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:59.784+0000 7f68555f2700 1 --2- 172.21.15.12:0/2113482284 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68501076b0 0x7f685019b460 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6840011740 tx=0x7f6840011b00 comp rx=0 tx=0).stop 2024-02-18T11:32:00.301 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:59.784+0000 7f68555f2700 1 -- 172.21.15.12:0/2113482284 shutdown_connections 2024-02-18T11:32:00.301 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:59.784+0000 7f68555f2700 1 --2- 172.21.15.12:0/2113482284 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f68380391b0 0x7f683803b670 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:32:00.302 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:59.784+0000 7f68555f2700 1 --2- 172.21.15.12:0/2113482284 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68501076b0 0x7f685019b460 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:32:00.302 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:59.784+0000 7f68555f2700 1 -- 172.21.15.12:0/2113482284 >> 172.21.15.12:0/2113482284 conn(0x7f6850102d90 msgr2=0x7f6850103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:32:00.302 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:59.784+0000 7f68555f2700 1 -- 172.21.15.12:0/2113482284 shutdown_connections 2024-02-18T11:32:00.302 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-02-18T11:31:59.784+0000 7f68555f2700 1 -- 172.21.15.12:0/2113482284 wait complete. 2024-02-18T11:32:00.302 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-02-18T11:32:00.302 INFO:teuthology.orchestra.run.smithi012.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-02-18T11:32:00.302 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:32:00.302 INFO:teuthology.orchestra.run.smithi012.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-02-18T11:32:00.302 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:32:00.302 INFO:teuthology.orchestra.run.smithi012.stdout:Or, if you are only running a single cluster on this host: 2024-02-18T11:32:00.302 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:32:00.302 INFO:teuthology.orchestra.run.smithi012.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-02-18T11:32:00.302 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:32:00.302 INFO:teuthology.orchestra.run.smithi012.stdout:Please consider enabling telemetry to help improve Ceph: 2024-02-18T11:32:00.302 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:32:00.302 INFO:teuthology.orchestra.run.smithi012.stdout: ceph telemetry on 2024-02-18T11:32:00.302 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:32:00.303 INFO:teuthology.orchestra.run.smithi012.stdout:For more information see: 2024-02-18T11:32:00.303 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:32:00.303 INFO:teuthology.orchestra.run.smithi012.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-02-18T11:32:00.303 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:32:00.303 INFO:teuthology.orchestra.run.smithi012.stdout:Bootstrap complete. 2024-02-18T11:32:00.342 INFO:tasks.cephadm:Fetching config... 2024-02-18T11:32:00.343 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-02-18T11:32:00.343 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-02-18T11:32:00.360 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-02-18T11:32:00.360 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-02-18T11:32:00.361 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-02-18T11:32:00.416 INFO:tasks.cephadm:Fetching mon keyring... 2024-02-18T11:32:00.416 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-02-18T11:32:00.416 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.a/keyring of=/dev/stdout 2024-02-18T11:32:00.484 INFO:tasks.cephadm:Fetching pub ssh key... 2024-02-18T11:32:00.484 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-02-18T11:32:00.484 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-02-18T11:32:00.539 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-02-18T11:32:00.539 DEBUG:teuthology.orchestra.run.smithi012:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCsOxk1tzTDdhxzE92jO/raSNrXiPGUOfiYVJqSk5YsD8hBS9Bx0f4NSwPvwXjzQkKXjuyMfFbsnTg/fJ+tB/wANj52SWtSK2/wUEhWe8Vk+uXVvGLAxVGO+mQER9iiQtPJp1zIrI8amzHbgn5rJ4wV5DgPizEIHv+EpCX24Ig7PrRM96G2Ym0tex6URkhEvLX78LEH0tVvwAVWuC69EcATwiIZYSpIQSs5SpLyj89KxUQB8wM5gUV5jSe3/I1f3FuoU5u+i0prXNwkpUFE2owpD3YkZ/2ffUxS55z0WDL1lUKfgkCM8dCKuoTOShPstAbu4i6+kjqEDiOO2/l1IV657k60uLKrHX5BqW2kFPeO7Be3uSoA2pEVME+MJ87XWlW7N3Ut+MwXuOR95MAquLieGs2WqRNZc116ESTEt7Izqq0H4P/73NJmpY9VWP4rXUFxJNNQUQUkkyyFfMIRKtu3Wj1UKOO1gx+LNOV7Z7gMvBBLdmjPbsde+7pdILDIxwk= ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-18T11:32:00.618 INFO:teuthology.orchestra.run.smithi012.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCsOxk1tzTDdhxzE92jO/raSNrXiPGUOfiYVJqSk5YsD8hBS9Bx0f4NSwPvwXjzQkKXjuyMfFbsnTg/fJ+tB/wANj52SWtSK2/wUEhWe8Vk+uXVvGLAxVGO+mQER9iiQtPJp1zIrI8amzHbgn5rJ4wV5DgPizEIHv+EpCX24Ig7PrRM96G2Ym0tex6URkhEvLX78LEH0tVvwAVWuC69EcATwiIZYSpIQSs5SpLyj89KxUQB8wM5gUV5jSe3/I1f3FuoU5u+i0prXNwkpUFE2owpD3YkZ/2ffUxS55z0WDL1lUKfgkCM8dCKuoTOShPstAbu4i6+kjqEDiOO2/l1IV657k60uLKrHX5BqW2kFPeO7Be3uSoA2pEVME+MJ87XWlW7N3Ut+MwXuOR95MAquLieGs2WqRNZc116ESTEt7Izqq0H4P/73NJmpY9VWP4rXUFxJNNQUQUkkyyFfMIRKtu3Wj1UKOO1gx+LNOV7Z7gMvBBLdmjPbsde+7pdILDIxwk= ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715 2024-02-18T11:32:00.631 DEBUG:teuthology.orchestra.run.smithi158:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCsOxk1tzTDdhxzE92jO/raSNrXiPGUOfiYVJqSk5YsD8hBS9Bx0f4NSwPvwXjzQkKXjuyMfFbsnTg/fJ+tB/wANj52SWtSK2/wUEhWe8Vk+uXVvGLAxVGO+mQER9iiQtPJp1zIrI8amzHbgn5rJ4wV5DgPizEIHv+EpCX24Ig7PrRM96G2Ym0tex6URkhEvLX78LEH0tVvwAVWuC69EcATwiIZYSpIQSs5SpLyj89KxUQB8wM5gUV5jSe3/I1f3FuoU5u+i0prXNwkpUFE2owpD3YkZ/2ffUxS55z0WDL1lUKfgkCM8dCKuoTOShPstAbu4i6+kjqEDiOO2/l1IV657k60uLKrHX5BqW2kFPeO7Be3uSoA2pEVME+MJ87XWlW7N3Ut+MwXuOR95MAquLieGs2WqRNZc116ESTEt7Izqq0H4P/73NJmpY9VWP4rXUFxJNNQUQUkkyyFfMIRKtu3Wj1UKOO1gx+LNOV7Z7gMvBBLdmjPbsde+7pdILDIxwk= ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-18T11:32:00.676 INFO:teuthology.orchestra.run.smithi158.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCsOxk1tzTDdhxzE92jO/raSNrXiPGUOfiYVJqSk5YsD8hBS9Bx0f4NSwPvwXjzQkKXjuyMfFbsnTg/fJ+tB/wANj52SWtSK2/wUEhWe8Vk+uXVvGLAxVGO+mQER9iiQtPJp1zIrI8amzHbgn5rJ4wV5DgPizEIHv+EpCX24Ig7PrRM96G2Ym0tex6URkhEvLX78LEH0tVvwAVWuC69EcATwiIZYSpIQSs5SpLyj89KxUQB8wM5gUV5jSe3/I1f3FuoU5u+i0prXNwkpUFE2owpD3YkZ/2ffUxS55z0WDL1lUKfgkCM8dCKuoTOShPstAbu4i6+kjqEDiOO2/l1IV657k60uLKrHX5BqW2kFPeO7Be3uSoA2pEVME+MJ87XWlW7N3Ut+MwXuOR95MAquLieGs2WqRNZc116ESTEt7Izqq0H4P/73NJmpY9VWP4rXUFxJNNQUQUkkyyFfMIRKtu3Wj1UKOO1gx+LNOV7Z7gMvBBLdmjPbsde+7pdILDIxwk= ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715 2024-02-18T11:32:00.691 DEBUG:teuthology.orchestra.run.smithi188:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCsOxk1tzTDdhxzE92jO/raSNrXiPGUOfiYVJqSk5YsD8hBS9Bx0f4NSwPvwXjzQkKXjuyMfFbsnTg/fJ+tB/wANj52SWtSK2/wUEhWe8Vk+uXVvGLAxVGO+mQER9iiQtPJp1zIrI8amzHbgn5rJ4wV5DgPizEIHv+EpCX24Ig7PrRM96G2Ym0tex6URkhEvLX78LEH0tVvwAVWuC69EcATwiIZYSpIQSs5SpLyj89KxUQB8wM5gUV5jSe3/I1f3FuoU5u+i0prXNwkpUFE2owpD3YkZ/2ffUxS55z0WDL1lUKfgkCM8dCKuoTOShPstAbu4i6+kjqEDiOO2/l1IV657k60uLKrHX5BqW2kFPeO7Be3uSoA2pEVME+MJ87XWlW7N3Ut+MwXuOR95MAquLieGs2WqRNZc116ESTEt7Izqq0H4P/73NJmpY9VWP4rXUFxJNNQUQUkkyyFfMIRKtu3Wj1UKOO1gx+LNOV7Z7gMvBBLdmjPbsde+7pdILDIxwk= ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-18T11:32:00.736 INFO:teuthology.orchestra.run.smithi188.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCsOxk1tzTDdhxzE92jO/raSNrXiPGUOfiYVJqSk5YsD8hBS9Bx0f4NSwPvwXjzQkKXjuyMfFbsnTg/fJ+tB/wANj52SWtSK2/wUEhWe8Vk+uXVvGLAxVGO+mQER9iiQtPJp1zIrI8amzHbgn5rJ4wV5DgPizEIHv+EpCX24Ig7PrRM96G2Ym0tex6URkhEvLX78LEH0tVvwAVWuC69EcATwiIZYSpIQSs5SpLyj89KxUQB8wM5gUV5jSe3/I1f3FuoU5u+i0prXNwkpUFE2owpD3YkZ/2ffUxS55z0WDL1lUKfgkCM8dCKuoTOShPstAbu4i6+kjqEDiOO2/l1IV657k60uLKrHX5BqW2kFPeO7Be3uSoA2pEVME+MJ87XWlW7N3Ut+MwXuOR95MAquLieGs2WqRNZc116ESTEt7Izqq0H4P/73NJmpY9VWP4rXUFxJNNQUQUkkyyFfMIRKtu3Wj1UKOO1gx+LNOV7Z7gMvBBLdmjPbsde+7pdILDIxwk= ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715 2024-02-18T11:32:00.751 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-02-18T11:32:00.993 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T11:32:01.115 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:00 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/2113482284' entity='client.admin' 2024-02-18T11:32:03.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:03.190+0000 7fb81b5c6700 1 -- 172.21.15.12:0/2828071311 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8140ffa00 msgr2=0x7fb8140ffe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:32:03.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:03.190+0000 7fb81b5c6700 1 --2- 172.21.15.12:0/2828071311 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8140ffa00 0x7fb8140ffe20 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb810009910 tx=0x7fb810009c20 comp rx=0 tx=0).stop 2024-02-18T11:32:03.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:03.190+0000 7fb81b5c6700 1 -- 172.21.15.12:0/2828071311 shutdown_connections 2024-02-18T11:32:03.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:03.190+0000 7fb81b5c6700 1 --2- 172.21.15.12:0/2828071311 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8140ffa00 0x7fb8140ffe20 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:32:03.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:03.190+0000 7fb81b5c6700 1 -- 172.21.15.12:0/2828071311 >> 172.21.15.12:0/2828071311 conn(0x7fb8140fb360 msgr2=0x7fb8140fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:32:03.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:03.190+0000 7fb81b5c6700 1 -- 172.21.15.12:0/2828071311 shutdown_connections 2024-02-18T11:32:03.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:03.190+0000 7fb81b5c6700 1 -- 172.21.15.12:0/2828071311 wait complete. 2024-02-18T11:32:03.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:03.191+0000 7fb81b5c6700 1 Processor -- start 2024-02-18T11:32:03.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:03.191+0000 7fb81b5c6700 1 -- start start 2024-02-18T11:32:03.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:03.191+0000 7fb81b5c6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8140ffa00 0x7fb814196bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:32:03.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:03.191+0000 7fb81b5c6700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb814197100 con 0x7fb8140ffa00 2024-02-18T11:32:03.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:03.191+0000 7fb819362700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8140ffa00 0x7fb814196bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:32:03.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:03.191+0000 7fb819362700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8140ffa00 0x7fb814196bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35630/0 (socket says 172.21.15.12:35630) 2024-02-18T11:32:03.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:03.191+0000 7fb819362700 1 -- 172.21.15.12:0/3971658838 learned_addr learned my addr 172.21.15.12:0/3971658838 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:32:03.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:03.191+0000 7fb819362700 1 -- 172.21.15.12:0/3971658838 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8100095f0 con 0x7fb8140ffa00 2024-02-18T11:32:03.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:03.191+0000 7fb819362700 1 --2- 172.21.15.12:0/3971658838 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8140ffa00 0x7fb814196bc0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb810005850 tx=0x7fb810004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:32:03.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:03.192+0000 7fb80a7fc700 1 -- 172.21.15.12:0/3971658838 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb81001d070 con 0x7fb8140ffa00 2024-02-18T11:32:03.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:03.192+0000 7fb80a7fc700 1 -- 172.21.15.12:0/3971658838 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb81000be40 con 0x7fb8140ffa00 2024-02-18T11:32:03.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:03.192+0000 7fb81b5c6700 1 -- 172.21.15.12:0/3971658838 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb814197360 con 0x7fb8140ffa00 2024-02-18T11:32:03.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:03.192+0000 7fb80a7fc700 1 -- 172.21.15.12:0/3971658838 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb810022ea0 con 0x7fb8140ffa00 2024-02-18T11:32:03.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:03.192+0000 7fb81b5c6700 1 -- 172.21.15.12:0/3971658838 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb81419a080 con 0x7fb8140ffa00 2024-02-18T11:32:03.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:03.192+0000 7fb80a7fc700 1 -- 172.21.15.12:0/3971658838 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) v1 ==== 46147+0+0 (secure 0 0 0) 0x7fb81000f4b0 con 0x7fb8140ffa00 2024-02-18T11:32:03.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:03.193+0000 7fb81b5c6700 1 -- 172.21.15.12:0/3971658838 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb814190530 con 0x7fb8140ffa00 2024-02-18T11:32:03.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:03.193+0000 7fb80a7fc700 1 --2- 172.21.15.12:0/3971658838 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fb8000391b0 0x7fb80003b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:32:03.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:03.193+0000 7fb80a7fc700 1 -- 172.21.15.12:0/3971658838 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb81004d500 con 0x7fb8140ffa00 2024-02-18T11:32:03.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:03.193+0000 7fb818b61700 1 --2- 172.21.15.12:0/3971658838 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fb8000391b0 0x7fb80003b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:32:03.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:03.193+0000 7fb818b61700 1 --2- 172.21.15.12:0/3971658838 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fb8000391b0 0x7fb80003b670 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb804006fd0 tx=0x7fb804006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:32:03.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:03.198+0000 7fb80a7fc700 1 -- 172.21.15.12:0/3971658838 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb810017440 con 0x7fb8140ffa00 2024-02-18T11:32:03.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:03.397+0000 7fb81b5c6700 1 -- 172.21.15.12:0/3971658838 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7fb8140632e0 con 0x7fb8140ffa00 2024-02-18T11:32:03.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:03.399+0000 7fb80a7fc700 1 -- 172.21.15.12:0/3971658838 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7fb810027030 con 0x7fb8140ffa00 2024-02-18T11:32:03.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:03.401+0000 7fb81b5c6700 1 -- 172.21.15.12:0/3971658838 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fb8000391b0 msgr2=0x7fb80003b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:32:03.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:03.401+0000 7fb81b5c6700 1 --2- 172.21.15.12:0/3971658838 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fb8000391b0 0x7fb80003b670 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb804006fd0 tx=0x7fb804006e40 comp rx=0 tx=0).stop 2024-02-18T11:32:03.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:03.401+0000 7fb81b5c6700 1 -- 172.21.15.12:0/3971658838 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8140ffa00 msgr2=0x7fb814196bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:32:03.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:03.401+0000 7fb81b5c6700 1 --2- 172.21.15.12:0/3971658838 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8140ffa00 0x7fb814196bc0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb810005850 tx=0x7fb810004ff0 comp rx=0 tx=0).stop 2024-02-18T11:32:03.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:03.401+0000 7fb81b5c6700 1 -- 172.21.15.12:0/3971658838 shutdown_connections 2024-02-18T11:32:03.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:03.401+0000 7fb81b5c6700 1 --2- 172.21.15.12:0/3971658838 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fb8000391b0 0x7fb80003b670 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:32:03.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:03.401+0000 7fb81b5c6700 1 --2- 172.21.15.12:0/3971658838 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8140ffa00 0x7fb814196bc0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:32:03.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:03.401+0000 7fb81b5c6700 1 -- 172.21.15.12:0/3971658838 >> 172.21.15.12:0/3971658838 conn(0x7fb8140fb360 msgr2=0x7fb8140fbde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:32:03.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:03.402+0000 7fb81b5c6700 1 -- 172.21.15.12:0/3971658838 shutdown_connections 2024-02-18T11:32:03.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:03.402+0000 7fb81b5c6700 1 -- 172.21.15.12:0/3971658838 wait complete. 2024-02-18T11:32:03.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:03 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:03.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:03 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:03.675 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:03 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi012", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:32:03.675 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:03 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:03.675 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:03 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:32:03.675 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:03 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:03.970 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-02-18T11:32:03.971 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-02-18T11:32:04.209 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T11:32:04.595 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:04 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/3971658838' entity='client.admin' 2024-02-18T11:32:04.595 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:04 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:32:04.595 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:04 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:05.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:05.554+0000 7fadca9c9700 1 -- 172.21.15.12:0/3130164113 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fadc41005f0 msgr2=0x7fadc4100a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:32:05.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:05.554+0000 7fadca9c9700 1 --2- 172.21.15.12:0/3130164113 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fadc41005f0 0x7fadc4100a10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fadb4009910 tx=0x7fadb4009c20 comp rx=0 tx=0).stop 2024-02-18T11:32:05.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:05.554+0000 7fadca9c9700 1 -- 172.21.15.12:0/3130164113 shutdown_connections 2024-02-18T11:32:05.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:05.554+0000 7fadca9c9700 1 --2- 172.21.15.12:0/3130164113 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fadc41005f0 0x7fadc4100a10 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:32:05.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:05.554+0000 7fadca9c9700 1 -- 172.21.15.12:0/3130164113 >> 172.21.15.12:0/3130164113 conn(0x7fadc40fbbb0 msgr2=0x7fadc40fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:32:05.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:05.554+0000 7fadca9c9700 1 -- 172.21.15.12:0/3130164113 shutdown_connections 2024-02-18T11:32:05.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:05.554+0000 7fadca9c9700 1 -- 172.21.15.12:0/3130164113 wait complete. 2024-02-18T11:32:05.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:05.555+0000 7fadca9c9700 1 Processor -- start 2024-02-18T11:32:05.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:05.555+0000 7fadca9c9700 1 -- start start 2024-02-18T11:32:05.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:05.555+0000 7fadca9c9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fadc41005f0 0x7fadc4196c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:32:05.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:05.555+0000 7fadca9c9700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc4197150 con 0x7fadc41005f0 2024-02-18T11:32:05.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:05.555+0000 7fadc3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fadc41005f0 0x7fadc4196c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:32:05.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:05.555+0000 7fadc3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fadc41005f0 0x7fadc4196c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35650/0 (socket says 172.21.15.12:35650) 2024-02-18T11:32:05.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:05.555+0000 7fadc3fff700 1 -- 172.21.15.12:0/1300188150 learned_addr learned my addr 172.21.15.12:0/1300188150 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:32:05.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:05.556+0000 7fadc3fff700 1 -- 172.21.15.12:0/1300188150 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadb40095f0 con 0x7fadc41005f0 2024-02-18T11:32:05.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:05.556+0000 7fadc3fff700 1 --2- 172.21.15.12:0/1300188150 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fadc41005f0 0x7fadc4196c10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fadb4005850 tx=0x7fadb4004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:32:05.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:05.556+0000 7fadc17fa700 1 -- 172.21.15.12:0/1300188150 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fadb401d070 con 0x7fadc41005f0 2024-02-18T11:32:05.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:05.556+0000 7fadc17fa700 1 -- 172.21.15.12:0/1300188150 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fadb400be40 con 0x7fadc41005f0 2024-02-18T11:32:05.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:05.556+0000 7fadca9c9700 1 -- 172.21.15.12:0/1300188150 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadc4197350 con 0x7fadc41005f0 2024-02-18T11:32:05.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:05.556+0000 7fadc17fa700 1 -- 172.21.15.12:0/1300188150 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fadb4022ea0 con 0x7fadc41005f0 2024-02-18T11:32:05.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:05.556+0000 7fadca9c9700 1 -- 172.21.15.12:0/1300188150 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadc41977f0 con 0x7fadc41005f0 2024-02-18T11:32:05.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:05.557+0000 7fadc17fa700 1 -- 172.21.15.12:0/1300188150 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) v1 ==== 46147+0+0 (secure 0 0 0) 0x7fadb400b960 con 0x7fadc41005f0 2024-02-18T11:32:05.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:05.557+0000 7fadca9c9700 1 -- 172.21.15.12:0/1300188150 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadc41905f0 con 0x7fadc41005f0 2024-02-18T11:32:05.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:05.557+0000 7fadc17fa700 1 --2- 172.21.15.12:0/1300188150 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fadac0391b0 0x7fadac03b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:32:05.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:05.557+0000 7fadc17fa700 1 -- 172.21.15.12:0/1300188150 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fadb404d3d0 con 0x7fadc41005f0 2024-02-18T11:32:05.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:05.557+0000 7fadc37fe700 1 --2- 172.21.15.12:0/1300188150 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fadac0391b0 0x7fadac03b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:32:05.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:05.558+0000 7fadc37fe700 1 --2- 172.21.15.12:0/1300188150 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fadac0391b0 0x7fadac03b670 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fadb8006fd0 tx=0x7fadb8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:32:05.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:05.562+0000 7fadc17fa700 1 -- 172.21.15.12:0/1300188150 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fadb402a870 con 0x7fadc41005f0 2024-02-18T11:32:05.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:05.767+0000 7fadca9c9700 1 -- 172.21.15.12:0/1300188150 --> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7fadc402d810 con 0x7fadac0391b0 2024-02-18T11:32:05.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:05.770+0000 7fadc17fa700 1 -- 172.21.15.12:0/1300188150 <== mgr.14152 v2:172.21.15.12:6800/1153736685 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fadc402d810 con 0x7fadac0391b0 2024-02-18T11:32:05.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:05.772+0000 7fadca9c9700 1 -- 172.21.15.12:0/1300188150 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fadac0391b0 msgr2=0x7fadac03b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:32:05.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:05.772+0000 7fadca9c9700 1 --2- 172.21.15.12:0/1300188150 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fadac0391b0 0x7fadac03b670 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fadb8006fd0 tx=0x7fadb8006e40 comp rx=0 tx=0).stop 2024-02-18T11:32:05.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:05.772+0000 7fadca9c9700 1 -- 172.21.15.12:0/1300188150 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fadc41005f0 msgr2=0x7fadc4196c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:32:05.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:05.772+0000 7fadca9c9700 1 --2- 172.21.15.12:0/1300188150 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fadc41005f0 0x7fadc4196c10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fadb4005850 tx=0x7fadb4004ff0 comp rx=0 tx=0).stop 2024-02-18T11:32:05.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:05.772+0000 7fadca9c9700 1 -- 172.21.15.12:0/1300188150 shutdown_connections 2024-02-18T11:32:05.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:05.772+0000 7fadca9c9700 1 --2- 172.21.15.12:0/1300188150 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fadac0391b0 0x7fadac03b670 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:32:05.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:05.772+0000 7fadca9c9700 1 --2- 172.21.15.12:0/1300188150 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fadc41005f0 0x7fadc4196c10 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:32:05.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:05.772+0000 7fadca9c9700 1 -- 172.21.15.12:0/1300188150 >> 172.21.15.12:0/1300188150 conn(0x7fadc40fbbb0 msgr2=0x7fadc406a070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:32:05.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:05.772+0000 7fadca9c9700 1 -- 172.21.15.12:0/1300188150 shutdown_connections 2024-02-18T11:32:05.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:05.772+0000 7fadca9c9700 1 -- 172.21.15.12:0/1300188150 wait complete. 2024-02-18T11:32:06.578 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi158 2024-02-18T11:32:06.578 DEBUG:teuthology.orchestra.run.smithi158:> set -ex 2024-02-18T11:32:06.578 DEBUG:teuthology.orchestra.run.smithi158:> dd of=/etc/ceph/ceph.conf 2024-02-18T11:32:06.597 DEBUG:teuthology.orchestra.run.smithi158:> set -ex 2024-02-18T11:32:06.598 DEBUG:teuthology.orchestra.run.smithi158:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-18T11:32:06.655 INFO:tasks.cephadm:Adding host smithi158 to orchestrator... 2024-02-18T11:32:06.655 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph orch host add smithi158 2024-02-18T11:32:06.895 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T11:32:07.016 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:06 smithi012 ceph-mon[32460]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:32:07.017 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:06 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:07.017 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:06 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:32:07.017 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:06 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:32:07.017 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:06 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:32:07.017 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:06 smithi012 ceph-mon[32460]: Updating smithi012:/etc/ceph/ceph.conf 2024-02-18T11:32:07.017 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:06 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:07.017 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:06 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:07.017 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:06 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:07.864 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:07 smithi012 ceph-mon[32460]: Updating smithi012:/var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-18T11:32:07.864 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:07 smithi012 ceph-mon[32460]: Updating smithi012:/etc/ceph/ceph.client.admin.keyring 2024-02-18T11:32:07.864 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:07 smithi012 ceph-mon[32460]: Updating smithi012:/var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-18T11:32:08.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:08.319+0000 7febfd1f1700 1 -- 172.21.15.12:0/1399861706 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7febf81028f0 msgr2=0x7febf8102cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:32:08.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:08.319+0000 7febfd1f1700 1 --2- 172.21.15.12:0/1399861706 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7febf81028f0 0x7febf8102cd0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7febf0009910 tx=0x7febf0009c20 comp rx=0 tx=0).stop 2024-02-18T11:32:08.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:08.319+0000 7febfd1f1700 1 -- 172.21.15.12:0/1399861706 shutdown_connections 2024-02-18T11:32:08.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:08.319+0000 7febfd1f1700 1 --2- 172.21.15.12:0/1399861706 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7febf81028f0 0x7febf8102cd0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:32:08.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:08.319+0000 7febfd1f1700 1 -- 172.21.15.12:0/1399861706 >> 172.21.15.12:0/1399861706 conn(0x7febf80fc080 msgr2=0x7febf80fe4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:32:08.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:08.319+0000 7febfd1f1700 1 -- 172.21.15.12:0/1399861706 shutdown_connections 2024-02-18T11:32:08.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:08.319+0000 7febfd1f1700 1 -- 172.21.15.12:0/1399861706 wait complete. 2024-02-18T11:32:08.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:08.320+0000 7febfd1f1700 1 Processor -- start 2024-02-18T11:32:08.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:08.320+0000 7febfd1f1700 1 -- start start 2024-02-18T11:32:08.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:08.320+0000 7febfd1f1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7febf81028f0 0x7febf8100550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:32:08.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:08.320+0000 7febfd1f1700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf81023f0 con 0x7febf81028f0 2024-02-18T11:32:08.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:08.320+0000 7febf6d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7febf81028f0 0x7febf8100550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:32:08.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:08.320+0000 7febf6d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7febf81028f0 0x7febf8100550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35674/0 (socket says 172.21.15.12:35674) 2024-02-18T11:32:08.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:08.320+0000 7febf6d9d700 1 -- 172.21.15.12:0/3366716833 learned_addr learned my addr 172.21.15.12:0/3366716833 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:32:08.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:08.320+0000 7febf6d9d700 1 -- 172.21.15.12:0/3366716833 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febf00095f0 con 0x7febf81028f0 2024-02-18T11:32:08.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:08.320+0000 7febf6d9d700 1 --2- 172.21.15.12:0/3366716833 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7febf81028f0 0x7febf8100550 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7febf0005850 tx=0x7febf0004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:32:08.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:08.321+0000 7febeffff700 1 -- 172.21.15.12:0/3366716833 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7febf001d070 con 0x7febf81028f0 2024-02-18T11:32:08.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:08.321+0000 7febeffff700 1 -- 172.21.15.12:0/3366716833 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7febf000bc30 con 0x7febf81028f0 2024-02-18T11:32:08.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:08.321+0000 7febfd1f1700 1 -- 172.21.15.12:0/3366716833 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febf8100a90 con 0x7febf81028f0 2024-02-18T11:32:08.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:08.321+0000 7febeffff700 1 -- 172.21.15.12:0/3366716833 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7febf0003a20 con 0x7febf81028f0 2024-02-18T11:32:08.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:08.321+0000 7febfd1f1700 1 -- 172.21.15.12:0/3366716833 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febf8100eb0 con 0x7febf81028f0 2024-02-18T11:32:08.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:08.322+0000 7febeffff700 1 -- 172.21.15.12:0/3366716833 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) v1 ==== 46147+0+0 (secure 0 0 0) 0x7febf0003c90 con 0x7febf81028f0 2024-02-18T11:32:08.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:08.322+0000 7febfd1f1700 1 -- 172.21.15.12:0/3366716833 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febf80632e0 con 0x7febf81028f0 2024-02-18T11:32:08.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:08.322+0000 7febeffff700 1 --2- 172.21.15.12:0/3366716833 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7febe40391b0 0x7febe403b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:32:08.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:08.322+0000 7febeffff700 1 -- 172.21.15.12:0/3366716833 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7febf004d0f0 con 0x7febf81028f0 2024-02-18T11:32:08.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:08.322+0000 7febf659c700 1 --2- 172.21.15.12:0/3366716833 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7febe40391b0 0x7febe403b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:32:08.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:08.322+0000 7febf659c700 1 --2- 172.21.15.12:0/3366716833 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7febe40391b0 0x7febe403b670 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7febe0006fd0 tx=0x7febe0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:32:08.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:08.327+0000 7febeffff700 1 -- 172.21.15.12:0/3366716833 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7febf000fd00 con 0x7febf81028f0 2024-02-18T11:32:08.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:08.535+0000 7febfd1f1700 1 -- 172.21.15.12:0/3366716833 --> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi158", "target": ["mon-mgr", ""]}) v1 -- 0x7febf810bbd0 con 0x7febe40391b0 2024-02-18T11:32:09.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:08 smithi012 ceph-mon[32460]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi158", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:32:10.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:09 smithi012 ceph-mon[32460]: Deploying cephadm binary to smithi158 2024-02-18T11:32:12.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:11 smithi012 ceph-mon[32460]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:32:12.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:12.804+0000 7febeffff700 1 -- 172.21.15.12:0/3366716833 <== mgr.14152 v2:172.21.15.12:6800/1153736685 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7febf810bbd0 con 0x7febe40391b0 2024-02-18T11:32:12.801 INFO:teuthology.orchestra.run.smithi012.stdout:Added host 'smithi158' with addr '172.21.15.158' 2024-02-18T11:32:12.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:12.807+0000 7febfd1f1700 1 -- 172.21.15.12:0/3366716833 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7febe40391b0 msgr2=0x7febe403b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:32:12.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:12.807+0000 7febfd1f1700 1 --2- 172.21.15.12:0/3366716833 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7febe40391b0 0x7febe403b670 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7febe0006fd0 tx=0x7febe0006e40 comp rx=0 tx=0).stop 2024-02-18T11:32:12.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:12.807+0000 7febfd1f1700 1 -- 172.21.15.12:0/3366716833 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7febf81028f0 msgr2=0x7febf8100550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:32:12.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:12.807+0000 7febfd1f1700 1 --2- 172.21.15.12:0/3366716833 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7febf81028f0 0x7febf8100550 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7febf0005850 tx=0x7febf0004ff0 comp rx=0 tx=0).stop 2024-02-18T11:32:12.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:12.807+0000 7febfd1f1700 1 -- 172.21.15.12:0/3366716833 shutdown_connections 2024-02-18T11:32:12.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:12.807+0000 7febfd1f1700 1 --2- 172.21.15.12:0/3366716833 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7febe40391b0 0x7febe403b670 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:32:12.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:12.807+0000 7febfd1f1700 1 --2- 172.21.15.12:0/3366716833 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7febf81028f0 0x7febf8100550 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:32:12.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:12.807+0000 7febfd1f1700 1 -- 172.21.15.12:0/3366716833 >> 172.21.15.12:0/3366716833 conn(0x7febf80fc080 msgr2=0x7febf81036a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:32:12.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:12.807+0000 7febfd1f1700 1 -- 172.21.15.12:0/3366716833 shutdown_connections 2024-02-18T11:32:12.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:12.807+0000 7febfd1f1700 1 -- 172.21.15.12:0/3366716833 wait complete. 2024-02-18T11:32:13.348 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph orch host ls --format=json 2024-02-18T11:32:13.585 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T11:32:13.965 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:13 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:13.965 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:13 smithi012 ceph-mon[32460]: Added host smithi158 2024-02-18T11:32:13.965 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:13 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:32:13.965 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:13 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:14.912 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:14 smithi012 ceph-mon[32460]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:32:14.912 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:14 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:14.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:14.944+0000 7f9b61445700 1 -- 172.21.15.12:0/2366631399 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b5c102880 msgr2=0x7f9b5c102c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:32:14.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:14.944+0000 7f9b61445700 1 --2- 172.21.15.12:0/2366631399 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b5c102880 0x7f9b5c102c60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c009910 tx=0x7f9b4c009c20 comp rx=0 tx=0).stop 2024-02-18T11:32:14.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:14.944+0000 7f9b61445700 1 -- 172.21.15.12:0/2366631399 shutdown_connections 2024-02-18T11:32:14.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:14.944+0000 7f9b61445700 1 --2- 172.21.15.12:0/2366631399 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b5c102880 0x7f9b5c102c60 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:32:14.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:14.944+0000 7f9b61445700 1 -- 172.21.15.12:0/2366631399 >> 172.21.15.12:0/2366631399 conn(0x7f9b5c0fc0f0 msgr2=0x7f9b5c0fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:32:14.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:14.944+0000 7f9b61445700 1 -- 172.21.15.12:0/2366631399 shutdown_connections 2024-02-18T11:32:14.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:14.944+0000 7f9b61445700 1 -- 172.21.15.12:0/2366631399 wait complete. 2024-02-18T11:32:14.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:14.945+0000 7f9b61445700 1 Processor -- start 2024-02-18T11:32:14.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:14.945+0000 7f9b61445700 1 -- start start 2024-02-18T11:32:14.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:14.945+0000 7f9b61445700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b5c102880 0x7f9b5c1004e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:32:14.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:14.945+0000 7f9b61445700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5c1023d0 con 0x7f9b5c102880 2024-02-18T11:32:14.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:14.945+0000 7f9b5affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b5c102880 0x7f9b5c1004e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:32:14.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:14.945+0000 7f9b5affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b5c102880 0x7f9b5c1004e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46878/0 (socket says 172.21.15.12:46878) 2024-02-18T11:32:14.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:14.945+0000 7f9b5affd700 1 -- 172.21.15.12:0/2678055117 learned_addr learned my addr 172.21.15.12:0/2678055117 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:32:14.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:14.946+0000 7f9b5affd700 1 -- 172.21.15.12:0/2678055117 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b4c0095f0 con 0x7f9b5c102880 2024-02-18T11:32:14.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:14.946+0000 7f9b5affd700 1 --2- 172.21.15.12:0/2678055117 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b5c102880 0x7f9b5c1004e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c005850 tx=0x7f9b4c004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:32:14.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:14.946+0000 7f9b43fff700 1 -- 172.21.15.12:0/2678055117 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9b4c01d070 con 0x7f9b5c102880 2024-02-18T11:32:14.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:14.946+0000 7f9b61445700 1 -- 172.21.15.12:0/2678055117 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b5c100a20 con 0x7f9b5c102880 2024-02-18T11:32:14.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:14.946+0000 7f9b43fff700 1 -- 172.21.15.12:0/2678055117 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9b4c00bc30 con 0x7f9b5c102880 2024-02-18T11:32:14.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:14.946+0000 7f9b43fff700 1 -- 172.21.15.12:0/2678055117 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9b4c003a20 con 0x7f9b5c102880 2024-02-18T11:32:14.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:14.946+0000 7f9b61445700 1 -- 172.21.15.12:0/2678055117 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b5c100e40 con 0x7f9b5c102880 2024-02-18T11:32:14.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:14.947+0000 7f9b43fff700 1 -- 172.21.15.12:0/2678055117 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) v1 ==== 46147+0+0 (secure 0 0 0) 0x7f9b4c003c90 con 0x7f9b5c102880 2024-02-18T11:32:14.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:14.947+0000 7f9b61445700 1 -- 172.21.15.12:0/2678055117 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b5c0632e0 con 0x7f9b5c102880 2024-02-18T11:32:14.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:14.947+0000 7f9b43fff700 1 --2- 172.21.15.12:0/2678055117 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f9b440391b0 0x7f9b4403b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:32:14.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:14.947+0000 7f9b43fff700 1 -- 172.21.15.12:0/2678055117 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f9b4c04d210 con 0x7f9b5c102880 2024-02-18T11:32:14.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:14.947+0000 7f9b5a7fc700 1 --2- 172.21.15.12:0/2678055117 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f9b440391b0 0x7f9b4403b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:32:14.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:14.947+0000 7f9b5a7fc700 1 --2- 172.21.15.12:0/2678055117 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f9b440391b0 0x7f9b4403b670 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9b50006fd0 tx=0x7f9b50006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:32:14.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:14.952+0000 7f9b43fff700 1 -- 172.21.15.12:0/2678055117 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9b4c00fd00 con 0x7f9b5c102880 2024-02-18T11:32:15.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:15.161+0000 7f9b61445700 1 -- 172.21.15.12:0/2678055117 --> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9b5c101160 con 0x7f9b440391b0 2024-02-18T11:32:15.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:15.162+0000 7f9b43fff700 1 -- 172.21.15.12:0/2678055117 <== mgr.14152 v2:172.21.15.12:6800/1153736685 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+160 (secure 0 0 0) 0x7f9b5c101160 con 0x7f9b440391b0 2024-02-18T11:32:15.159 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:32:15.159 INFO:teuthology.orchestra.run.smithi012.stdout:[{"addr": "172.21.15.12", "hostname": "smithi012", "labels": [], "status": ""}, {"addr": "172.21.15.158", "hostname": "smithi158", "labels": [], "status": ""}] 2024-02-18T11:32:15.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:15.164+0000 7f9b61445700 1 -- 172.21.15.12:0/2678055117 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f9b440391b0 msgr2=0x7f9b4403b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:32:15.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:15.164+0000 7f9b61445700 1 --2- 172.21.15.12:0/2678055117 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f9b440391b0 0x7f9b4403b670 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9b50006fd0 tx=0x7f9b50006e40 comp rx=0 tx=0).stop 2024-02-18T11:32:15.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:15.164+0000 7f9b61445700 1 -- 172.21.15.12:0/2678055117 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b5c102880 msgr2=0x7f9b5c1004e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:32:15.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:15.164+0000 7f9b61445700 1 --2- 172.21.15.12:0/2678055117 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b5c102880 0x7f9b5c1004e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c005850 tx=0x7f9b4c004ff0 comp rx=0 tx=0).stop 2024-02-18T11:32:15.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:15.164+0000 7f9b61445700 1 -- 172.21.15.12:0/2678055117 shutdown_connections 2024-02-18T11:32:15.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:15.164+0000 7f9b61445700 1 --2- 172.21.15.12:0/2678055117 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f9b440391b0 0x7f9b4403b670 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:32:15.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:15.164+0000 7f9b61445700 1 --2- 172.21.15.12:0/2678055117 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b5c102880 0x7f9b5c1004e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:32:15.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:15.164+0000 7f9b61445700 1 -- 172.21.15.12:0/2678055117 >> 172.21.15.12:0/2678055117 conn(0x7f9b5c0fc0f0 msgr2=0x7f9b5c103630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:32:15.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:15.164+0000 7f9b61445700 1 -- 172.21.15.12:0/2678055117 shutdown_connections 2024-02-18T11:32:15.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:15.164+0000 7f9b61445700 1 -- 172.21.15.12:0/2678055117 wait complete. 2024-02-18T11:32:15.759 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi188 2024-02-18T11:32:15.760 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-02-18T11:32:15.760 DEBUG:teuthology.orchestra.run.smithi188:> dd of=/etc/ceph/ceph.conf 2024-02-18T11:32:15.779 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-02-18T11:32:15.779 DEBUG:teuthology.orchestra.run.smithi188:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-18T11:32:15.836 INFO:tasks.cephadm:Adding host smithi188 to orchestrator... 2024-02-18T11:32:15.836 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph orch host add smithi188 2024-02-18T11:32:16.075 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T11:32:16.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:16 smithi012 ceph-mon[32460]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:32:16.486 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:16 smithi012 ceph-mon[32460]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T11:32:16.486 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:16 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:17.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:17.586+0000 7fbcc0307700 1 -- 172.21.15.12:0/1143708198 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbcb8107690 msgr2=0x7fbcb8107a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:32:17.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:17.586+0000 7fbcc0307700 1 --2- 172.21.15.12:0/1143708198 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbcb8107690 0x7fbcb8107a70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbcb4009910 tx=0x7fbcb4009c20 comp rx=0 tx=0).stop 2024-02-18T11:32:17.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:17.586+0000 7fbcc0307700 1 -- 172.21.15.12:0/1143708198 shutdown_connections 2024-02-18T11:32:17.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:17.586+0000 7fbcc0307700 1 --2- 172.21.15.12:0/1143708198 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbcb8107690 0x7fbcb8107a70 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:32:17.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:17.586+0000 7fbcc0307700 1 -- 172.21.15.12:0/1143708198 >> 172.21.15.12:0/1143708198 conn(0x7fbcb80fb800 msgr2=0x7fbcb80fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:32:17.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:17.586+0000 7fbcc0307700 1 -- 172.21.15.12:0/1143708198 shutdown_connections 2024-02-18T11:32:17.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:17.586+0000 7fbcc0307700 1 -- 172.21.15.12:0/1143708198 wait complete. 2024-02-18T11:32:17.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:17.586+0000 7fbcc0307700 1 Processor -- start 2024-02-18T11:32:17.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:17.586+0000 7fbcc0307700 1 -- start start 2024-02-18T11:32:17.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:17.587+0000 7fbcc0307700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbcb8107690 0x7fbcb8196fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:32:17.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:17.587+0000 7fbcc0307700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb8197500 con 0x7fbcb8107690 2024-02-18T11:32:17.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:17.587+0000 7fbcbe0a3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbcb8107690 0x7fbcb8196fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:32:17.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:17.587+0000 7fbcbe0a3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbcb8107690 0x7fbcb8196fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46904/0 (socket says 172.21.15.12:46904) 2024-02-18T11:32:17.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:17.587+0000 7fbcbe0a3700 1 -- 172.21.15.12:0/1913059263 learned_addr learned my addr 172.21.15.12:0/1913059263 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:32:17.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:17.587+0000 7fbcbe0a3700 1 -- 172.21.15.12:0/1913059263 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcb40095f0 con 0x7fbcb8107690 2024-02-18T11:32:17.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:17.587+0000 7fbcbe0a3700 1 --2- 172.21.15.12:0/1913059263 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbcb8107690 0x7fbcb8196fc0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbcb4006010 tx=0x7fbcb4004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:32:17.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:17.587+0000 7fbcaf7fe700 1 -- 172.21.15.12:0/1913059263 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbcb401d070 con 0x7fbcb8107690 2024-02-18T11:32:17.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:17.587+0000 7fbcaf7fe700 1 -- 172.21.15.12:0/1913059263 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbcb400bbd0 con 0x7fbcb8107690 2024-02-18T11:32:17.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:17.587+0000 7fbcc0307700 1 -- 172.21.15.12:0/1913059263 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcb8197760 con 0x7fbcb8107690 2024-02-18T11:32:17.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:17.587+0000 7fbcaf7fe700 1 -- 172.21.15.12:0/1913059263 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbcb4003880 con 0x7fbcb8107690 2024-02-18T11:32:17.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:17.587+0000 7fbcc0307700 1 -- 172.21.15.12:0/1913059263 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcb8197be0 con 0x7fbcb8107690 2024-02-18T11:32:17.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:17.588+0000 7fbcc0307700 1 -- 172.21.15.12:0/1913059263 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcb80632e0 con 0x7fbcb8107690 2024-02-18T11:32:17.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:17.588+0000 7fbcaf7fe700 1 -- 172.21.15.12:0/1913059263 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) v1 ==== 46147+0+0 (secure 0 0 0) 0x7fbcb400bd40 con 0x7fbcb8107690 2024-02-18T11:32:17.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:17.588+0000 7fbcaf7fe700 1 --2- 172.21.15.12:0/1913059263 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fbca40391b0 0x7fbca403b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:32:17.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:17.589+0000 7fbcaf7fe700 1 -- 172.21.15.12:0/1913059263 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fbcb404d020 con 0x7fbcb8107690 2024-02-18T11:32:17.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:17.589+0000 7fbcbd8a2700 1 --2- 172.21.15.12:0/1913059263 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fbca40391b0 0x7fbca403b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:32:17.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:17.589+0000 7fbcbd8a2700 1 --2- 172.21.15.12:0/1913059263 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fbca40391b0 0x7fbca403b670 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fbca8006fd0 tx=0x7fbca8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:32:17.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:17.593+0000 7fbcaf7fe700 1 -- 172.21.15.12:0/1913059263 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbcb402ac00 con 0x7fbcb8107690 2024-02-18T11:32:17.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:17.802+0000 7fbcc0307700 1 -- 172.21.15.12:0/1913059263 --> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi188", "target": ["mon-mgr", ""]}) v1 -- 0x7fbcb810bc80 con 0x7fbca40391b0 2024-02-18T11:32:18.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:18 smithi012 ceph-mon[32460]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:32:18.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:18 smithi012 ceph-mon[32460]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi188", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:32:18.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:18 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:18.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:18 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:18.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:18 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:18.675 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:18 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:18.675 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:18 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi158", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:32:18.675 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:18 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:32:18.675 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:18 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:32:19.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:19 smithi012 ceph-mon[32460]: Updating smithi158:/etc/ceph/ceph.conf 2024-02-18T11:32:19.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:19 smithi012 ceph-mon[32460]: Deploying cephadm binary to smithi188 2024-02-18T11:32:19.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:19 smithi012 ceph-mon[32460]: Updating smithi158:/var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-18T11:32:19.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:19 smithi012 ceph-mon[32460]: Updating smithi158:/etc/ceph/ceph.client.admin.keyring 2024-02-18T11:32:20.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:20 smithi012 ceph-mon[32460]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:32:22.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:22.057+0000 7fbcaf7fe700 1 -- 172.21.15.12:0/1913059263 <== mgr.14152 v2:172.21.15.12:6800/1153736685 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fbcb810bc80 con 0x7fbca40391b0 2024-02-18T11:32:22.054 INFO:teuthology.orchestra.run.smithi012.stdout:Added host 'smithi188' with addr '172.21.15.188' 2024-02-18T11:32:22.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:22.059+0000 7fbcc0307700 1 -- 172.21.15.12:0/1913059263 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fbca40391b0 msgr2=0x7fbca403b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:32:22.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:22.059+0000 7fbcc0307700 1 --2- 172.21.15.12:0/1913059263 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fbca40391b0 0x7fbca403b670 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fbca8006fd0 tx=0x7fbca8006e40 comp rx=0 tx=0).stop 2024-02-18T11:32:22.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:22.059+0000 7fbcc0307700 1 -- 172.21.15.12:0/1913059263 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbcb8107690 msgr2=0x7fbcb8196fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:32:22.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:22.059+0000 7fbcc0307700 1 --2- 172.21.15.12:0/1913059263 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbcb8107690 0x7fbcb8196fc0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbcb4006010 tx=0x7fbcb4004ff0 comp rx=0 tx=0).stop 2024-02-18T11:32:22.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:22.059+0000 7fbcc0307700 1 -- 172.21.15.12:0/1913059263 shutdown_connections 2024-02-18T11:32:22.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:22.059+0000 7fbcc0307700 1 --2- 172.21.15.12:0/1913059263 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fbca40391b0 0x7fbca403b670 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:32:22.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:22.059+0000 7fbcc0307700 1 --2- 172.21.15.12:0/1913059263 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbcb8107690 0x7fbcb8196fc0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:32:22.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:22.059+0000 7fbcc0307700 1 -- 172.21.15.12:0/1913059263 >> 172.21.15.12:0/1913059263 conn(0x7fbcb80fb800 msgr2=0x7fbcb80fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:32:22.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:22.059+0000 7fbcc0307700 1 -- 172.21.15.12:0/1913059263 shutdown_connections 2024-02-18T11:32:22.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:22.059+0000 7fbcc0307700 1 -- 172.21.15.12:0/1913059263 wait complete. 2024-02-18T11:32:22.672 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph orch host ls --format=json 2024-02-18T11:32:22.911 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T11:32:22.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:22 smithi012 ceph-mon[32460]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:32:22.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:22 smithi012 ceph-mon[32460]: Updating smithi158:/var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-18T11:32:22.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:22 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:22.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:22 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:22.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:22 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:22.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:22 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:22.925 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:22 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:32:22.925 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:22 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:23.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:23 smithi012 ceph-mon[32460]: Added host smithi188 2024-02-18T11:32:23.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:23 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:24.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:24.277+0000 7fef7b50a700 1 -- 172.21.15.12:0/3948999908 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef741028e0 msgr2=0x7fef74102cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:32:24.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:24.277+0000 7fef7b50a700 1 --2- 172.21.15.12:0/3948999908 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef741028e0 0x7fef74102cc0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fef6c009910 tx=0x7fef6c009c20 comp rx=0 tx=0).stop 2024-02-18T11:32:24.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:24.277+0000 7fef7b50a700 1 -- 172.21.15.12:0/3948999908 shutdown_connections 2024-02-18T11:32:24.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:24.277+0000 7fef7b50a700 1 --2- 172.21.15.12:0/3948999908 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef741028e0 0x7fef74102cc0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:32:24.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:24.277+0000 7fef7b50a700 1 -- 172.21.15.12:0/3948999908 >> 172.21.15.12:0/3948999908 conn(0x7fef740771f0 msgr2=0x7fef740fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:32:24.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:24.277+0000 7fef7b50a700 1 -- 172.21.15.12:0/3948999908 shutdown_connections 2024-02-18T11:32:24.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:24.277+0000 7fef7b50a700 1 -- 172.21.15.12:0/3948999908 wait complete. 2024-02-18T11:32:24.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:24.278+0000 7fef7b50a700 1 Processor -- start 2024-02-18T11:32:24.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:24.278+0000 7fef7b50a700 1 -- start start 2024-02-18T11:32:24.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:24.278+0000 7fef7b50a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef741028e0 0x7fef741970a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:32:24.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:24.278+0000 7fef7b50a700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef741975e0 con 0x7fef741028e0 2024-02-18T11:32:24.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:24.278+0000 7fef792a6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef741028e0 0x7fef741970a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:32:24.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:24.278+0000 7fef792a6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef741028e0 0x7fef741970a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54600/0 (socket says 172.21.15.12:54600) 2024-02-18T11:32:24.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:24.278+0000 7fef792a6700 1 -- 172.21.15.12:0/3853909 learned_addr learned my addr 172.21.15.12:0/3853909 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:32:24.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:24.279+0000 7fef792a6700 1 -- 172.21.15.12:0/3853909 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef6c0095f0 con 0x7fef741028e0 2024-02-18T11:32:24.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:24.279+0000 7fef792a6700 1 --2- 172.21.15.12:0/3853909 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef741028e0 0x7fef741970a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fef6c005850 tx=0x7fef6c004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:32:24.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:24.279+0000 7fef727fc700 1 -- 172.21.15.12:0/3853909 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fef6c01d070 con 0x7fef741028e0 2024-02-18T11:32:24.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:24.279+0000 7fef7b50a700 1 -- 172.21.15.12:0/3853909 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef741977e0 con 0x7fef741028e0 2024-02-18T11:32:24.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:24.279+0000 7fef727fc700 1 -- 172.21.15.12:0/3853909 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fef6c00bc30 con 0x7fef741028e0 2024-02-18T11:32:24.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:24.279+0000 7fef727fc700 1 -- 172.21.15.12:0/3853909 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fef6c003a20 con 0x7fef741028e0 2024-02-18T11:32:24.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:24.279+0000 7fef7b50a700 1 -- 172.21.15.12:0/3853909 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef74197c00 con 0x7fef741028e0 2024-02-18T11:32:24.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:24.280+0000 7fef727fc700 1 -- 172.21.15.12:0/3853909 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) v1 ==== 46147+0+0 (secure 0 0 0) 0x7fef6c003c90 con 0x7fef741028e0 2024-02-18T11:32:24.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:24.280+0000 7fef727fc700 1 --2- 172.21.15.12:0/3853909 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fef5c0391b0 0x7fef5c03b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:32:24.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:24.280+0000 7fef727fc700 1 -- 172.21.15.12:0/3853909 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fef6c04d0b0 con 0x7fef741028e0 2024-02-18T11:32:24.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:24.280+0000 7fef7b50a700 1 -- 172.21.15.12:0/3853909 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef740698e0 con 0x7fef741028e0 2024-02-18T11:32:24.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:24.280+0000 7fef78aa5700 1 --2- 172.21.15.12:0/3853909 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fef5c0391b0 0x7fef5c03b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:32:24.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:24.280+0000 7fef78aa5700 1 --2- 172.21.15.12:0/3853909 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fef5c0391b0 0x7fef5c03b670 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fef68006fd0 tx=0x7fef68006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:32:24.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:24.285+0000 7fef727fc700 1 -- 172.21.15.12:0/3853909 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fef6c026020 con 0x7fef741028e0 2024-02-18T11:32:24.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:24.494+0000 7fef7b50a700 1 -- 172.21.15.12:0/3853909 --> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fef741984b0 con 0x7fef5c0391b0 2024-02-18T11:32:24.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:24.495+0000 7fef727fc700 1 -- 172.21.15.12:0/3853909 <== mgr.14152 v2:172.21.15.12:6800/1153736685 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+240 (secure 0 0 0) 0x7fef741984b0 con 0x7fef5c0391b0 2024-02-18T11:32:24.492 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:32:24.492 INFO:teuthology.orchestra.run.smithi012.stdout:[{"addr": "172.21.15.12", "hostname": "smithi012", "labels": [], "status": ""}, {"addr": "172.21.15.158", "hostname": "smithi158", "labels": [], "status": ""}, {"addr": "172.21.15.188", "hostname": "smithi188", "labels": [], "status": ""}] 2024-02-18T11:32:24.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:24.497+0000 7fef7b50a700 1 -- 172.21.15.12:0/3853909 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fef5c0391b0 msgr2=0x7fef5c03b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:32:24.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:24.497+0000 7fef7b50a700 1 --2- 172.21.15.12:0/3853909 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fef5c0391b0 0x7fef5c03b670 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fef68006fd0 tx=0x7fef68006e40 comp rx=0 tx=0).stop 2024-02-18T11:32:24.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:24.497+0000 7fef7b50a700 1 -- 172.21.15.12:0/3853909 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef741028e0 msgr2=0x7fef741970a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:32:24.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:24.497+0000 7fef7b50a700 1 --2- 172.21.15.12:0/3853909 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef741028e0 0x7fef741970a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fef6c005850 tx=0x7fef6c004ff0 comp rx=0 tx=0).stop 2024-02-18T11:32:24.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:24.497+0000 7fef7b50a700 1 -- 172.21.15.12:0/3853909 shutdown_connections 2024-02-18T11:32:24.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:24.497+0000 7fef7b50a700 1 --2- 172.21.15.12:0/3853909 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fef5c0391b0 0x7fef5c03b670 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:32:24.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:24.497+0000 7fef7b50a700 1 --2- 172.21.15.12:0/3853909 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef741028e0 0x7fef741970a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:32:24.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:24.497+0000 7fef7b50a700 1 -- 172.21.15.12:0/3853909 >> 172.21.15.12:0/3853909 conn(0x7fef740771f0 msgr2=0x7fef74103690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:32:24.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:24.498+0000 7fef7b50a700 1 -- 172.21.15.12:0/3853909 shutdown_connections 2024-02-18T11:32:24.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:24.498+0000 7fef7b50a700 1 -- 172.21.15.12:0/3853909 wait complete. 2024-02-18T11:32:24.865 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:24 smithi012 ceph-mon[32460]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:32:25.152 INFO:tasks.cephadm:Setting crush tunables to default 2024-02-18T11:32:25.152 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph osd crush tunables default 2024-02-18T11:32:25.390 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T11:32:25.793 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:25 smithi012 ceph-mon[32460]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T11:32:25.793 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:25 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:26.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:26.857+0000 7f825ebc6700 1 -- 172.21.15.12:0/729244392 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8258107690 msgr2=0x7f8258107a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:32:26.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:26.857+0000 7f825ebc6700 1 --2- 172.21.15.12:0/729244392 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8258107690 0x7f8258107a70 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f824c009910 tx=0x7f824c009c20 comp rx=0 tx=0).stop 2024-02-18T11:32:26.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:26.857+0000 7f825ebc6700 1 -- 172.21.15.12:0/729244392 shutdown_connections 2024-02-18T11:32:26.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:26.857+0000 7f825ebc6700 1 --2- 172.21.15.12:0/729244392 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8258107690 0x7f8258107a70 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:32:26.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:26.857+0000 7f825ebc6700 1 -- 172.21.15.12:0/729244392 >> 172.21.15.12:0/729244392 conn(0x7f82580fd1c0 msgr2=0x7f82580ff5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:32:26.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:26.857+0000 7f825ebc6700 1 -- 172.21.15.12:0/729244392 shutdown_connections 2024-02-18T11:32:26.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:26.857+0000 7f825ebc6700 1 -- 172.21.15.12:0/729244392 wait complete. 2024-02-18T11:32:26.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:26.857+0000 7f825ebc6700 1 Processor -- start 2024-02-18T11:32:26.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:26.857+0000 7f825ebc6700 1 -- start start 2024-02-18T11:32:26.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:26.858+0000 7f825ebc6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8258107690 0x7f8258196fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:32:26.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:26.858+0000 7f825ebc6700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8258197520 con 0x7f8258107690 2024-02-18T11:32:26.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:26.858+0000 7f825c962700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8258107690 0x7f8258196fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:32:26.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:26.858+0000 7f825c962700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8258107690 0x7f8258196fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54608/0 (socket says 172.21.15.12:54608) 2024-02-18T11:32:26.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:26.858+0000 7f825c962700 1 -- 172.21.15.12:0/2854546643 learned_addr learned my addr 172.21.15.12:0/2854546643 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:32:26.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:26.858+0000 7f825c962700 1 -- 172.21.15.12:0/2854546643 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f824c0095f0 con 0x7f8258107690 2024-02-18T11:32:26.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:26.858+0000 7f825c962700 1 --2- 172.21.15.12:0/2854546643 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8258107690 0x7f8258196fe0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f824c004f40 tx=0x7f824c005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:32:26.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:26.858+0000 7f8255ffb700 1 -- 172.21.15.12:0/2854546643 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f824c01d070 con 0x7f8258107690 2024-02-18T11:32:26.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:26.858+0000 7f8255ffb700 1 -- 172.21.15.12:0/2854546643 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f824c00bbd0 con 0x7f8258107690 2024-02-18T11:32:26.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:26.858+0000 7f825ebc6700 1 -- 172.21.15.12:0/2854546643 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8258197780 con 0x7f8258107690 2024-02-18T11:32:26.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:26.858+0000 7f8255ffb700 1 -- 172.21.15.12:0/2854546643 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f824c003900 con 0x7f8258107690 2024-02-18T11:32:26.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:26.859+0000 7f825ebc6700 1 -- 172.21.15.12:0/2854546643 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8258197c00 con 0x7f8258107690 2024-02-18T11:32:26.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:26.859+0000 7f8255ffb700 1 -- 172.21.15.12:0/2854546643 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) v1 ==== 46147+0+0 (secure 0 0 0) 0x7f824c003b70 con 0x7f8258107690 2024-02-18T11:32:26.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:26.859+0000 7f825ebc6700 1 -- 172.21.15.12:0/2854546643 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8258102dd0 con 0x7f8258107690 2024-02-18T11:32:26.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:26.860+0000 7f8255ffb700 1 --2- 172.21.15.12:0/2854546643 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f8248039210 0x7f824803b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:32:26.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:26.860+0000 7f8255ffb700 1 -- 172.21.15.12:0/2854546643 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f824c04d2e0 con 0x7f8258107690 2024-02-18T11:32:26.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:26.860+0000 7f8257fff700 1 --2- 172.21.15.12:0/2854546643 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f8248039210 0x7f824803b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:32:26.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:26.860+0000 7f8257fff700 1 --2- 172.21.15.12:0/2854546643 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f8248039210 0x7f824803b6d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8244006fd0 tx=0x7f8244006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:32:26.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:26.866+0000 7f8255ffb700 1 -- 172.21.15.12:0/2854546643 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f824c029950 con 0x7f8258107690 2024-02-18T11:32:26.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:26 smithi012 ceph-mon[32460]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:32:27.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:27.074+0000 7f825ebc6700 1 -- 172.21.15.12:0/2854546643 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f82580632e0 con 0x7f8258107690 2024-02-18T11:32:27.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:27.642+0000 7f8255ffb700 1 -- 172.21.15.12:0/2854546643 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f824c026070 con 0x7f8258107690 2024-02-18T11:32:27.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:27.645+0000 7f825ebc6700 1 -- 172.21.15.12:0/2854546643 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f8248039210 msgr2=0x7f824803b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:32:27.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:27.645+0000 7f825ebc6700 1 --2- 172.21.15.12:0/2854546643 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f8248039210 0x7f824803b6d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8244006fd0 tx=0x7f8244006e40 comp rx=0 tx=0).stop 2024-02-18T11:32:27.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:27.645+0000 7f825ebc6700 1 -- 172.21.15.12:0/2854546643 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8258107690 msgr2=0x7f8258196fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:32:27.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:27.645+0000 7f825ebc6700 1 --2- 172.21.15.12:0/2854546643 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8258107690 0x7f8258196fe0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f824c004f40 tx=0x7f824c005020 comp rx=0 tx=0).stop 2024-02-18T11:32:27.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:27.645+0000 7f825ebc6700 1 -- 172.21.15.12:0/2854546643 shutdown_connections 2024-02-18T11:32:27.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:27.645+0000 7f825ebc6700 1 --2- 172.21.15.12:0/2854546643 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f8248039210 0x7f824803b6d0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:32:27.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:27.645+0000 7f825ebc6700 1 --2- 172.21.15.12:0/2854546643 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8258107690 0x7f8258196fe0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:32:27.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:27.645+0000 7f825ebc6700 1 -- 172.21.15.12:0/2854546643 >> 172.21.15.12:0/2854546643 conn(0x7f82580fd1c0 msgr2=0x7f82581091b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:32:27.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:27.645+0000 7f825ebc6700 1 -- 172.21.15.12:0/2854546643 shutdown_connections 2024-02-18T11:32:27.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:27.645+0000 7f825ebc6700 1 -- 172.21.15.12:0/2854546643 wait complete. 2024-02-18T11:32:27.642 INFO:teuthology.orchestra.run.smithi012.stderr:adjusted tunables profile to default 2024-02-18T11:32:27.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:27 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/2854546643' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-02-18T11:32:28.239 INFO:tasks.cephadm:Adding mon.a on smithi012 2024-02-18T11:32:28.239 INFO:tasks.cephadm:Adding mon.b on smithi158 2024-02-18T11:32:28.239 INFO:tasks.cephadm:Adding mon.c on smithi188 2024-02-18T11:32:28.239 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph orch apply mon '3;smithi012:172.21.15.12=a;smithi158:172.21.15.158=b;smithi188:172.21.15.188=c' 2024-02-18T11:32:28.473 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-18T11:32:28.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:28 smithi012 ceph-mon[32460]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:32:28.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:28 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/2854546643' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-02-18T11:32:28.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:28 smithi012 ceph-mon[32460]: osdmap e4: 0 total, 0 up, 0 in 2024-02-18T11:32:28.925 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:28 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:28.925 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:28 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:28.925 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:28 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:28.925 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:28 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:28.925 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:28 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi188", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:32:28.925 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:28 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:32:28.925 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:28 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:32:28.925 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:28 smithi012 ceph-mon[32460]: Updating smithi188:/etc/ceph/ceph.conf 2024-02-18T11:32:28.925 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:28 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:28.925 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:28 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:28.925 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:28 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:29.831 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:29.834+0000 7fadf8660700 1 -- 172.21.15.188:0/956116027 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fadf0107690 msgr2=0x7fadf0107a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:32:29.831 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:29.834+0000 7fadf8660700 1 --2- 172.21.15.188:0/956116027 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fadf0107690 0x7fadf0107a70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fade4009910 tx=0x7fade4009c20 comp rx=0 tx=0).stop 2024-02-18T11:32:29.831 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:29.834+0000 7fadf8660700 1 -- 172.21.15.188:0/956116027 shutdown_connections 2024-02-18T11:32:29.831 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:29.834+0000 7fadf8660700 1 --2- 172.21.15.188:0/956116027 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fadf0107690 0x7fadf0107a70 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:32:29.831 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:29.834+0000 7fadf8660700 1 -- 172.21.15.188:0/956116027 >> 172.21.15.188:0/956116027 conn(0x7fadf00fd1c0 msgr2=0x7fadf00ff5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:32:29.831 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:29.834+0000 7fadf8660700 1 -- 172.21.15.188:0/956116027 shutdown_connections 2024-02-18T11:32:29.831 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:29.834+0000 7fadf8660700 1 -- 172.21.15.188:0/956116027 wait complete. 2024-02-18T11:32:29.831 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:29.834+0000 7fadf8660700 1 Processor -- start 2024-02-18T11:32:29.831 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:29.834+0000 7fadf8660700 1 -- start start 2024-02-18T11:32:29.831 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:29.835+0000 7fadf8660700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fadf0107690 0x7fadf0196fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:32:29.831 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:29.835+0000 7fadf8660700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf0197500 con 0x7fadf0107690 2024-02-18T11:32:29.831 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:29.835+0000 7fadf63fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fadf0107690 0x7fadf0196fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:32:29.832 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:29.835+0000 7fadf63fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fadf0107690 0x7fadf0196fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.188:49260/0 (socket says 172.21.15.188:49260) 2024-02-18T11:32:29.832 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:29.835+0000 7fadf63fc700 1 -- 172.21.15.188:0/270495859 learned_addr learned my addr 172.21.15.188:0/270495859 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-02-18T11:32:29.833 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:29.835+0000 7fadf63fc700 1 -- 172.21.15.188:0/270495859 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fade40095f0 con 0x7fadf0107690 2024-02-18T11:32:29.833 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:29.835+0000 7fadf63fc700 1 --2- 172.21.15.188:0/270495859 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fadf0107690 0x7fadf0196fc0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fade4004f40 tx=0x7fade4005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:32:29.834 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:29.836+0000 7fade37fe700 1 -- 172.21.15.188:0/270495859 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fade401d070 con 0x7fadf0107690 2024-02-18T11:32:29.834 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:29.836+0000 7fadf8660700 1 -- 172.21.15.188:0/270495859 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadf0197760 con 0x7fadf0107690 2024-02-18T11:32:29.834 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:29.836+0000 7fade37fe700 1 -- 172.21.15.188:0/270495859 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fade400bbd0 con 0x7fadf0107690 2024-02-18T11:32:29.834 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:29.836+0000 7fade37fe700 1 -- 172.21.15.188:0/270495859 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fade4003900 con 0x7fadf0107690 2024-02-18T11:32:29.834 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:29.836+0000 7fadf8660700 1 -- 172.21.15.188:0/270495859 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadf0197be0 con 0x7fadf0107690 2024-02-18T11:32:29.834 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:29.836+0000 7fadf8660700 1 -- 172.21.15.188:0/270495859 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadf0102dd0 con 0x7fadf0107690 2024-02-18T11:32:29.834 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:29.837+0000 7fade37fe700 1 -- 172.21.15.188:0/270495859 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) v1 ==== 46147+0+0 (secure 0 0 0) 0x7fade400bd40 con 0x7fadf0107690 2024-02-18T11:32:29.834 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:29.837+0000 7fade37fe700 1 --2- 172.21.15.188:0/270495859 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7faddc0391b0 0x7faddc03b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:32:29.834 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:29.837+0000 7fade37fe700 1 -- 172.21.15.188:0/270495859 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fade404d0f0 con 0x7fadf0107690 2024-02-18T11:32:29.834 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:29.837+0000 7fadf5bfb700 1 --2- 172.21.15.188:0/270495859 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7faddc0391b0 0x7faddc03b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:32:29.834 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:29.837+0000 7fadf5bfb700 1 --2- 172.21.15.188:0/270495859 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7faddc0391b0 0x7faddc03b670 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fadec006fd0 tx=0x7fadec006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:32:29.838 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:29.842+0000 7fade37fe700 1 -- 172.21.15.188:0/270495859 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fade4029360 con 0x7fadf0107690 2024-02-18T11:32:29.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:29 smithi012 ceph-mon[32460]: Updating smithi188:/var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-18T11:32:29.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:29 smithi012 ceph-mon[32460]: Updating smithi188:/etc/ceph/ceph.client.admin.keyring 2024-02-18T11:32:29.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:29 smithi012 ceph-mon[32460]: Updating smithi188:/var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-18T11:32:30.039 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:30.043+0000 7fadf8660700 1 -- 172.21.15.188:0/270495859 --> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi012:172.21.15.12=a;smithi158:172.21.15.158=b;smithi188:172.21.15.188=c", "target": ["mon-mgr", ""]}) v1 -- 0x7fadf010bca0 con 0x7faddc0391b0 2024-02-18T11:32:30.047 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:30.050+0000 7fade37fe700 1 -- 172.21.15.188:0/270495859 <== mgr.14152 v2:172.21.15.12:6800/1153736685 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fadf010bca0 con 0x7faddc0391b0 2024-02-18T11:32:30.048 INFO:teuthology.orchestra.run.smithi188.stdout:Scheduled mon update... 2024-02-18T11:32:30.048 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:30.052+0000 7fadf8660700 1 -- 172.21.15.188:0/270495859 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7faddc0391b0 msgr2=0x7faddc03b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:32:30.048 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:30.052+0000 7fadf8660700 1 --2- 172.21.15.188:0/270495859 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7faddc0391b0 0x7faddc03b670 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fadec006fd0 tx=0x7fadec006e40 comp rx=0 tx=0).stop 2024-02-18T11:32:30.049 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:30.052+0000 7fadf8660700 1 -- 172.21.15.188:0/270495859 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fadf0107690 msgr2=0x7fadf0196fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:32:30.049 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:30.052+0000 7fadf8660700 1 --2- 172.21.15.188:0/270495859 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fadf0107690 0x7fadf0196fc0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fade4004f40 tx=0x7fade4005020 comp rx=0 tx=0).stop 2024-02-18T11:32:30.049 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:30.052+0000 7fadf8660700 1 -- 172.21.15.188:0/270495859 shutdown_connections 2024-02-18T11:32:30.049 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:30.052+0000 7fadf8660700 1 --2- 172.21.15.188:0/270495859 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7faddc0391b0 0x7faddc03b670 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:32:30.049 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:30.052+0000 7fadf8660700 1 --2- 172.21.15.188:0/270495859 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fadf0107690 0x7fadf0196fc0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:32:30.049 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:30.052+0000 7fadf8660700 1 -- 172.21.15.188:0/270495859 >> 172.21.15.188:0/270495859 conn(0x7fadf00fd1c0 msgr2=0x7fadf0109190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:32:30.049 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:30.052+0000 7fadf8660700 1 -- 172.21.15.188:0/270495859 shutdown_connections 2024-02-18T11:32:30.049 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:30.052+0000 7fadf8660700 1 -- 172.21.15.188:0/270495859 wait complete. 2024-02-18T11:32:30.735 DEBUG:teuthology.orchestra.run.smithi158:mon.b> sudo journalctl -f -n 0 -u ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@mon.b.service 2024-02-18T11:32:30.738 DEBUG:teuthology.orchestra.run.smithi188:mon.c> sudo journalctl -f -n 0 -u ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@mon.c.service 2024-02-18T11:32:30.740 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-18T11:32:30.740 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-18T11:32:30.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:30 smithi012 ceph-mon[32460]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:32:30.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:30 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:30.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:30 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:32:30.925 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:30 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:32:30.925 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:30 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:32:30.925 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:30 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:30.925 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:30 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-18T11:32:30.925 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:30 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:32:31.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:31 smithi012 ceph-mon[32460]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi012:172.21.15.12=a;smithi158:172.21.15.158=b;smithi188:172.21.15.188=c", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:32:31.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:31 smithi012 ceph-mon[32460]: Saving service mon spec with placement smithi012:172.21.15.12=a;smithi158:172.21.15.158=b;smithi188:172.21.15.188=c;count:3 2024-02-18T11:32:31.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:31 smithi012 ceph-mon[32460]: Deploying daemon mon.c on smithi188 2024-02-18T11:32:32.333 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-18T11:32:32.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:32 smithi012 ceph-mon[32460]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:32:34.551 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:34.553+0000 7feac2c69700 1 -- 172.21.15.188:0/3173290694 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feabc106640 msgr2=0x7feabc106a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:32:34.551 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:34.553+0000 7feac2c69700 1 --2- 172.21.15.188:0/3173290694 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feabc106640 0x7feabc106a20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7feab0009910 tx=0x7feab0009c20 comp rx=0 tx=0).stop 2024-02-18T11:32:34.551 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:34.553+0000 7feac2c69700 1 -- 172.21.15.188:0/3173290694 shutdown_connections 2024-02-18T11:32:34.552 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:34.553+0000 7feac2c69700 1 --2- 172.21.15.188:0/3173290694 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feabc106640 0x7feabc106a20 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:32:34.552 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:34.553+0000 7feac2c69700 1 -- 172.21.15.188:0/3173290694 >> 172.21.15.188:0/3173290694 conn(0x7feabc0fc060 msgr2=0x7feabc0fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:32:34.552 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:34.553+0000 7feac2c69700 1 -- 172.21.15.188:0/3173290694 shutdown_connections 2024-02-18T11:32:34.552 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:34.553+0000 7feac2c69700 1 -- 172.21.15.188:0/3173290694 wait complete. 2024-02-18T11:32:34.552 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:34.554+0000 7feac2c69700 1 Processor -- start 2024-02-18T11:32:34.552 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:34.554+0000 7feac2c69700 1 -- start start 2024-02-18T11:32:34.552 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:34.554+0000 7feac2c69700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feabc106640 0x7feabc197d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:32:34.552 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:34.554+0000 7feac2c69700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feabc06a8e0 con 0x7feabc106640 2024-02-18T11:32:34.552 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:34.555+0000 7feac0a05700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feabc106640 0x7feabc197d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:32:34.552 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:34.555+0000 7feac0a05700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feabc106640 0x7feabc197d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.188:51742/0 (socket says 172.21.15.188:51742) 2024-02-18T11:32:34.552 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:34.555+0000 7feac0a05700 1 -- 172.21.15.188:0/2039730474 learned_addr learned my addr 172.21.15.188:0/2039730474 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-02-18T11:32:34.552 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:34.555+0000 7feac0a05700 1 -- 172.21.15.188:0/2039730474 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feab00095f0 con 0x7feabc106640 2024-02-18T11:32:34.552 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:34.555+0000 7feac0a05700 1 --2- 172.21.15.188:0/2039730474 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feabc106640 0x7feabc197d00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7feab0004f40 tx=0x7feab0005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:32:34.552 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:34.555+0000 7feab9ffb700 1 -- 172.21.15.188:0/2039730474 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feab001d070 con 0x7feabc106640 2024-02-18T11:32:34.553 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:34.555+0000 7feac2c69700 1 -- 172.21.15.188:0/2039730474 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feabc198300 con 0x7feabc106640 2024-02-18T11:32:34.553 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:34.555+0000 7feab9ffb700 1 -- 172.21.15.188:0/2039730474 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feab000bc30 con 0x7feabc106640 2024-02-18T11:32:34.553 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:34.555+0000 7feab9ffb700 1 -- 172.21.15.188:0/2039730474 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feab0003970 con 0x7feabc106640 2024-02-18T11:32:34.553 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:34.556+0000 7feac2c69700 1 -- 172.21.15.188:0/2039730474 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feabc198780 con 0x7feabc106640 2024-02-18T11:32:34.554 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:34.556+0000 7feac2c69700 1 -- 172.21.15.188:0/2039730474 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feabc101cb0 con 0x7feabc106640 2024-02-18T11:32:34.554 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:34.556+0000 7feab9ffb700 1 -- 172.21.15.188:0/2039730474 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) v1 ==== 46147+0+0 (secure 0 0 0) 0x7feab0003c30 con 0x7feabc106640 2024-02-18T11:32:34.554 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:34.557+0000 7feab9ffb700 1 --2- 172.21.15.188:0/2039730474 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7feaac0391b0 0x7feaac03b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:32:34.554 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:34.557+0000 7feab9ffb700 1 -- 172.21.15.188:0/2039730474 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7feab004d1a0 con 0x7feabc106640 2024-02-18T11:32:34.554 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:34.557+0000 7feabbfff700 1 --2- 172.21.15.188:0/2039730474 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7feaac0391b0 0x7feaac03b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:32:34.554 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:34.557+0000 7feabbfff700 1 --2- 172.21.15.188:0/2039730474 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7feaac0391b0 0x7feaac03b670 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7feaa8006fd0 tx=0x7feaa8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:32:34.558 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:34.562+0000 7feab9ffb700 1 -- 172.21.15.188:0/2039730474 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7feab0029540 con 0x7feabc106640 2024-02-18T11:32:34.835 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:34.839+0000 7feac2c69700 1 -- 172.21.15.188:0/2039730474 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7feabc198b30 con 0x7feabc106640 2024-02-18T11:32:34.836 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:34.839+0000 7feab9ffb700 1 -- 172.21.15.188:0/2039730474 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+745 (secure 0 0 0) 0x7feab0026070 con 0x7feabc106640 2024-02-18T11:32:34.836 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-02-18T11:32:34.836 INFO:teuthology.orchestra.run.smithi188.stdout:{"epoch":1,"fsid":"0c8bb0de-ce51-11ee-95ba-87774f69a715","modified":"2024-02-18T11:30:43.044504Z","created":"2024-02-18T11:30:43.044504Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:3300","nonce":0},{"type":"v1","addr":"172.21.15.12:6789","nonce":0}]},"addr":"172.21.15.12:6789/0","public_addr":"172.21.15.12:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-18T11:32:34.838 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:34.842+0000 7feac2c69700 1 -- 172.21.15.188:0/2039730474 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7feaac0391b0 msgr2=0x7feaac03b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:32:34.838 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:34.842+0000 7feac2c69700 1 --2- 172.21.15.188:0/2039730474 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7feaac0391b0 0x7feaac03b670 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7feaa8006fd0 tx=0x7feaa8006e40 comp rx=0 tx=0).stop 2024-02-18T11:32:34.838 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:34.842+0000 7feac2c69700 1 -- 172.21.15.188:0/2039730474 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feabc106640 msgr2=0x7feabc197d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:32:34.838 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:34.842+0000 7feac2c69700 1 --2- 172.21.15.188:0/2039730474 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feabc106640 0x7feabc197d00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7feab0004f40 tx=0x7feab0005020 comp rx=0 tx=0).stop 2024-02-18T11:32:34.838 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:34.842+0000 7feac2c69700 1 -- 172.21.15.188:0/2039730474 shutdown_connections 2024-02-18T11:32:34.838 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:34.842+0000 7feac2c69700 1 --2- 172.21.15.188:0/2039730474 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7feaac0391b0 0x7feaac03b670 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:32:34.838 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:34.842+0000 7feac2c69700 1 --2- 172.21.15.188:0/2039730474 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feabc106640 0x7feabc197d00 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:32:34.838 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:34.842+0000 7feac2c69700 1 -- 172.21.15.188:0/2039730474 >> 172.21.15.188:0/2039730474 conn(0x7feabc0fc060 msgr2=0x7feabc0fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:32:34.838 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:34.842+0000 7feac2c69700 1 -- 172.21.15.188:0/2039730474 shutdown_connections 2024-02-18T11:32:34.838 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:34.842+0000 7feac2c69700 1 -- 172.21.15.188:0/2039730474 wait complete. 2024-02-18T11:32:34.839 INFO:teuthology.orchestra.run.smithi188.stderr:dumped monmap epoch 1 2024-02-18T11:32:34.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:34 smithi012 ceph-mon[32460]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:32:35.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:34 smithi188 ceph-mon[36925]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-02-18T11:32:35.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:35 smithi188 bash[36877]: 7875018d620ce6f669d4e4488106f1078a142ed107f42cda925fa5c6fdc76223 2024-02-18T11:32:35.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:35 smithi188 systemd[1]: Started Ceph mon.c for 0c8bb0de-ce51-11ee-95ba-87774f69a715. 2024-02-18T11:32:36.452 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-18T11:32:36.453 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-18T11:32:36.692 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.c/config 2024-02-18T11:32:40.355 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:40 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-18T11:32:40.356 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:40 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T11:32:40.356 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:40 smithi188 ceph-mon[36925]: mon.a calling monitor election 2024-02-18T11:32:40.356 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:40 smithi188 ceph-mon[36925]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:32:40.356 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:40 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T11:32:40.356 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:40 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T11:32:40.356 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:40 smithi188 ceph-mon[36925]: mon.c calling monitor election 2024-02-18T11:32:40.356 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:40 smithi188 ceph-mon[36925]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:32:40.356 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:40 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T11:32:40.356 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:40 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T11:32:40.356 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:40 smithi188 ceph-mon[36925]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:32:40.357 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:40 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T11:32:40.357 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:40 smithi188 ceph-mon[36925]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-18T11:32:40.357 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:40 smithi188 ceph-mon[36925]: monmap e2: 2 mons at {a=[v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0],c=[v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-18T11:32:40.357 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:40 smithi188 ceph-mon[36925]: fsmap 2024-02-18T11:32:40.357 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:40 smithi188 ceph-mon[36925]: osdmap e4: 0 total, 0 up, 0 in 2024-02-18T11:32:40.357 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:40 smithi188 ceph-mon[36925]: mgrmap e13: x(active, since 49s) 2024-02-18T11:32:40.357 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:40 smithi188 ceph-mon[36925]: overall HEALTH_OK 2024-02-18T11:32:40.357 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:40 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:40.357 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:40 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:40.357 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:40 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:40.357 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:40 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-18T11:32:40.357 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:40 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:32:40.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:40 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-18T11:32:40.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:40 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T11:32:40.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:40 smithi012 ceph-mon[32460]: mon.a calling monitor election 2024-02-18T11:32:40.425 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:40 smithi012 ceph-mon[32460]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:32:40.425 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:40 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T11:32:40.425 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:40 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T11:32:40.425 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:40 smithi012 ceph-mon[32460]: mon.c calling monitor election 2024-02-18T11:32:40.425 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:40 smithi012 ceph-mon[32460]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:32:40.425 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:40 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T11:32:40.425 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:40 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T11:32:40.425 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:40 smithi012 ceph-mon[32460]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:32:40.425 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:40 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T11:32:40.425 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:40 smithi012 ceph-mon[32460]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-18T11:32:40.425 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:40 smithi012 ceph-mon[32460]: monmap e2: 2 mons at {a=[v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0],c=[v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-18T11:32:40.425 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:40 smithi012 ceph-mon[32460]: fsmap 2024-02-18T11:32:40.425 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:40 smithi012 ceph-mon[32460]: osdmap e4: 0 total, 0 up, 0 in 2024-02-18T11:32:40.425 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:40 smithi012 ceph-mon[32460]: mgrmap e13: x(active, since 49s) 2024-02-18T11:32:40.426 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:40 smithi012 ceph-mon[32460]: overall HEALTH_OK 2024-02-18T11:32:40.426 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:40 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:40.426 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:40 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:40.426 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:40 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:40.426 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:40 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-18T11:32:40.426 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:40 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:32:41.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:41 smithi188 ceph-mon[36925]: Deploying daemon mon.b on smithi158 2024-02-18T11:32:41.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:41 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T11:32:41.202 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:41.205+0000 7f10698fe700 1 -- 172.21.15.188:0/3659628857 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f104c0035c0 msgr2=0x7f104c005a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:32:41.202 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:41.205+0000 7f10698fe700 1 --2- 172.21.15.188:0/3659628857 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f104c0035c0 0x7f104c005a30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f10641005a0 tx=0x7f105400feb0 comp rx=0 tx=0).stop 2024-02-18T11:32:41.202 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:41.206+0000 7f10698fe700 1 -- 172.21.15.188:0/3659628857 shutdown_connections 2024-02-18T11:32:41.202 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:41.206+0000 7f10698fe700 1 --2- 172.21.15.188:0/3659628857 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f104c0035c0 0x7f104c005a30 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:32:41.202 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:41.206+0000 7f10698fe700 1 -- 172.21.15.188:0/3659628857 >> 172.21.15.188:0/3659628857 conn(0x7f10640fc160 msgr2=0x7f10640fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:32:41.202 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:41.206+0000 7f10698fe700 1 -- 172.21.15.188:0/3659628857 shutdown_connections 2024-02-18T11:32:41.202 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:41.206+0000 7f10698fe700 1 -- 172.21.15.188:0/3659628857 wait complete. 2024-02-18T11:32:41.204 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:41.206+0000 7f10698fe700 1 Processor -- start 2024-02-18T11:32:41.204 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:41.207+0000 7f10698fe700 1 -- start start 2024-02-18T11:32:41.204 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:41.207+0000 7f10698fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f104c0035c0 0x7f1064197150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:32:41.204 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:41.207+0000 7f10698fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1064197690 0x7f106419ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:32:41.204 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:41.207+0000 7f10698fe700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1064197c20 con 0x7f1064197690 2024-02-18T11:32:41.205 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:41.207+0000 7f10698fe700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1064197d90 con 0x7f104c0035c0 2024-02-18T11:32:41.205 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:41.207+0000 7f1062ffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f104c0035c0 0x7f1064197150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:32:41.205 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:41.207+0000 7f1062ffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f104c0035c0 0x7f1064197150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:55254/0 (socket says 172.21.15.188:55254) 2024-02-18T11:32:41.205 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:41.207+0000 7f10627fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1064197690 0x7f106419ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:32:41.205 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:41.207+0000 7f1062ffd700 1 -- 172.21.15.188:0/1112484636 learned_addr learned my addr 172.21.15.188:0/1112484636 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-02-18T11:32:41.205 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:41.208+0000 7f1062ffd700 1 -- 172.21.15.188:0/1112484636 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f104c0035c0 msgr2=0x7f1064197150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-02-18T11:32:41.205 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:41.208+0000 7f1062ffd700 1 -- 172.21.15.188:0/1112484636 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f104c0035c0 msgr2=0x7f1064197150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-02-18T11:32:41.205 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:41.208+0000 7f1062ffd700 1 --2- 172.21.15.188:0/1112484636 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f104c0035c0 0x7f1064197150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-02-18T11:32:41.205 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:41.208+0000 7f1062ffd700 1 --2- 172.21.15.188:0/1112484636 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f104c0035c0 0x7f1064197150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-18T11:32:41.205 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:41.208+0000 7f10627fc700 1 -- 172.21.15.188:0/1112484636 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f104c0035c0 msgr2=0x7f1064197150 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-18T11:32:41.205 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:41.208+0000 7f10627fc700 1 --2- 172.21.15.188:0/1112484636 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f104c0035c0 0x7f1064197150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:32:41.205 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:41.208+0000 7f10627fc700 1 -- 172.21.15.188:0/1112484636 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f105400f900 con 0x7f1064197690 2024-02-18T11:32:41.206 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:41.208+0000 7f10627fc700 1 --2- 172.21.15.188:0/1112484636 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1064197690 0x7f106419ba20 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f105800b4d0 tx=0x7f105800b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:32:41.206 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:41.208+0000 7f10688fc700 1 -- 172.21.15.188:0/1112484636 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f1058011630 con 0x7f1064197690 2024-02-18T11:32:41.206 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:41.208+0000 7f10688fc700 1 -- 172.21.15.188:0/1112484636 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1058011c70 con 0x7f1064197690 2024-02-18T11:32:41.206 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:41.208+0000 7f10698fe700 1 -- 172.21.15.188:0/1112484636 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f106419bfc0 con 0x7f1064197690 2024-02-18T11:32:41.206 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:41.208+0000 7f10688fc700 1 -- 172.21.15.188:0/1112484636 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f1058010de0 con 0x7f1064197690 2024-02-18T11:32:41.206 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:41.208+0000 7f10698fe700 1 -- 172.21.15.188:0/1112484636 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f106419c510 con 0x7f1064197690 2024-02-18T11:32:41.207 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:41.209+0000 7f10688fc700 1 -- 172.21.15.188:0/1112484636 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) v1 ==== 46147+0+0 (secure 0 0 0) 0x7f105801a400 con 0x7f1064197690 2024-02-18T11:32:41.207 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:41.209+0000 7f10698fe700 1 -- 172.21.15.188:0/1112484636 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1064101db0 con 0x7f1064197690 2024-02-18T11:32:41.207 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:41.210+0000 7f10688fc700 1 --2- 172.21.15.188:0/1112484636 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f1050038fa0 0x7f105003b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:32:41.207 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:41.210+0000 7f10688fc700 1 -- 172.21.15.188:0/1112484636 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f105804ba60 con 0x7f1064197690 2024-02-18T11:32:41.207 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:41.210+0000 7f1062ffd700 1 --2- 172.21.15.188:0/1112484636 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f1050038fa0 0x7f105003b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:32:41.207 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:41.210+0000 7f1062ffd700 1 --2- 172.21.15.188:0/1112484636 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f1050038fa0 0x7f105003b460 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1054005b40 tx=0x7f1054005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:32:41.211 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:41.215+0000 7f10688fc700 1 -- 172.21.15.188:0/1112484636 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1058017680 con 0x7f1064197690 2024-02-18T11:32:41.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:41 smithi012 ceph-mon[32460]: Deploying daemon mon.b on smithi158 2024-02-18T11:32:41.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:41 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T11:32:41.490 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:41.494+0000 7f10698fe700 1 -- 172.21.15.188:0/1112484636 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f1064108520 con 0x7f1064197690 2024-02-18T11:32:41.491 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:41.495+0000 7f10688fc700 1 -- 172.21.15.188:0/1112484636 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1015 (secure 0 0 0) 0x7f10580172c0 con 0x7f1064197690 2024-02-18T11:32:41.491 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-02-18T11:32:41.491 INFO:teuthology.orchestra.run.smithi188.stdout:{"epoch":2,"fsid":"0c8bb0de-ce51-11ee-95ba-87774f69a715","modified":"2024-02-18T11:32:34.917850Z","created":"2024-02-18T11:30:43.044504Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:3300","nonce":0},{"type":"v1","addr":"172.21.15.12:6789","nonce":0}]},"addr":"172.21.15.12:6789/0","public_addr":"172.21.15.12:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:3300","nonce":0},{"type":"v1","addr":"172.21.15.188:6789","nonce":0}]},"addr":"172.21.15.188:6789/0","public_addr":"172.21.15.188:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-02-18T11:32:41.493 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:41.497+0000 7f10698fe700 1 -- 172.21.15.188:0/1112484636 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f1050038fa0 msgr2=0x7f105003b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:32:41.493 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:41.497+0000 7f10698fe700 1 --2- 172.21.15.188:0/1112484636 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f1050038fa0 0x7f105003b460 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1054005b40 tx=0x7f1054005a90 comp rx=0 tx=0).stop 2024-02-18T11:32:41.493 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:41.497+0000 7f10698fe700 1 -- 172.21.15.188:0/1112484636 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1064197690 msgr2=0x7f106419ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:32:41.493 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:41.497+0000 7f10698fe700 1 --2- 172.21.15.188:0/1112484636 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1064197690 0x7f106419ba20 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f105800b4d0 tx=0x7f105800b890 comp rx=0 tx=0).stop 2024-02-18T11:32:41.493 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:41.497+0000 7f10698fe700 1 -- 172.21.15.188:0/1112484636 shutdown_connections 2024-02-18T11:32:41.493 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:41.497+0000 7f10698fe700 1 --2- 172.21.15.188:0/1112484636 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f104c0035c0 0x7f1064197150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:32:41.494 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:41.497+0000 7f10698fe700 1 --2- 172.21.15.188:0/1112484636 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f1050038fa0 0x7f105003b460 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:32:41.494 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:41.497+0000 7f10698fe700 1 --2- 172.21.15.188:0/1112484636 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1064197690 0x7f106419ba20 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:32:41.494 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:41.497+0000 7f10698fe700 1 -- 172.21.15.188:0/1112484636 >> 172.21.15.188:0/1112484636 conn(0x7f10640fc160 msgr2=0x7f1064069aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:32:41.494 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:41.497+0000 7f10698fe700 1 -- 172.21.15.188:0/1112484636 shutdown_connections 2024-02-18T11:32:41.494 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:41.497+0000 7f10698fe700 1 -- 172.21.15.188:0/1112484636 wait complete. 2024-02-18T11:32:41.494 INFO:teuthology.orchestra.run.smithi188.stderr:dumped monmap epoch 2 2024-02-18T11:32:42.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:42 smithi188 ceph-mon[36925]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:32:42.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:42 smithi188 ceph-mon[36925]: from='client.? 172.21.15.188:0/1112484636' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-18T11:32:42.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:42 smithi012 ceph-mon[32460]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:32:42.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:42 smithi012 ceph-mon[32460]: from='client.? 172.21.15.188:0/1112484636' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-18T11:32:43.139 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-18T11:32:43.139 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-18T11:32:43.380 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.c/config 2024-02-18T11:32:43.841 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: mon.b@-1(???) e0 preinit fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 2024-02-18T11:32:43.841 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: mon.b@-1(synchronizing).mds e1 new map 2024-02-18T11:32:43.841 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: mon.b@-1(synchronizing).mds e1 print_map 2024-02-18T11:32:43.841 INFO:journalctl@ceph.mon.b.smithi158.stdout: e1 2024-02-18T11:32:43.841 INFO:journalctl@ceph.mon.b.smithi158.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-18T11:32:43.841 INFO:journalctl@ceph.mon.b.smithi158.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-18T11:32:43.841 INFO:journalctl@ceph.mon.b.smithi158.stdout: legacy client fscid: -1 2024-02-18T11:32:43.841 INFO:journalctl@ceph.mon.b.smithi158.stdout: 2024-02-18T11:32:43.841 INFO:journalctl@ceph.mon.b.smithi158.stdout: No filesystems configured 2024-02-18T11:32:43.841 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-02-18T11:32:43.841 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-02-18T11:32:44.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-02-18T11:32:44.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-02-18T11:32:44.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-02-18T11:32:44.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-02-18T11:32:44.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-18T11:32:44.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-18T11:32:44.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-18T11:32:44.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-18T11:32:44.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:32:44.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:44.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:32:44.247 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T11:32:44.247 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:44.247 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:32:44.247 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi188", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:32:44.247 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:44.247 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:44.247 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:44.247 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:44.247 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi158", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:32:44.247 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:32:44.247 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:32:44.247 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: Updating smithi158:/etc/ceph/ceph.conf 2024-02-18T11:32:44.247 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: Deploying cephadm binary to smithi188 2024-02-18T11:32:44.247 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: Updating smithi158:/var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-18T11:32:44.248 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: Updating smithi158:/etc/ceph/ceph.client.admin.keyring 2024-02-18T11:32:44.248 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:32:44.248 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:32:44.248 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: Updating smithi158:/var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-18T11:32:44.248 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:44.248 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:44.248 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:44.248 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:44.248 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:32:44.249 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:44.249 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: Added host smithi188 2024-02-18T11:32:44.249 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:44.249 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:32:44.249 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T11:32:44.249 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:44.249 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:32:44.249 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/2854546643' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-02-18T11:32:44.249 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:32:44.249 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/2854546643' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-02-18T11:32:44.249 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: osdmap e4: 0 total, 0 up, 0 in 2024-02-18T11:32:44.249 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:44.250 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:44.250 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:44.250 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:44.250 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi188", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:32:44.250 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:32:44.250 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:32:44.250 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: Updating smithi188:/etc/ceph/ceph.conf 2024-02-18T11:32:44.250 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:44.250 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:44.250 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:44.250 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: Updating smithi188:/var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-18T11:32:44.250 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: Updating smithi188:/etc/ceph/ceph.client.admin.keyring 2024-02-18T11:32:44.250 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: Updating smithi188:/var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-18T11:32:44.250 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:32:44.251 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:44.251 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:32:44.251 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:32:44.251 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:32:44.251 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:44.251 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-18T11:32:44.251 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:32:44.251 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi012:172.21.15.12=a;smithi158:172.21.15.158=b;smithi188:172.21.15.188=c", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:32:44.251 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: Saving service mon spec with placement smithi012:172.21.15.12=a;smithi158:172.21.15.158=b;smithi188:172.21.15.188=c;count:3 2024-02-18T11:32:44.251 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: Deploying daemon mon.c on smithi188 2024-02-18T11:32:44.251 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:32:44.251 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:32:44.251 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-18T11:32:44.251 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T11:32:44.251 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: mon.a calling monitor election 2024-02-18T11:32:44.252 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:32:44.252 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T11:32:44.252 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T11:32:44.252 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: mon.c calling monitor election 2024-02-18T11:32:44.252 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:32:44.252 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T11:32:44.252 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T11:32:44.252 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:32:44.252 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T11:32:44.252 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-18T11:32:44.252 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: monmap e2: 2 mons at {a=[v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0],c=[v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-18T11:32:44.252 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: fsmap 2024-02-18T11:32:44.252 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: osdmap e4: 0 total, 0 up, 0 in 2024-02-18T11:32:44.253 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: mgrmap e13: x(active, since 49s) 2024-02-18T11:32:44.253 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: overall HEALTH_OK 2024-02-18T11:32:44.253 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:44.253 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:44.253 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:44.253 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-18T11:32:44.253 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:32:44.253 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: Deploying daemon mon.b on smithi158 2024-02-18T11:32:44.253 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T11:32:44.253 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:32:44.253 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: from='client.? 172.21.15.188:0/1112484636' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-18T11:32:44.254 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 ceph-mon[36626]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-02-18T11:32:44.254 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 bash[36608]: bceb8ce2e0f307e0a94029737af1fc45a62ec71d8c9a2697b7c45ef81ef282ee 2024-02-18T11:32:44.254 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:43 smithi158 systemd[1]: Started Ceph mon.b for 0c8bb0de-ce51-11ee-95ba-87774f69a715. 2024-02-18T11:32:49.216 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:49.221+0000 7fc0e0b76700 1 -- 172.21.15.188:0/225464262 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc0c40035c0 msgr2=0x7fc0c4005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:32:49.216 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:49.221+0000 7fc0e0b76700 1 --2- 172.21.15.188:0/225464262 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc0c40035c0 0x7fc0c4005a50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc077eb0 tx=0x7fc0cc00fe10 comp rx=0 tx=0).stop 2024-02-18T11:32:49.216 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:49.221+0000 7fc0e0b76700 1 -- 172.21.15.188:0/225464262 shutdown_connections 2024-02-18T11:32:49.216 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:49.221+0000 7fc0e0b76700 1 --2- 172.21.15.188:0/225464262 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc0c40035c0 0x7fc0c4005a50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:32:49.217 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:49.221+0000 7fc0e0b76700 1 -- 172.21.15.188:0/225464262 >> 172.21.15.188:0/225464262 conn(0x7fc0dc076ca0 msgr2=0x7fc0dc0770b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:32:49.217 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:49.221+0000 7fc0e0b76700 1 -- 172.21.15.188:0/225464262 shutdown_connections 2024-02-18T11:32:49.217 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:49.221+0000 7fc0e0b76700 1 -- 172.21.15.188:0/225464262 wait complete. 2024-02-18T11:32:49.217 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:48 smithi188 ceph-mon[36925]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:32:49.217 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:48 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-18T11:32:49.217 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:48 smithi188 ceph-mon[36925]: mon.a calling monitor election 2024-02-18T11:32:49.217 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:48 smithi188 ceph-mon[36925]: mon.c calling monitor election 2024-02-18T11:32:49.217 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:48 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-18T11:32:49.217 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:48 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T11:32:49.217 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:48 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-18T11:32:49.218 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:48 smithi188 ceph-mon[36925]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:32:49.218 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:48 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-18T11:32:49.218 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:48 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-18T11:32:49.218 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:48 smithi188 ceph-mon[36925]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:32:49.218 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:48 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-18T11:32:49.218 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:48 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-18T11:32:49.218 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:48 smithi188 ceph-mon[36925]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-18T11:32:49.218 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:48 smithi188 ceph-mon[36925]: monmap e3: 3 mons at {a=[v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0],b=[v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0],c=[v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-18T11:32:49.218 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:48 smithi188 ceph-mon[36925]: fsmap 2024-02-18T11:32:49.218 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:48 smithi188 ceph-mon[36925]: osdmap e4: 0 total, 0 up, 0 in 2024-02-18T11:32:49.218 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:48 smithi188 ceph-mon[36925]: mgrmap e13: x(active, since 57s) 2024-02-18T11:32:49.218 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:48 smithi188 ceph-mon[36925]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-02-18T11:32:49.218 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:48 smithi188 ceph-mon[36925]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-02-18T11:32:49.218 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:48 smithi188 ceph-mon[36925]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-02-18T11:32:49.218 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:48 smithi188 ceph-mon[36925]: mon.b (rank 2) addr [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] is down (out of quorum) 2024-02-18T11:32:49.219 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:48 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:49.219 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:48 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:49.219 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:48 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:49.219 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:48 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:49.221 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:49.222+0000 7fc0e0b76700 1 Processor -- start 2024-02-18T11:32:49.221 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:49.222+0000 7fc0e0b76700 1 -- start start 2024-02-18T11:32:49.221 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:49.222+0000 7fc0e0b76700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc0c40035c0 0x7fc0dc1010a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:32:49.221 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:49.222+0000 7fc0e0b76700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0dc1015e0 0x7fc0dc101a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:32:49.221 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:49.222+0000 7fc0e0b76700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc0dc1a34b0 0x7fc0dc1a58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:32:49.221 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:49.222+0000 7fc0e0b76700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0dc1a5ea0 con 0x7fc0c40035c0 2024-02-18T11:32:49.221 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:49.222+0000 7fc0e0b76700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0dc1a6010 con 0x7fc0dc1a34b0 2024-02-18T11:32:49.222 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:49.222+0000 7fc0e0b76700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0dc1a61d0 con 0x7fc0dc1015e0 2024-02-18T11:32:49.222 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:49.222+0000 7fc0d9d9b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0dc1015e0 0x7fc0dc101a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:32:49.222 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:49.222+0000 7fc0d9d9b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0dc1015e0 0x7fc0dc101a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:54108/0 (socket says 172.21.15.188:54108) 2024-02-18T11:32:49.222 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:49.222+0000 7fc0d9d9b700 1 -- 172.21.15.188:0/3910062517 learned_addr learned my addr 172.21.15.188:0/3910062517 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-02-18T11:32:49.222 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:49.222+0000 7fc0da59c700 1 --2- 172.21.15.188:0/3910062517 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc0c40035c0 0x7fc0dc1010a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:32:49.222 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:49.223+0000 7fc0dad9d700 1 --2- 172.21.15.188:0/3910062517 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc0dc1a34b0 0x7fc0dc1a58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:32:49.222 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:49.223+0000 7fc0d9d9b700 1 -- 172.21.15.188:0/3910062517 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0dc1015e0 msgr2=0x7fc0dc101a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-02-18T11:32:49.222 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:49.223+0000 7fc0d9d9b700 1 -- 172.21.15.188:0/3910062517 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0dc1015e0 msgr2=0x7fc0dc101a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-02-18T11:32:49.222 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:49.223+0000 7fc0d9d9b700 1 --2- 172.21.15.188:0/3910062517 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0dc1015e0 0x7fc0dc101a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-02-18T11:32:49.222 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:49.223+0000 7fc0d9d9b700 1 --2- 172.21.15.188:0/3910062517 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0dc1015e0 0x7fc0dc101a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-18T11:32:49.222 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:49.223+0000 7fc0dad9d700 1 -- 172.21.15.188:0/3910062517 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc0dc1a34b0 msgr2=0x7fc0dc1a58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-02-18T11:32:49.222 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:49.223+0000 7fc0dad9d700 1 -- 172.21.15.188:0/3910062517 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc0dc1a34b0 msgr2=0x7fc0dc1a58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-02-18T11:32:49.222 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:49.223+0000 7fc0dad9d700 1 --2- 172.21.15.188:0/3910062517 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc0dc1a34b0 0x7fc0dc1a58a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-02-18T11:32:49.223 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:49.223+0000 7fc0dad9d700 1 --2- 172.21.15.188:0/3910062517 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc0dc1a34b0 0x7fc0dc1a58a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-18T11:32:49.223 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:49.223+0000 7fc0da59c700 1 -- 172.21.15.188:0/3910062517 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0dc1015e0 msgr2=0x7fc0dc101a60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-18T11:32:49.223 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:49.223+0000 7fc0da59c700 1 --2- 172.21.15.188:0/3910062517 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0dc1015e0 0x7fc0dc101a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:32:49.223 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:49.223+0000 7fc0da59c700 1 -- 172.21.15.188:0/3910062517 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc0dc1a34b0 msgr2=0x7fc0dc1a58a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-18T11:32:49.223 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:49.223+0000 7fc0da59c700 1 --2- 172.21.15.188:0/3910062517 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc0dc1a34b0 0x7fc0dc1a58a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:32:49.223 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:49.223+0000 7fc0da59c700 1 -- 172.21.15.188:0/3910062517 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0cc00f860 con 0x7fc0c40035c0 2024-02-18T11:32:49.223 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:49.223+0000 7fc0da59c700 1 --2- 172.21.15.188:0/3910062517 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc0c40035c0 0x7fc0dc1010a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc01f000 tx=0x7fc0cc012a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:32:49.223 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:49.223+0000 7fc0cb7fe700 1 -- 172.21.15.188:0/3910062517 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0cc004be0 con 0x7fc0c40035c0 2024-02-18T11:32:49.223 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:49.223+0000 7fc0cb7fe700 1 -- 172.21.15.188:0/3910062517 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc0cc004d40 con 0x7fc0c40035c0 2024-02-18T11:32:49.223 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:49.224+0000 7fc0e0b76700 1 -- 172.21.15.188:0/3910062517 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0dc1a6450 con 0x7fc0c40035c0 2024-02-18T11:32:49.223 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:49.224+0000 7fc0cb7fe700 1 -- 172.21.15.188:0/3910062517 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0cc0277f0 con 0x7fc0c40035c0 2024-02-18T11:32:49.223 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:49.224+0000 7fc0e0b76700 1 -- 172.21.15.188:0/3910062517 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0dc1a6950 con 0x7fc0c40035c0 2024-02-18T11:32:49.223 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:49.225+0000 7fc0cb7fe700 1 -- 172.21.15.188:0/3910062517 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) v1 ==== 46147+0+0 (secure 0 0 0) 0x7fc0cc024050 con 0x7fc0c40035c0 2024-02-18T11:32:49.223 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:49.225+0000 7fc0cb7fe700 1 --2- 172.21.15.188:0/3910062517 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fc0bc039640 0x7fc0bc03bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:32:49.223 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:49.225+0000 7fc0cb7fe700 1 -- 172.21.15.188:0/3910062517 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc0cc04ef80 con 0x7fc0c40035c0 2024-02-18T11:32:49.224 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:49.225+0000 7fc0e0b76700 1 -- 172.21.15.188:0/3910062517 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0dc0632e0 con 0x7fc0c40035c0 2024-02-18T11:32:49.224 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:49.225+0000 7fc0d9d9b700 1 --2- 172.21.15.188:0/3910062517 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fc0bc039640 0x7fc0bc03bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:32:49.225 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:49.226+0000 7fc0d9d9b700 1 --2- 172.21.15.188:0/3910062517 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fc0bc039640 0x7fc0bc03bb00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc0d0009180 tx=0x7fc0d0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:32:49.225 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:49.230+0000 7fc0cb7fe700 1 -- 172.21.15.188:0/3910062517 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc0cc02b080 con 0x7fc0c40035c0 2024-02-18T11:32:49.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:48 smithi012 ceph-mon[32460]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:32:49.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:48 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-18T11:32:49.425 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:48 smithi012 ceph-mon[32460]: mon.a calling monitor election 2024-02-18T11:32:49.425 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:48 smithi012 ceph-mon[32460]: mon.c calling monitor election 2024-02-18T11:32:49.425 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:48 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-18T11:32:49.425 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:48 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T11:32:49.425 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:48 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-18T11:32:49.425 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:48 smithi012 ceph-mon[32460]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:32:49.425 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:48 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-18T11:32:49.425 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:48 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-18T11:32:49.425 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:48 smithi012 ceph-mon[32460]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:32:49.425 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:48 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-18T11:32:49.425 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:48 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-18T11:32:49.425 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:48 smithi012 ceph-mon[32460]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-18T11:32:49.425 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:48 smithi012 ceph-mon[32460]: monmap e3: 3 mons at {a=[v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0],b=[v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0],c=[v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-18T11:32:49.425 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:48 smithi012 ceph-mon[32460]: fsmap 2024-02-18T11:32:49.426 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:48 smithi012 ceph-mon[32460]: osdmap e4: 0 total, 0 up, 0 in 2024-02-18T11:32:49.426 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:48 smithi012 ceph-mon[32460]: mgrmap e13: x(active, since 57s) 2024-02-18T11:32:49.426 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:48 smithi012 ceph-mon[32460]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-02-18T11:32:49.426 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:48 smithi012 ceph-mon[32460]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-02-18T11:32:49.426 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:48 smithi012 ceph-mon[32460]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-02-18T11:32:49.426 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:48 smithi012 ceph-mon[32460]: mon.b (rank 2) addr [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] is down (out of quorum) 2024-02-18T11:32:49.426 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:48 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:49.426 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:48 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:49.426 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:48 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:49.426 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:48 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:49.508 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:49.513+0000 7fc0e0b76700 1 -- 172.21.15.188:0/3910062517 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fc0dc109660 con 0x7fc0c40035c0 2024-02-18T11:32:49.508 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:49.513+0000 7fc0cb7fe700 1 -- 172.21.15.188:0/3910062517 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1283 (secure 0 0 0) 0x7fc0cc022020 con 0x7fc0c40035c0 2024-02-18T11:32:49.509 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-02-18T11:32:49.509 INFO:teuthology.orchestra.run.smithi188.stdout:{"epoch":3,"fsid":"0c8bb0de-ce51-11ee-95ba-87774f69a715","modified":"2024-02-18T11:32:43.852518Z","created":"2024-02-18T11:30:43.044504Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:3300","nonce":0},{"type":"v1","addr":"172.21.15.12:6789","nonce":0}]},"addr":"172.21.15.12:6789/0","public_addr":"172.21.15.12:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:3300","nonce":0},{"type":"v1","addr":"172.21.15.188:6789","nonce":0}]},"addr":"172.21.15.188:6789/0","public_addr":"172.21.15.188:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:3300","nonce":0},{"type":"v1","addr":"172.21.15.158:6789","nonce":0}]},"addr":"172.21.15.158:6789/0","public_addr":"172.21.15.158:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-02-18T11:32:49.511 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:49.516+0000 7fc0e0b76700 1 -- 172.21.15.188:0/3910062517 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fc0bc039640 msgr2=0x7fc0bc03bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:32:49.511 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:49.516+0000 7fc0e0b76700 1 --2- 172.21.15.188:0/3910062517 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fc0bc039640 0x7fc0bc03bb00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc0d0009180 tx=0x7fc0d0008040 comp rx=0 tx=0).stop 2024-02-18T11:32:49.511 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:49.516+0000 7fc0e0b76700 1 -- 172.21.15.188:0/3910062517 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc0c40035c0 msgr2=0x7fc0dc1010a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:32:49.511 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:49.516+0000 7fc0e0b76700 1 --2- 172.21.15.188:0/3910062517 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc0c40035c0 0x7fc0dc1010a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc01f000 tx=0x7fc0cc012a00 comp rx=0 tx=0).stop 2024-02-18T11:32:49.511 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:49.516+0000 7fc0e0b76700 1 -- 172.21.15.188:0/3910062517 shutdown_connections 2024-02-18T11:32:49.511 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:49.516+0000 7fc0e0b76700 1 --2- 172.21.15.188:0/3910062517 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc0dc1a34b0 0x7fc0dc1a58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:32:49.511 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:49.516+0000 7fc0e0b76700 1 --2- 172.21.15.188:0/3910062517 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fc0bc039640 0x7fc0bc03bb00 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:32:49.511 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:49.516+0000 7fc0e0b76700 1 --2- 172.21.15.188:0/3910062517 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc0c40035c0 0x7fc0dc1010a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:32:49.511 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:49.516+0000 7fc0e0b76700 1 --2- 172.21.15.188:0/3910062517 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0dc1015e0 0x7fc0dc101a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:32:49.511 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:49.516+0000 7fc0e0b76700 1 -- 172.21.15.188:0/3910062517 >> 172.21.15.188:0/3910062517 conn(0x7fc0dc076ca0 msgr2=0x7fc0dc0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:32:49.511 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:49.516+0000 7fc0e0b76700 1 -- 172.21.15.188:0/3910062517 shutdown_connections 2024-02-18T11:32:49.511 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:49.516+0000 7fc0e0b76700 1 -- 172.21.15.188:0/3910062517 wait complete. 2024-02-18T11:32:49.512 INFO:teuthology.orchestra.run.smithi188.stderr:dumped monmap epoch 3 2024-02-18T11:32:50.040 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-02-18T11:32:50.041 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph config generate-minimal-conf 2024-02-18T11:32:50.067 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:49 smithi012 ceph-mon[32460]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:32:50.067 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:49 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:32:50.067 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:49 smithi012 ceph-mon[32460]: from='client.? 172.21.15.188:0/3910062517' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-18T11:32:50.067 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:49 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-18T11:32:50.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:49 smithi188 ceph-mon[36925]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:32:50.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:49 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:32:50.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:49 smithi188 ceph-mon[36925]: from='client.? 172.21.15.188:0/3910062517' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-18T11:32:50.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:49 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-18T11:32:50.287 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T11:32:51.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:50 smithi158 ceph-mon[36626]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:32:51.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:50 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-18T11:32:51.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:50 smithi158 ceph-mon[36626]: mon.a calling monitor election 2024-02-18T11:32:51.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:50 smithi158 ceph-mon[36626]: mon.c calling monitor election 2024-02-18T11:32:51.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:50 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-18T11:32:51.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:50 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T11:32:51.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:50 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-18T11:32:51.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:50 smithi158 ceph-mon[36626]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:32:51.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:50 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-18T11:32:51.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:50 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-18T11:32:51.247 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:50 smithi158 ceph-mon[36626]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:32:51.247 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:50 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-18T11:32:51.247 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:50 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-18T11:32:51.247 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:50 smithi158 ceph-mon[36626]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-18T11:32:51.247 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:50 smithi158 ceph-mon[36626]: monmap e3: 3 mons at {a=[v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0],b=[v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0],c=[v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-18T11:32:51.247 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:50 smithi158 ceph-mon[36626]: fsmap 2024-02-18T11:32:51.247 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:50 smithi158 ceph-mon[36626]: osdmap e4: 0 total, 0 up, 0 in 2024-02-18T11:32:51.247 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:50 smithi158 ceph-mon[36626]: mgrmap e13: x(active, since 57s) 2024-02-18T11:32:51.247 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:50 smithi158 ceph-mon[36626]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-02-18T11:32:51.247 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:50 smithi158 ceph-mon[36626]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-02-18T11:32:51.247 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:50 smithi158 ceph-mon[36626]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-02-18T11:32:51.247 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:50 smithi158 ceph-mon[36626]: mon.b (rank 2) addr [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] is down (out of quorum) 2024-02-18T11:32:51.247 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:50 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:51.247 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:50 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:51.248 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:50 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:51.248 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:50 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:51.248 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:50 smithi158 ceph-mon[36626]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:32:51.248 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:50 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:32:51.248 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:50 smithi158 ceph-mon[36626]: from='client.? 172.21.15.188:0/3910062517' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-18T11:32:51.248 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:50 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-18T11:32:51.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:51.862+0000 7efc96edf700 1 -- 172.21.15.12:0/1940601761 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc90106650 msgr2=0x7efc90106a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:32:51.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:51.862+0000 7efc96edf700 1 --2- 172.21.15.12:0/1940601761 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc90106650 0x7efc90106a30 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7efc84009910 tx=0x7efc84009c20 comp rx=0 tx=0).stop 2024-02-18T11:32:51.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:51.862+0000 7efc96edf700 1 -- 172.21.15.12:0/1940601761 shutdown_connections 2024-02-18T11:32:51.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:51.862+0000 7efc96edf700 1 --2- 172.21.15.12:0/1940601761 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc90106650 0x7efc90106a30 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:32:51.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:51.862+0000 7efc96edf700 1 -- 172.21.15.12:0/1940601761 >> 172.21.15.12:0/1940601761 conn(0x7efc900771f0 msgr2=0x7efc900fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:32:51.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:51.862+0000 7efc96edf700 1 -- 172.21.15.12:0/1940601761 shutdown_connections 2024-02-18T11:32:51.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:51.862+0000 7efc96edf700 1 -- 172.21.15.12:0/1940601761 wait complete. 2024-02-18T11:32:51.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:51.863+0000 7efc96edf700 1 Processor -- start 2024-02-18T11:32:51.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:51.863+0000 7efc96edf700 1 -- start start 2024-02-18T11:32:51.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:51.863+0000 7efc96edf700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc90106650 0x7efc90197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:32:51.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:51.863+0000 7efc96edf700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efc901984e0 0x7efc90198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:32:51.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:51.863+0000 7efc96edf700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efc901a1010 0x7efc901a3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:32:51.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:51.863+0000 7efc96edf700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc901a3ae0 con 0x7efc90106650 2024-02-18T11:32:51.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:51.863+0000 7efc96edf700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc901a3c50 con 0x7efc901a1010 2024-02-18T11:32:51.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:51.863+0000 7efc96edf700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc901a3e10 con 0x7efc901984e0 2024-02-18T11:32:51.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:51.863+0000 7efc94c7b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc90106650 0x7efc90197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:32:51.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:51.863+0000 7efc94c7b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc90106650 0x7efc90197fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34542/0 (socket says 172.21.15.12:34542) 2024-02-18T11:32:51.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:51.863+0000 7efc94c7b700 1 -- 172.21.15.12:0/3352211942 learned_addr learned my addr 172.21.15.12:0/3352211942 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:32:51.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:51.863+0000 7efc9547c700 1 --2- 172.21.15.12:0/3352211942 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efc901a1010 0x7efc901a3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:32:51.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:51.863+0000 7efc8ffff700 1 --2- 172.21.15.12:0/3352211942 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efc901984e0 0x7efc90198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:32:51.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:51.863+0000 7efc94c7b700 1 -- 172.21.15.12:0/3352211942 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efc901984e0 msgr2=0x7efc90198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:32:51.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:51.863+0000 7efc94c7b700 1 --2- 172.21.15.12:0/3352211942 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efc901984e0 0x7efc90198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:32:51.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:51.863+0000 7efc94c7b700 1 -- 172.21.15.12:0/3352211942 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efc901a1010 msgr2=0x7efc901a3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:32:51.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:51.863+0000 7efc94c7b700 1 --2- 172.21.15.12:0/3352211942 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efc901a1010 0x7efc901a3400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:32:51.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:51.863+0000 7efc94c7b700 1 -- 172.21.15.12:0/3352211942 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc840095f0 con 0x7efc90106650 2024-02-18T11:32:51.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:51.863+0000 7efc8ffff700 1 --2- 172.21.15.12:0/3352211942 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efc901984e0 0x7efc90198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:32:51.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:51.864+0000 7efc9547c700 1 --2- 172.21.15.12:0/3352211942 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efc901a1010 0x7efc901a3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:32:51.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:51.864+0000 7efc94c7b700 1 --2- 172.21.15.12:0/3352211942 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc90106650 0x7efc90197fa0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7efc84005b40 tx=0x7efc8400bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:32:51.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:51.864+0000 7efc8dffb700 1 -- 172.21.15.12:0/3352211942 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc8401d070 con 0x7efc90106650 2024-02-18T11:32:51.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:51.864+0000 7efc8dffb700 1 -- 172.21.15.12:0/3352211942 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efc8400fbd0 con 0x7efc90106650 2024-02-18T11:32:51.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:51.864+0000 7efc96edf700 1 -- 172.21.15.12:0/3352211942 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc901a40f0 con 0x7efc90106650 2024-02-18T11:32:51.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:51.864+0000 7efc8dffb700 1 -- 172.21.15.12:0/3352211942 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc84022d40 con 0x7efc90106650 2024-02-18T11:32:51.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:51.864+0000 7efc96edf700 1 -- 172.21.15.12:0/3352211942 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc901a4640 con 0x7efc90106650 2024-02-18T11:32:51.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:51.865+0000 7efc8dffb700 1 -- 172.21.15.12:0/3352211942 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) v1 ==== 46147+0+0 (secure 0 0 0) 0x7efc84017440 con 0x7efc90106650 2024-02-18T11:32:51.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:51.865+0000 7efc96edf700 1 -- 172.21.15.12:0/3352211942 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc90101cc0 con 0x7efc90106650 2024-02-18T11:32:51.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:51.865+0000 7efc8dffb700 1 --2- 172.21.15.12:0/3352211942 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7efc80039640 0x7efc8003bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:32:51.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:51.865+0000 7efc8dffb700 1 -- 172.21.15.12:0/3352211942 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7efc8404d2e0 con 0x7efc90106650 2024-02-18T11:32:51.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:51.866+0000 7efc8ffff700 1 --2- 172.21.15.12:0/3352211942 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7efc80039640 0x7efc8003bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:32:51.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:51.866+0000 7efc8ffff700 1 --2- 172.21.15.12:0/3352211942 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7efc80039640 0x7efc8003bb00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7efc7c0079b0 tx=0x7efc7c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:32:51.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:51.871+0000 7efc8dffb700 1 -- 172.21.15.12:0/3352211942 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7efc8402c440 con 0x7efc90106650 2024-02-18T11:32:51.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:51 smithi012 ceph-mon[32460]: mon.b calling monitor election 2024-02-18T11:32:51.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:51 smithi012 ceph-mon[32460]: Updating smithi012:/etc/ceph/ceph.conf 2024-02-18T11:32:51.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:51 smithi012 ceph-mon[32460]: Updating smithi158:/etc/ceph/ceph.conf 2024-02-18T11:32:51.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:51 smithi012 ceph-mon[32460]: Updating smithi188:/etc/ceph/ceph.conf 2024-02-18T11:32:51.925 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:51 smithi012 ceph-mon[32460]: mon.b calling monitor election 2024-02-18T11:32:51.925 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:51 smithi012 ceph-mon[32460]: mon.c calling monitor election 2024-02-18T11:32:51.925 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:51 smithi012 ceph-mon[32460]: mon.a calling monitor election 2024-02-18T11:32:51.925 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:51 smithi012 ceph-mon[32460]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-02-18T11:32:51.925 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:51 smithi012 ceph-mon[32460]: monmap e3: 3 mons at {a=[v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0],b=[v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0],c=[v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-18T11:32:51.925 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:51 smithi012 ceph-mon[32460]: fsmap 2024-02-18T11:32:51.925 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:51 smithi012 ceph-mon[32460]: osdmap e4: 0 total, 0 up, 0 in 2024-02-18T11:32:51.925 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:51 smithi012 ceph-mon[32460]: mgrmap e13: x(active, since 59s) 2024-02-18T11:32:51.925 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:51 smithi012 ceph-mon[32460]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-02-18T11:32:51.925 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:51 smithi012 ceph-mon[32460]: Cluster is now healthy 2024-02-18T11:32:51.925 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:51 smithi012 ceph-mon[32460]: overall HEALTH_OK 2024-02-18T11:32:51.925 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:51 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:51.925 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:51 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:51.925 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:51 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:51.926 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:51 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:51.926 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:51 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:51.926 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:51 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:51.926 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:51 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:51.926 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:51 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:51.926 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:51 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:51.926 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:51 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:51.926 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:51 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:51.926 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:51 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:51.926 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:51 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-18T11:32:51.926 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:51 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-18T11:32:51.926 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:51 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:32:52.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:52.084+0000 7efc96edf700 1 -- 172.21.15.12:0/3352211942 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7efc9010a940 con 0x7efc90106650 2024-02-18T11:32:52.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:52.084+0000 7efc8dffb700 1 -- 172.21.15.12:0/3352211942 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+275 (secure 0 0 0) 0x7efc840206f0 con 0x7efc90106650 2024-02-18T11:32:52.080 INFO:teuthology.orchestra.run.smithi012.stdout:# minimal ceph.conf for 0c8bb0de-ce51-11ee-95ba-87774f69a715 2024-02-18T11:32:52.080 INFO:teuthology.orchestra.run.smithi012.stdout:[global] 2024-02-18T11:32:52.080 INFO:teuthology.orchestra.run.smithi012.stdout: fsid = 0c8bb0de-ce51-11ee-95ba-87774f69a715 2024-02-18T11:32:52.081 INFO:teuthology.orchestra.run.smithi012.stdout: mon_host = [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] 2024-02-18T11:32:52.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:52.087+0000 7efc96edf700 1 -- 172.21.15.12:0/3352211942 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7efc80039640 msgr2=0x7efc8003bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:32:52.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:52.087+0000 7efc96edf700 1 --2- 172.21.15.12:0/3352211942 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7efc80039640 0x7efc8003bb00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7efc7c0079b0 tx=0x7efc7c008040 comp rx=0 tx=0).stop 2024-02-18T11:32:52.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:52.087+0000 7efc96edf700 1 -- 172.21.15.12:0/3352211942 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc90106650 msgr2=0x7efc90197fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:32:52.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:52.087+0000 7efc96edf700 1 --2- 172.21.15.12:0/3352211942 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc90106650 0x7efc90197fa0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7efc84005b40 tx=0x7efc8400bda0 comp rx=0 tx=0).stop 2024-02-18T11:32:52.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:52.087+0000 7efc96edf700 1 -- 172.21.15.12:0/3352211942 shutdown_connections 2024-02-18T11:32:52.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:52.087+0000 7efc96edf700 1 --2- 172.21.15.12:0/3352211942 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efc901a1010 0x7efc901a3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:32:52.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:52.087+0000 7efc96edf700 1 --2- 172.21.15.12:0/3352211942 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7efc80039640 0x7efc8003bb00 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:32:52.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:52.087+0000 7efc96edf700 1 --2- 172.21.15.12:0/3352211942 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc90106650 0x7efc90197fa0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:32:52.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:52.087+0000 7efc96edf700 1 --2- 172.21.15.12:0/3352211942 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efc901984e0 0x7efc90198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:32:52.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:52.087+0000 7efc96edf700 1 -- 172.21.15.12:0/3352211942 >> 172.21.15.12:0/3352211942 conn(0x7efc900771f0 msgr2=0x7efc900fc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:32:52.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:52.087+0000 7efc96edf700 1 -- 172.21.15.12:0/3352211942 shutdown_connections 2024-02-18T11:32:52.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:32:52.087+0000 7efc96edf700 1 -- 172.21.15.12:0/3352211942 wait complete. 2024-02-18T11:32:52.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:51 smithi188 ceph-mon[36925]: mon.b calling monitor election 2024-02-18T11:32:52.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:51 smithi188 ceph-mon[36925]: Updating smithi012:/etc/ceph/ceph.conf 2024-02-18T11:32:52.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:51 smithi188 ceph-mon[36925]: Updating smithi158:/etc/ceph/ceph.conf 2024-02-18T11:32:52.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:51 smithi188 ceph-mon[36925]: Updating smithi188:/etc/ceph/ceph.conf 2024-02-18T11:32:52.106 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:51 smithi188 ceph-mon[36925]: mon.b calling monitor election 2024-02-18T11:32:52.106 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:51 smithi188 ceph-mon[36925]: mon.c calling monitor election 2024-02-18T11:32:52.106 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:51 smithi188 ceph-mon[36925]: mon.a calling monitor election 2024-02-18T11:32:52.106 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:51 smithi188 ceph-mon[36925]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-02-18T11:32:52.106 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:51 smithi188 ceph-mon[36925]: monmap e3: 3 mons at {a=[v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0],b=[v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0],c=[v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-18T11:32:52.106 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:51 smithi188 ceph-mon[36925]: fsmap 2024-02-18T11:32:52.106 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:51 smithi188 ceph-mon[36925]: osdmap e4: 0 total, 0 up, 0 in 2024-02-18T11:32:52.106 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:51 smithi188 ceph-mon[36925]: mgrmap e13: x(active, since 59s) 2024-02-18T11:32:52.106 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:51 smithi188 ceph-mon[36925]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-02-18T11:32:52.106 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:51 smithi188 ceph-mon[36925]: Cluster is now healthy 2024-02-18T11:32:52.106 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:51 smithi188 ceph-mon[36925]: overall HEALTH_OK 2024-02-18T11:32:52.106 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:51 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:52.106 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:51 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:52.106 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:51 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:52.106 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:51 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:52.107 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:51 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:52.107 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:51 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:52.107 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:51 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:52.107 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:51 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:52.107 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:51 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:52.107 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:51 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:52.107 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:51 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:52.107 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:51 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:52.107 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:51 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-18T11:32:52.107 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:51 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-18T11:32:52.107 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:51 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:32:52.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:51 smithi158 ceph-mon[36626]: mon.b calling monitor election 2024-02-18T11:32:52.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:51 smithi158 ceph-mon[36626]: Updating smithi012:/etc/ceph/ceph.conf 2024-02-18T11:32:52.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:51 smithi158 ceph-mon[36626]: Updating smithi158:/etc/ceph/ceph.conf 2024-02-18T11:32:52.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:51 smithi158 ceph-mon[36626]: Updating smithi188:/etc/ceph/ceph.conf 2024-02-18T11:32:52.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:51 smithi158 ceph-mon[36626]: mon.b calling monitor election 2024-02-18T11:32:52.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:51 smithi158 ceph-mon[36626]: mon.c calling monitor election 2024-02-18T11:32:52.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:51 smithi158 ceph-mon[36626]: mon.a calling monitor election 2024-02-18T11:32:52.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:51 smithi158 ceph-mon[36626]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-02-18T11:32:52.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:51 smithi158 ceph-mon[36626]: monmap e3: 3 mons at {a=[v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0],b=[v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0],c=[v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-18T11:32:52.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:51 smithi158 ceph-mon[36626]: fsmap 2024-02-18T11:32:52.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:51 smithi158 ceph-mon[36626]: osdmap e4: 0 total, 0 up, 0 in 2024-02-18T11:32:52.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:51 smithi158 ceph-mon[36626]: mgrmap e13: x(active, since 59s) 2024-02-18T11:32:52.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:51 smithi158 ceph-mon[36626]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-02-18T11:32:52.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:51 smithi158 ceph-mon[36626]: Cluster is now healthy 2024-02-18T11:32:52.247 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:51 smithi158 ceph-mon[36626]: overall HEALTH_OK 2024-02-18T11:32:52.247 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:51 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:52.247 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:51 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:52.247 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:51 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:52.247 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:51 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:52.247 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:51 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:52.247 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:51 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:52.247 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:51 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:52.247 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:51 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:52.247 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:51 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:52.247 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:51 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:52.247 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:51 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:52.247 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:51 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:52.247 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:51 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-18T11:32:52.247 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:51 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-18T11:32:52.247 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:51 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:32:53.077 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:52 smithi012 ceph-mon[32460]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:32:53.077 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:52 smithi012 ceph-mon[32460]: Updating smithi012:/var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-18T11:32:53.077 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:52 smithi012 ceph-mon[32460]: Updating smithi188:/var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-18T11:32:53.077 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:52 smithi012 ceph-mon[32460]: Updating smithi158:/var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-18T11:32:53.077 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:52 smithi012 ceph-mon[32460]: Reconfiguring mon.a (unknown last config time)... 2024-02-18T11:32:53.077 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:52 smithi012 ceph-mon[32460]: Reconfiguring daemon mon.a on smithi012 2024-02-18T11:32:53.077 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:52 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-18T11:32:53.078 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:52 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/3352211942' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:32:53.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:52 smithi188 ceph-mon[36925]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:32:53.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:52 smithi188 ceph-mon[36925]: Updating smithi012:/var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-18T11:32:53.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:52 smithi188 ceph-mon[36925]: Updating smithi188:/var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-18T11:32:53.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:52 smithi188 ceph-mon[36925]: Updating smithi158:/var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-18T11:32:53.106 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:52 smithi188 ceph-mon[36925]: Reconfiguring mon.a (unknown last config time)... 2024-02-18T11:32:53.106 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:52 smithi188 ceph-mon[36925]: Reconfiguring daemon mon.a on smithi012 2024-02-18T11:32:53.106 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:52 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-18T11:32:53.106 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:52 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/3352211942' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:32:53.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:52 smithi158 ceph-mon[36626]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:32:53.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:52 smithi158 ceph-mon[36626]: Updating smithi012:/var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-18T11:32:53.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:52 smithi158 ceph-mon[36626]: Updating smithi188:/var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-18T11:32:53.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:52 smithi158 ceph-mon[36626]: Updating smithi158:/var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-18T11:32:53.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:52 smithi158 ceph-mon[36626]: Reconfiguring mon.a (unknown last config time)... 2024-02-18T11:32:53.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:52 smithi158 ceph-mon[36626]: Reconfiguring daemon mon.a on smithi012 2024-02-18T11:32:53.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:52 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-18T11:32:53.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:52 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/3352211942' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:32:53.475 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-02-18T11:32:53.476 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-02-18T11:32:53.476 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/ceph/ceph.conf 2024-02-18T11:32:53.547 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-02-18T11:32:53.547 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-18T11:32:53.616 DEBUG:teuthology.orchestra.run.smithi158:> set -ex 2024-02-18T11:32:53.616 DEBUG:teuthology.orchestra.run.smithi158:> sudo dd of=/etc/ceph/ceph.conf 2024-02-18T11:32:53.646 DEBUG:teuthology.orchestra.run.smithi158:> set -ex 2024-02-18T11:32:53.646 DEBUG:teuthology.orchestra.run.smithi158:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-18T11:32:53.714 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-02-18T11:32:53.714 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/etc/ceph/ceph.conf 2024-02-18T11:32:53.749 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-02-18T11:32:53.749 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-18T11:32:53.818 INFO:tasks.cephadm:Adding mgr.x on smithi012 2024-02-18T11:32:53.819 INFO:tasks.cephadm:Adding mgr.y on smithi158 2024-02-18T11:32:53.819 INFO:tasks.cephadm:Adding mgr.z on smithi188 2024-02-18T11:32:53.819 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph orch apply mgr '3;smithi012=x;smithi158=y;smithi188=z' 2024-02-18T11:32:54.090 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.c/config 2024-02-18T11:32:54.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:54 smithi158 ceph-mon[36626]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:32:54.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:54 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:54.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:54 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:54.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:54 smithi158 ceph-mon[36626]: Reconfiguring mon.b (monmap changed)... 2024-02-18T11:32:54.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:54 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-18T11:32:54.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:54 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-18T11:32:54.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:54 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:32:54.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:54 smithi158 ceph-mon[36626]: Reconfiguring daemon mon.b on smithi158 2024-02-18T11:32:54.420 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:54 smithi188 ceph-mon[36925]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:32:54.420 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:54 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:54.420 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:54 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:54.421 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:54 smithi188 ceph-mon[36925]: Reconfiguring mon.b (monmap changed)... 2024-02-18T11:32:54.421 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:54 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-18T11:32:54.421 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:54 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-18T11:32:54.421 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:54 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:32:54.421 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:54 smithi188 ceph-mon[36925]: Reconfiguring daemon mon.b on smithi158 2024-02-18T11:32:54.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:54 smithi012 ceph-mon[32460]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:32:54.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:54 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:54.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:54 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:54.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:54 smithi012 ceph-mon[32460]: Reconfiguring mon.b (monmap changed)... 2024-02-18T11:32:54.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:54 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-18T11:32:54.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:54 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-18T11:32:54.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:54 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:32:54.425 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:54 smithi012 ceph-mon[32460]: Reconfiguring daemon mon.b on smithi158 2024-02-18T11:32:55.548 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:55.551+0000 7f0d57911700 1 -- 172.21.15.188:0/638084628 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d50102ef0 msgr2=0x7f0d50103310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:32:55.548 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:55.551+0000 7f0d57911700 1 --2- 172.21.15.188:0/638084628 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d50102ef0 0x7f0d50103310 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c009910 tx=0x7f0d4c009c20 comp rx=0 tx=0).stop 2024-02-18T11:32:55.548 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:55.551+0000 7f0d57911700 1 -- 172.21.15.188:0/638084628 shutdown_connections 2024-02-18T11:32:55.548 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:55.551+0000 7f0d57911700 1 --2- 172.21.15.188:0/638084628 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d50102ef0 0x7f0d50103310 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:32:55.548 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:55.551+0000 7f0d57911700 1 -- 172.21.15.188:0/638084628 >> 172.21.15.188:0/638084628 conn(0x7f0d500fcdb0 msgr2=0x7f0d500ff1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:32:55.548 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:55.551+0000 7f0d57911700 1 -- 172.21.15.188:0/638084628 shutdown_connections 2024-02-18T11:32:55.548 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:55.551+0000 7f0d57911700 1 -- 172.21.15.188:0/638084628 wait complete. 2024-02-18T11:32:55.549 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:55.552+0000 7f0d57911700 1 Processor -- start 2024-02-18T11:32:55.549 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:55.552+0000 7f0d57911700 1 -- start start 2024-02-18T11:32:55.549 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:55.552+0000 7f0d57911700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d50102ef0 0x7f0d50196f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:32:55.549 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:55.552+0000 7f0d57911700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0d50197480 0x7f0d5019c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:32:55.549 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:55.552+0000 7f0d57911700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d50197900 0x7f0d5019ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:32:55.549 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:55.552+0000 7f0d57911700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d5019f220 con 0x7f0d50197900 2024-02-18T11:32:55.549 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:55.552+0000 7f0d57911700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d5019f390 con 0x7f0d50197480 2024-02-18T11:32:55.549 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:55.552+0000 7f0d57911700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d5019f550 con 0x7f0d50102ef0 2024-02-18T11:32:55.549 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:55.552+0000 7f0d556ad700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d50102ef0 0x7f0d50196f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:32:55.549 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:55.552+0000 7f0d55eae700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d50197900 0x7f0d5019ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:32:55.549 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:55.552+0000 7f0d556ad700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d50102ef0 0x7f0d50196f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:52708/0 (socket says 172.21.15.188:52708) 2024-02-18T11:32:55.549 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:55.552+0000 7f0d556ad700 1 -- 172.21.15.188:0/2897306363 learned_addr learned my addr 172.21.15.188:0/2897306363 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-02-18T11:32:55.549 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:55.552+0000 7f0d54eac700 1 --2- 172.21.15.188:0/2897306363 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0d50197480 0x7f0d5019c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:32:55.549 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:55.552+0000 7f0d556ad700 1 -- 172.21.15.188:0/2897306363 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0d50197480 msgr2=0x7f0d5019c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:32:55.549 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:55.552+0000 7f0d556ad700 1 --2- 172.21.15.188:0/2897306363 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0d50197480 0x7f0d5019c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:32:55.550 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:55.552+0000 7f0d556ad700 1 -- 172.21.15.188:0/2897306363 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d50197900 msgr2=0x7f0d5019ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:32:55.550 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:55.552+0000 7f0d556ad700 1 --2- 172.21.15.188:0/2897306363 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d50197900 0x7f0d5019ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:32:55.550 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:55.552+0000 7f0d556ad700 1 -- 172.21.15.188:0/2897306363 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d4c0095f0 con 0x7f0d50102ef0 2024-02-18T11:32:55.550 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:55.552+0000 7f0d55eae700 1 --2- 172.21.15.188:0/2897306363 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d50197900 0x7f0d5019ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-18T11:32:55.550 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:55.552+0000 7f0d556ad700 1 --2- 172.21.15.188:0/2897306363 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d50102ef0 0x7f0d50196f40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c00bcf0 tx=0x7f0d4c00bd20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:32:55.550 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:55.553+0000 7f0d467fc700 1 -- 172.21.15.188:0/2897306363 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d4c01d070 con 0x7f0d50102ef0 2024-02-18T11:32:55.550 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:55.553+0000 7f0d467fc700 1 -- 172.21.15.188:0/2897306363 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0d4c004e00 con 0x7f0d50102ef0 2024-02-18T11:32:55.550 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:55.553+0000 7f0d57911700 1 -- 172.21.15.188:0/2897306363 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d50073ee0 con 0x7f0d50102ef0 2024-02-18T11:32:55.550 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:55.553+0000 7f0d467fc700 1 -- 172.21.15.188:0/2897306363 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d4c022d20 con 0x7f0d50102ef0 2024-02-18T11:32:55.550 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:55.553+0000 7f0d57911700 1 -- 172.21.15.188:0/2897306363 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d50074400 con 0x7f0d50102ef0 2024-02-18T11:32:55.551 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:55.554+0000 7f0d467fc700 1 -- 172.21.15.188:0/2897306363 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 13) v1 ==== 46147+0+0 (secure 0 0 0) 0x7f0d4c022470 con 0x7f0d50102ef0 2024-02-18T11:32:55.551 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:55.554+0000 7f0d57911700 1 -- 172.21.15.188:0/2897306363 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d50190720 con 0x7f0d50102ef0 2024-02-18T11:32:55.551 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:55.554+0000 7f0d467fc700 1 --2- 172.21.15.188:0/2897306363 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f0d3c039640 0x7f0d3c03bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:32:55.551 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:55.554+0000 7f0d467fc700 1 -- 172.21.15.188:0/2897306363 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f0d4c04d570 con 0x7f0d50102ef0 2024-02-18T11:32:55.551 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:55.554+0000 7f0d54eac700 1 --2- 172.21.15.188:0/2897306363 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f0d3c039640 0x7f0d3c03bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:32:55.551 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:55.555+0000 7f0d54eac700 1 --2- 172.21.15.188:0/2897306363 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f0d3c039640 0x7f0d3c03bb00 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0d40009ad0 tx=0x7f0d40009210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:32:55.556 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:55.562+0000 7f0d467fc700 1 -- 172.21.15.188:0/2897306363 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0d4c00e5c0 con 0x7f0d50102ef0 2024-02-18T11:32:55.766 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:55.771+0000 7f0d57911700 1 -- 172.21.15.188:0/2897306363 --> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi012=x;smithi158=y;smithi188=z", "target": ["mon-mgr", ""]}) v1 -- 0x7f0d500620f0 con 0x7f0d3c039640 2024-02-18T11:32:55.772 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:55.777+0000 7f0d467fc700 1 -- 172.21.15.188:0/2897306363 <== mgr.14152 v2:172.21.15.12:6800/1153736685 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f0d500620f0 con 0x7f0d3c039640 2024-02-18T11:32:55.772 INFO:teuthology.orchestra.run.smithi188.stdout:Scheduled mgr update... 2024-02-18T11:32:55.774 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:55.779+0000 7f0d57911700 1 -- 172.21.15.188:0/2897306363 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f0d3c039640 msgr2=0x7f0d3c03bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:32:55.774 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:55.779+0000 7f0d57911700 1 --2- 172.21.15.188:0/2897306363 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f0d3c039640 0x7f0d3c03bb00 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0d40009ad0 tx=0x7f0d40009210 comp rx=0 tx=0).stop 2024-02-18T11:32:55.775 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:55.779+0000 7f0d57911700 1 -- 172.21.15.188:0/2897306363 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d50102ef0 msgr2=0x7f0d50196f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:32:55.775 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:55.779+0000 7f0d57911700 1 --2- 172.21.15.188:0/2897306363 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d50102ef0 0x7f0d50196f40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c00bcf0 tx=0x7f0d4c00bd20 comp rx=0 tx=0).stop 2024-02-18T11:32:55.775 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:55.779+0000 7f0d57911700 1 -- 172.21.15.188:0/2897306363 shutdown_connections 2024-02-18T11:32:55.775 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:55.779+0000 7f0d57911700 1 --2- 172.21.15.188:0/2897306363 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f0d3c039640 0x7f0d3c03bb00 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:32:55.775 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:55.779+0000 7f0d57911700 1 --2- 172.21.15.188:0/2897306363 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d50197900 0x7f0d5019ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:32:55.775 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:55.779+0000 7f0d57911700 1 --2- 172.21.15.188:0/2897306363 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d50102ef0 0x7f0d50196f40 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:32:55.776 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:55.779+0000 7f0d57911700 1 --2- 172.21.15.188:0/2897306363 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0d50197480 0x7f0d5019c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:32:55.776 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:55.779+0000 7f0d57911700 1 -- 172.21.15.188:0/2897306363 >> 172.21.15.188:0/2897306363 conn(0x7f0d500fcdb0 msgr2=0x7f0d500fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:32:55.776 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:55.779+0000 7f0d57911700 1 -- 172.21.15.188:0/2897306363 shutdown_connections 2024-02-18T11:32:55.776 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:32:55.779+0000 7f0d57911700 1 -- 172.21.15.188:0/2897306363 wait complete. 2024-02-18T11:32:55.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:55 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:55.996 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:55 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:55.996 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:55 smithi158 ceph-mon[36626]: Reconfiguring mon.c (monmap changed)... 2024-02-18T11:32:55.996 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:55 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-18T11:32:55.996 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:55 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-18T11:32:55.996 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:55 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:32:55.996 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:55 smithi158 ceph-mon[36626]: Reconfiguring daemon mon.c on smithi188 2024-02-18T11:32:56.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:55 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:56.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:55 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:56.106 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:55 smithi188 ceph-mon[36925]: Reconfiguring mon.c (monmap changed)... 2024-02-18T11:32:56.106 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:55 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-18T11:32:56.106 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:55 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-18T11:32:56.106 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:55 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:32:56.106 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:55 smithi188 ceph-mon[36925]: Reconfiguring daemon mon.c on smithi188 2024-02-18T11:32:56.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:55 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:56.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:55 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:56.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:55 smithi012 ceph-mon[32460]: Reconfiguring mon.c (monmap changed)... 2024-02-18T11:32:56.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:55 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-18T11:32:56.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:55 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-18T11:32:56.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:55 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:32:56.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:55 smithi012 ceph-mon[32460]: Reconfiguring daemon mon.c on smithi188 2024-02-18T11:32:57.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:56 smithi188 ceph-mon[36925]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:32:57.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:56 smithi188 ceph-mon[36925]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi012=x;smithi158=y;smithi188=z", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:32:57.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:56 smithi188 ceph-mon[36925]: Saving service mgr spec with placement smithi012=x;smithi158=y;smithi188=z;count:3 2024-02-18T11:32:57.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:56 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:57.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:56 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:57.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:56 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:57.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:56 smithi012 ceph-mon[32460]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:32:57.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:56 smithi012 ceph-mon[32460]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi012=x;smithi158=y;smithi188=z", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:32:57.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:56 smithi012 ceph-mon[32460]: Saving service mgr spec with placement smithi012=x;smithi158=y;smithi188=z;count:3 2024-02-18T11:32:57.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:56 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:57.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:56 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:57.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:56 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:57.179 DEBUG:teuthology.orchestra.run.smithi158:mgr.y> sudo journalctl -f -n 0 -u ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@mgr.y.service 2024-02-18T11:32:57.182 DEBUG:teuthology.orchestra.run.smithi188:mgr.z> sudo journalctl -f -n 0 -u ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@mgr.z.service 2024-02-18T11:32:57.184 INFO:tasks.cephadm:Deploying OSDs... 2024-02-18T11:32:57.185 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-02-18T11:32:57.185 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/scratch_devs of=/dev/stdout 2024-02-18T11:32:57.199 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-18T11:32:57.200 DEBUG:teuthology.orchestra.run.smithi012:> stat /dev/vg_nvme/lv_1 2024-02-18T11:32:57.208 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:56 smithi158 ceph-mon[36626]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:32:57.208 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:56 smithi158 ceph-mon[36626]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi012=x;smithi158=y;smithi188=z", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:32:57.208 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:56 smithi158 ceph-mon[36626]: Saving service mgr spec with placement smithi012=x;smithi158=y;smithi188=z;count:3 2024-02-18T11:32:57.208 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:56 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:57.208 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:56 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:57.208 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:56 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:57.254 INFO:teuthology.orchestra.run.smithi012.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-18T11:32:57.255 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-18T11:32:57.255 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-02-18T11:32:57.255 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-18T11:32:57.255 INFO:teuthology.orchestra.run.smithi012.stdout:Context: system_u:object_r:device_t:s0 2024-02-18T11:32:57.255 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2024-02-18 11:32:01.324086782 +0000 2024-02-18T11:32:57.255 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2024-02-18 11:27:36.641364274 +0000 2024-02-18T11:32:57.255 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2024-02-18 11:27:36.641364274 +0000 2024-02-18T11:32:57.255 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: 2024-02-18 11:27:36.641364274 +0000 2024-02-18T11:32:57.255 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-18T11:32:57.324 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records in 2024-02-18T11:32:57.324 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records out 2024-02-18T11:32:57.324 INFO:teuthology.orchestra.run.smithi012.stderr:512 bytes copied, 0.000209219 s, 2.4 MB/s 2024-02-18T11:32:57.326 DEBUG:teuthology.orchestra.run.smithi012:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-18T11:32:57.382 DEBUG:teuthology.orchestra.run.smithi012:> stat /dev/vg_nvme/lv_2 2024-02-18T11:32:57.437 INFO:teuthology.orchestra.run.smithi012.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-18T11:32:57.437 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-18T11:32:57.437 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-02-18T11:32:57.437 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-18T11:32:57.437 INFO:teuthology.orchestra.run.smithi012.stdout:Context: system_u:object_r:device_t:s0 2024-02-18T11:32:57.437 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2024-02-18 11:32:01.358086542 +0000 2024-02-18T11:32:57.437 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2024-02-18 11:27:36.995361993 +0000 2024-02-18T11:32:57.437 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2024-02-18 11:27:36.995361993 +0000 2024-02-18T11:32:57.437 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: 2024-02-18 11:27:36.995361993 +0000 2024-02-18T11:32:57.438 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-18T11:32:57.504 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records in 2024-02-18T11:32:57.505 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records out 2024-02-18T11:32:57.505 INFO:teuthology.orchestra.run.smithi012.stderr:512 bytes copied, 0.000206445 s, 2.5 MB/s 2024-02-18T11:32:57.505 DEBUG:teuthology.orchestra.run.smithi012:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-18T11:32:57.560 DEBUG:teuthology.orchestra.run.smithi012:> stat /dev/vg_nvme/lv_3 2024-02-18T11:32:57.614 INFO:teuthology.orchestra.run.smithi012.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-18T11:32:57.615 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-18T11:32:57.615 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-02-18T11:32:57.615 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-18T11:32:57.615 INFO:teuthology.orchestra.run.smithi012.stdout:Context: system_u:object_r:device_t:s0 2024-02-18T11:32:57.615 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2024-02-18 11:32:01.390086316 +0000 2024-02-18T11:32:57.615 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2024-02-18 11:27:37.342359757 +0000 2024-02-18T11:32:57.615 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2024-02-18 11:27:37.342359757 +0000 2024-02-18T11:32:57.615 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: 2024-02-18 11:27:37.342359757 +0000 2024-02-18T11:32:57.615 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-18T11:32:57.680 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records in 2024-02-18T11:32:57.680 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records out 2024-02-18T11:32:57.680 INFO:teuthology.orchestra.run.smithi012.stderr:512 bytes copied, 0.000211773 s, 2.4 MB/s 2024-02-18T11:32:57.681 DEBUG:teuthology.orchestra.run.smithi012:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-18T11:32:57.737 DEBUG:teuthology.orchestra.run.smithi012:> stat /dev/vg_nvme/lv_4 2024-02-18T11:32:57.793 INFO:teuthology.orchestra.run.smithi012.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-18T11:32:57.793 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-18T11:32:57.793 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-02-18T11:32:57.793 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-18T11:32:57.793 INFO:teuthology.orchestra.run.smithi012.stdout:Context: system_u:object_r:device_t:s0 2024-02-18T11:32:57.793 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2024-02-18 11:32:01.431086026 +0000 2024-02-18T11:32:57.793 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2024-02-18 11:27:37.664357682 +0000 2024-02-18T11:32:57.793 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2024-02-18 11:27:37.664357682 +0000 2024-02-18T11:32:57.794 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: 2024-02-18 11:27:37.664357682 +0000 2024-02-18T11:32:57.794 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-18T11:32:57.858 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records in 2024-02-18T11:32:57.858 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records out 2024-02-18T11:32:57.858 INFO:teuthology.orchestra.run.smithi012.stderr:512 bytes copied, 0.000184788 s, 2.8 MB/s 2024-02-18T11:32:57.859 DEBUG:teuthology.orchestra.run.smithi012:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-18T11:32:57.916 DEBUG:teuthology.orchestra.run.smithi158:> set -ex 2024-02-18T11:32:57.917 DEBUG:teuthology.orchestra.run.smithi158:> dd if=/scratch_devs of=/dev/stdout 2024-02-18T11:32:57.935 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-18T11:32:57.935 DEBUG:teuthology.orchestra.run.smithi158:> stat /dev/vg_nvme/lv_1 2024-02-18T11:32:57.992 INFO:teuthology.orchestra.run.smithi158.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-18T11:32:57.992 INFO:teuthology.orchestra.run.smithi158.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-18T11:32:57.992 INFO:teuthology.orchestra.run.smithi158.stdout:Device: 5h/5d Inode: 663 Links: 1 2024-02-18T11:32:57.992 INFO:teuthology.orchestra.run.smithi158.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-18T11:32:57.992 INFO:teuthology.orchestra.run.smithi158.stdout:Context: system_u:object_r:device_t:s0 2024-02-18T11:32:57.992 INFO:teuthology.orchestra.run.smithi158.stdout:Access: 2024-02-18 11:32:43.308073053 +0000 2024-02-18T11:32:57.993 INFO:teuthology.orchestra.run.smithi158.stdout:Modify: 2024-02-18 11:27:47.203184896 +0000 2024-02-18T11:32:57.993 INFO:teuthology.orchestra.run.smithi158.stdout:Change: 2024-02-18 11:27:47.203184896 +0000 2024-02-18T11:32:57.993 INFO:teuthology.orchestra.run.smithi158.stdout: Birth: 2024-02-18 11:27:47.203184896 +0000 2024-02-18T11:32:57.993 DEBUG:teuthology.orchestra.run.smithi158:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-18T11:32:58.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:57 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:32:58.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:57 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:32:58.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:57 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:32:58.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:57 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:58.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:57 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-18T11:32:58.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:57 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-18T11:32:58.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:57 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-18T11:32:58.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:57 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:32:58.058 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:57 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:32:58.058 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:57 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:32:58.059 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:57 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:32:58.059 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:57 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:58.059 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:57 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-18T11:32:58.059 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:57 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-18T11:32:58.059 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:57 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-18T11:32:58.059 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:57 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:32:58.060 INFO:teuthology.orchestra.run.smithi158.stderr:1+0 records in 2024-02-18T11:32:58.061 INFO:teuthology.orchestra.run.smithi158.stderr:1+0 records out 2024-02-18T11:32:58.061 INFO:teuthology.orchestra.run.smithi158.stderr:512 bytes copied, 9.8554e-05 s, 5.2 MB/s 2024-02-18T11:32:58.062 DEBUG:teuthology.orchestra.run.smithi158:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-18T11:32:58.121 DEBUG:teuthology.orchestra.run.smithi158:> stat /dev/vg_nvme/lv_2 2024-02-18T11:32:58.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:57 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:32:58.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:57 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:32:58.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:57 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:32:58.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:57 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:32:58.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:57 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-18T11:32:58.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:57 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-18T11:32:58.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:57 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-18T11:32:58.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:57 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:32:58.177 INFO:teuthology.orchestra.run.smithi158.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-18T11:32:58.177 INFO:teuthology.orchestra.run.smithi158.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-18T11:32:58.177 INFO:teuthology.orchestra.run.smithi158.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-02-18T11:32:58.177 INFO:teuthology.orchestra.run.smithi158.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-18T11:32:58.177 INFO:teuthology.orchestra.run.smithi158.stdout:Context: system_u:object_r:device_t:s0 2024-02-18T11:32:58.177 INFO:teuthology.orchestra.run.smithi158.stdout:Access: 2024-02-18 11:32:43.308073053 +0000 2024-02-18T11:32:58.177 INFO:teuthology.orchestra.run.smithi158.stdout:Modify: 2024-02-18 11:27:47.548181215 +0000 2024-02-18T11:32:58.177 INFO:teuthology.orchestra.run.smithi158.stdout:Change: 2024-02-18 11:27:47.548181215 +0000 2024-02-18T11:32:58.177 INFO:teuthology.orchestra.run.smithi158.stdout: Birth: 2024-02-18 11:27:47.548181215 +0000 2024-02-18T11:32:58.178 DEBUG:teuthology.orchestra.run.smithi158:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-18T11:32:58.243 INFO:teuthology.orchestra.run.smithi158.stderr:1+0 records in 2024-02-18T11:32:58.243 INFO:teuthology.orchestra.run.smithi158.stderr:1+0 records out 2024-02-18T11:32:58.243 INFO:teuthology.orchestra.run.smithi158.stderr:512 bytes copied, 0.000120386 s, 4.3 MB/s 2024-02-18T11:32:58.245 DEBUG:teuthology.orchestra.run.smithi158:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-18T11:32:58.301 DEBUG:teuthology.orchestra.run.smithi158:> stat /dev/vg_nvme/lv_3 2024-02-18T11:32:58.357 INFO:teuthology.orchestra.run.smithi158.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-18T11:32:58.357 INFO:teuthology.orchestra.run.smithi158.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-18T11:32:58.357 INFO:teuthology.orchestra.run.smithi158.stdout:Device: 5h/5d Inode: 699 Links: 1 2024-02-18T11:32:58.357 INFO:teuthology.orchestra.run.smithi158.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-18T11:32:58.357 INFO:teuthology.orchestra.run.smithi158.stdout:Context: system_u:object_r:device_t:s0 2024-02-18T11:32:58.357 INFO:teuthology.orchestra.run.smithi158.stdout:Access: 2024-02-18 11:32:43.309073042 +0000 2024-02-18T11:32:58.357 INFO:teuthology.orchestra.run.smithi158.stdout:Modify: 2024-02-18 11:27:47.873177747 +0000 2024-02-18T11:32:58.357 INFO:teuthology.orchestra.run.smithi158.stdout:Change: 2024-02-18 11:27:47.873177747 +0000 2024-02-18T11:32:58.358 INFO:teuthology.orchestra.run.smithi158.stdout: Birth: 2024-02-18 11:27:47.873177747 +0000 2024-02-18T11:32:58.358 DEBUG:teuthology.orchestra.run.smithi158:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-18T11:32:58.425 INFO:teuthology.orchestra.run.smithi158.stderr:1+0 records in 2024-02-18T11:32:58.425 INFO:teuthology.orchestra.run.smithi158.stderr:1+0 records out 2024-02-18T11:32:58.425 INFO:teuthology.orchestra.run.smithi158.stderr:512 bytes copied, 8.9561e-05 s, 5.7 MB/s 2024-02-18T11:32:58.425 DEBUG:teuthology.orchestra.run.smithi158:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-18T11:32:58.481 DEBUG:teuthology.orchestra.run.smithi158:> stat /dev/vg_nvme/lv_4 2024-02-18T11:32:58.536 INFO:teuthology.orchestra.run.smithi158.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-18T11:32:58.536 INFO:teuthology.orchestra.run.smithi158.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-18T11:32:58.536 INFO:teuthology.orchestra.run.smithi158.stdout:Device: 5h/5d Inode: 718 Links: 1 2024-02-18T11:32:58.536 INFO:teuthology.orchestra.run.smithi158.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-18T11:32:58.536 INFO:teuthology.orchestra.run.smithi158.stdout:Context: system_u:object_r:device_t:s0 2024-02-18T11:32:58.537 INFO:teuthology.orchestra.run.smithi158.stdout:Access: 2024-02-18 11:32:43.309073042 +0000 2024-02-18T11:32:58.537 INFO:teuthology.orchestra.run.smithi158.stdout:Modify: 2024-02-18 11:27:48.202174236 +0000 2024-02-18T11:32:58.537 INFO:teuthology.orchestra.run.smithi158.stdout:Change: 2024-02-18 11:27:48.202174236 +0000 2024-02-18T11:32:58.537 INFO:teuthology.orchestra.run.smithi158.stdout: Birth: 2024-02-18 11:27:48.202174236 +0000 2024-02-18T11:32:58.537 DEBUG:teuthology.orchestra.run.smithi158:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-18T11:32:58.602 INFO:teuthology.orchestra.run.smithi158.stderr:1+0 records in 2024-02-18T11:32:58.602 INFO:teuthology.orchestra.run.smithi158.stderr:1+0 records out 2024-02-18T11:32:58.602 INFO:teuthology.orchestra.run.smithi158.stderr:512 bytes copied, 0.000127694 s, 4.0 MB/s 2024-02-18T11:32:58.603 DEBUG:teuthology.orchestra.run.smithi158:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-18T11:32:58.658 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-02-18T11:32:58.658 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/scratch_devs of=/dev/stdout 2024-02-18T11:32:58.678 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-18T11:32:58.678 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_1 2024-02-18T11:32:58.735 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-18T11:32:58.735 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-18T11:32:58.735 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 5h/5d Inode: 660 Links: 1 2024-02-18T11:32:58.735 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-18T11:32:58.735 INFO:teuthology.orchestra.run.smithi188.stdout:Context: system_u:object_r:device_t:s0 2024-02-18T11:32:58.735 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2024-02-18 11:32:33.797582051 +0000 2024-02-18T11:32:58.736 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2024-02-18 11:27:51.245743832 +0000 2024-02-18T11:32:58.736 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2024-02-18 11:27:51.245743832 +0000 2024-02-18T11:32:58.736 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: 2024-02-18 11:27:51.245743832 +0000 2024-02-18T11:32:58.736 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-18T11:32:58.806 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2024-02-18T11:32:58.806 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2024-02-18T11:32:58.806 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.000281741 s, 1.8 MB/s 2024-02-18T11:32:58.807 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-18T11:32:58.866 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_2 2024-02-18T11:32:58.924 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-18T11:32:58.924 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-18T11:32:58.924 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 5h/5d Inode: 679 Links: 1 2024-02-18T11:32:58.924 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-18T11:32:58.924 INFO:teuthology.orchestra.run.smithi188.stdout:Context: system_u:object_r:device_t:s0 2024-02-18T11:32:58.924 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2024-02-18 11:32:33.798582034 +0000 2024-02-18T11:32:58.924 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2024-02-18 11:27:51.576737948 +0000 2024-02-18T11:32:58.925 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2024-02-18 11:27:51.576737948 +0000 2024-02-18T11:32:58.925 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: 2024-02-18 11:27:51.576737948 +0000 2024-02-18T11:32:58.925 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-18T11:32:58.994 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2024-02-18T11:32:58.994 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2024-02-18T11:32:58.994 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.000212045 s, 2.4 MB/s 2024-02-18T11:32:58.995 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-18T11:32:59.031 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:58 smithi188 ceph-mon[36925]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:32:59.031 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:32:58 smithi188 ceph-mon[36925]: Deploying daemon mgr.z on smithi188 2024-02-18T11:32:59.048 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_3 2024-02-18T11:32:59.105 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-18T11:32:59.105 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-18T11:32:59.106 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 5h/5d Inode: 698 Links: 1 2024-02-18T11:32:59.106 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-18T11:32:59.106 INFO:teuthology.orchestra.run.smithi188.stdout:Context: system_u:object_r:device_t:s0 2024-02-18T11:32:59.106 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2024-02-18 11:32:33.798582034 +0000 2024-02-18T11:32:59.106 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2024-02-18 11:27:51.917731887 +0000 2024-02-18T11:32:59.106 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2024-02-18 11:27:51.917731887 +0000 2024-02-18T11:32:59.106 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: 2024-02-18 11:27:51.917731887 +0000 2024-02-18T11:32:59.106 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-18T11:32:59.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:58 smithi012 ceph-mon[32460]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:32:59.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:32:58 smithi012 ceph-mon[32460]: Deploying daemon mgr.z on smithi188 2024-02-18T11:32:59.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:58 smithi158 ceph-mon[36626]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:32:59.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:32:58 smithi158 ceph-mon[36626]: Deploying daemon mgr.z on smithi188 2024-02-18T11:32:59.314 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2024-02-18T11:32:59.314 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2024-02-18T11:32:59.314 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.000241317 s, 2.1 MB/s 2024-02-18T11:32:59.315 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-18T11:32:59.330 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_4 2024-02-18T11:32:59.390 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-18T11:32:59.390 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-18T11:32:59.390 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 5h/5d Inode: 718 Links: 1 2024-02-18T11:32:59.390 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-18T11:32:59.390 INFO:teuthology.orchestra.run.smithi188.stdout:Context: system_u:object_r:device_t:s0 2024-02-18T11:32:59.391 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2024-02-18 11:32:59.260143303 +0000 2024-02-18T11:32:59.391 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2024-02-18 11:27:52.261725773 +0000 2024-02-18T11:32:59.391 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2024-02-18 11:27:52.261725773 +0000 2024-02-18T11:32:59.391 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: 2024-02-18 11:27:52.261725773 +0000 2024-02-18T11:32:59.391 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-18T11:32:59.514 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2024-02-18T11:32:59.514 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2024-02-18T11:32:59.514 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.000183989 s, 2.8 MB/s 2024-02-18T11:32:59.515 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-18T11:32:59.530 INFO:tasks.cephadm:Deploying osd.0 on smithi012 with /dev/vg_nvme/lv_4... 2024-02-18T11:32:59.530 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-02-18T11:32:59.605 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Feb 18 11:32:59 smithi188 systemd[1]: Starting Ceph mgr.z for 0c8bb0de-ce51-11ee-95ba-87774f69a715... 2024-02-18T11:32:59.777 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T11:32:59.991 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Feb 18 11:32:59 smithi188 podman[38780]: 2024-02-18 11:32:59.643089996 +0000 UTC m=+0.018000320 image pull 53c33ee3bf62dc411b650f86eedbbee7698efb1001b4c063271dcd8d65b41165 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1 2024-02-18T11:32:59.991 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Feb 18 11:32:59 smithi188 podman[38780]: 2024-02-18 11:32:59.740966229 +0000 UTC m=+0.115876535 container create 9be720b49c12510eca4c7f89186401828e2a11c54adfb1ba092b0f421af76b8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-z, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-18T11:32:59.991 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Feb 18 11:32:59 smithi188 podman[38780]: 2024-02-18 11:32:59.968379978 +0000 UTC m=+0.343290288 container init 9be720b49c12510eca4c7f89186401828e2a11c54adfb1ba092b0f421af76b8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-z, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-18T11:32:59.992 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Feb 18 11:32:59 smithi188 podman[38780]: 2024-02-18 11:32:59.970953384 +0000 UTC m=+0.345863687 container start 9be720b49c12510eca4c7f89186401828e2a11c54adfb1ba092b0f421af76b8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-z, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240217.015652-1c920af) 2024-02-18T11:33:00.355 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Feb 18 11:32:59 smithi188 ceph-mgr[38799]: -- 172.21.15.188:0/2470997844 <== mon.1 v2:172.21.15.188:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x564cf870a1a0 con 0x564cf86e0c00 2024-02-18T11:33:00.355 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Feb 18 11:33:00 smithi188 bash[38780]: 9be720b49c12510eca4c7f89186401828e2a11c54adfb1ba092b0f421af76b8d 2024-02-18T11:33:00.355 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Feb 18 11:33:00 smithi188 systemd[1]: Started Ceph mgr.z for 0c8bb0de-ce51-11ee-95ba-87774f69a715. 2024-02-18T11:33:00.780 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Feb 18 11:33:00 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-z[38795]: 2024-02-18T11:33:00.528+0000 7fd890244200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-18T11:33:00.780 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Feb 18 11:33:00 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-z[38795]: 2024-02-18T11:33:00.622+0000 7fd890244200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-18T11:33:00.897 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:00 smithi012 ceph-mon[32460]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:33:00.897 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:00 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:00.897 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:00 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:00.897 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:00 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:00.897 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:00 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-18T11:33:00.897 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:00 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-18T11:33:00.897 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:00 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-18T11:33:00.897 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:00 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:33:00.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:00 smithi158 ceph-mon[36626]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:33:00.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:00 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:00.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:00 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:00.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:00 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:00.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:00 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-18T11:33:00.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:00 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-18T11:33:00.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:00 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-18T11:33:00.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:00 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:33:01.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:00 smithi188 ceph-mon[36925]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:33:01.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:00 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:01.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:00 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:01.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:00 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:01.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:00 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-18T11:33:01.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:00 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-18T11:33:01.106 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:00 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-18T11:33:01.106 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:00 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:33:01.463 INFO:teuthology.orchestra.run.smithi012.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-02-18T11:33:01.463 INFO:teuthology.orchestra.run.smithi012.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-02-18T11:33:01.463 INFO:teuthology.orchestra.run.smithi012.stderr: stderr: 10+0 records in 2024-02-18T11:33:01.463 INFO:teuthology.orchestra.run.smithi012.stderr:10+0 records out 2024-02-18T11:33:01.463 INFO:teuthology.orchestra.run.smithi012.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011103 s, 944 MB/s 2024-02-18T11:33:01.464 INFO:teuthology.orchestra.run.smithi012.stderr:--> Zapping successful for: 2024-02-18T11:33:01.498 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Feb 18 11:33:01 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-z[38795]: 2024-02-18T11:33:01.262+0000 7fd890244200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-18T11:33:01.498 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Feb 18 11:33:01 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-z[38795]: 2024-02-18T11:33:01.354+0000 7fd890244200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-18T11:33:01.498 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Feb 18 11:33:01 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-z[38795]: 2024-02-18T11:33:01.503+0000 7fd890244200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-18T11:33:01.783 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Feb 18 11:33:01 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-z[38795]: 2024-02-18T11:33:01.599+0000 7fd890244200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-18T11:33:01.784 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Feb 18 11:33:01 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-z[38795]: 2024-02-18T11:33:01.682+0000 7fd890244200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-18T11:33:01.883 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:01 smithi158 ceph-mon[36626]: Deploying daemon mgr.y on smithi158 2024-02-18T11:33:02.041 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:01 smithi012 ceph-mon[32460]: Deploying daemon mgr.y on smithi158 2024-02-18T11:33:02.052 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:01 smithi188 ceph-mon[36925]: Deploying daemon mgr.y on smithi158 2024-02-18T11:33:02.052 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Feb 18 11:33:01 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-z[38795]: 2024-02-18T11:33:01.950+0000 7fd890244200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-18T11:33:02.052 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Feb 18 11:33:02 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-z[38795]: 2024-02-18T11:33:02.057+0000 7fd890244200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-18T11:33:02.068 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi012:vg_nvme/lv_4 2024-02-18T11:33:02.240 INFO:journalctl@ceph.mgr.y.smithi158.stdout:Feb 18 11:33:02 smithi158 systemd[1]: Starting Ceph mgr.y for 0c8bb0de-ce51-11ee-95ba-87774f69a715... 2024-02-18T11:33:02.315 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T11:33:02.355 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Feb 18 11:33:02 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-z[38795]: 2024-02-18T11:33:02.143+0000 7fd890244200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-18T11:33:02.355 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Feb 18 11:33:02 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-z[38795]: 2024-02-18T11:33:02.241+0000 7fd890244200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-18T11:33:02.496 INFO:journalctl@ceph.mgr.y.smithi158.stdout:Feb 18 11:33:02 smithi158 podman[37998]: 2024-02-18 11:33:02.190039153 +0000 UTC m=+0.025274185 image pull 53c33ee3bf62dc411b650f86eedbbee7698efb1001b4c063271dcd8d65b41165 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1 2024-02-18T11:33:02.496 INFO:journalctl@ceph.mgr.y.smithi158.stdout:Feb 18 11:33:02 smithi158 podman[37998]: 2024-02-18 11:33:02.298032875 +0000 UTC m=+0.133267885 container create 7fc0f5c3029b2bd800258c2143e4d71ec90f08a71c72491e0b15cc8957462650 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-y, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-18T11:33:02.785 INFO:journalctl@ceph.mgr.y.smithi158.stdout:Feb 18 11:33:02 smithi158 podman[37998]: 2024-02-18 11:33:02.508624332 +0000 UTC m=+0.343859351 container init 7fc0f5c3029b2bd800258c2143e4d71ec90f08a71c72491e0b15cc8957462650 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-y, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-02-18T11:33:02.785 INFO:journalctl@ceph.mgr.y.smithi158.stdout:Feb 18 11:33:02 smithi158 podman[37998]: 2024-02-18 11:33:02.511257802 +0000 UTC m=+0.346492821 container start 7fc0f5c3029b2bd800258c2143e4d71ec90f08a71c72491e0b15cc8957462650 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-02-18T11:33:02.785 INFO:journalctl@ceph.mgr.y.smithi158.stdout:Feb 18 11:33:02 smithi158 ceph-mgr[38018]: -- 172.21.15.158:0/2049924294 <== mon.2 v2:172.21.15.158:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55faf84e41a0 con 0x55faf84ba800 2024-02-18T11:33:02.785 INFO:journalctl@ceph.mgr.y.smithi158.stdout:Feb 18 11:33:02 smithi158 bash[37998]: 7fc0f5c3029b2bd800258c2143e4d71ec90f08a71c72491e0b15cc8957462650 2024-02-18T11:33:02.785 INFO:journalctl@ceph.mgr.y.smithi158.stdout:Feb 18 11:33:02 smithi158 systemd[1]: Started Ceph mgr.y for 0c8bb0de-ce51-11ee-95ba-87774f69a715. 2024-02-18T11:33:02.786 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:02 smithi188 ceph-mon[36925]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:33:02.786 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:02 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:02.786 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:02 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:02.786 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:02 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:02.786 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:02 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:02.786 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Feb 18 11:33:02 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-z[38795]: 2024-02-18T11:33:02.424+0000 7fd890244200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-18T11:33:03.075 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:02 smithi158 ceph-mon[36626]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:33:03.075 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:02 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:03.076 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:02 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:03.076 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:02 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:03.076 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:02 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:03.172 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:02 smithi012 ceph-mon[32460]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:33:03.172 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:02 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:03.172 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:02 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:03.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:02 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:03.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:02 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:03.495 INFO:journalctl@ceph.mgr.y.smithi158.stdout:Feb 18 11:33:03 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-y[38014]: 2024-02-18T11:33:03.080+0000 7f881ff14200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-18T11:33:03.496 INFO:journalctl@ceph.mgr.y.smithi158.stdout:Feb 18 11:33:03 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-y[38014]: 2024-02-18T11:33:03.175+0000 7f881ff14200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-18T11:33:03.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:33:03.676+0000 7fe8f13c5700 1 -- 172.21.15.12:0/4244100058 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe8ec1051b0 msgr2=0x7fe8ec1075a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:33:03.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:33:03.676+0000 7fe8f13c5700 1 --2- 172.21.15.12:0/4244100058 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe8ec1051b0 0x7fe8ec1075a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4009990 tx=0x7fe8d4009ca0 comp rx=0 tx=0).stop 2024-02-18T11:33:03.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:33:03.676+0000 7fe8f13c5700 1 -- 172.21.15.12:0/4244100058 shutdown_connections 2024-02-18T11:33:03.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:33:03.676+0000 7fe8f13c5700 1 --2- 172.21.15.12:0/4244100058 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8ec107ae0 0x7fe8ec109ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:33:03.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:33:03.676+0000 7fe8f13c5700 1 --2- 172.21.15.12:0/4244100058 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe8ec102880 0x7fe8ec104c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:33:03.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:33:03.676+0000 7fe8f13c5700 1 --2- 172.21.15.12:0/4244100058 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe8ec1051b0 0x7fe8ec1075a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:33:03.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:33:03.676+0000 7fe8f13c5700 1 -- 172.21.15.12:0/4244100058 >> 172.21.15.12:0/4244100058 conn(0x7fe8ec0fc210 msgr2=0x7fe8ec0fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:33:03.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:33:03.676+0000 7fe8f13c5700 1 -- 172.21.15.12:0/4244100058 shutdown_connections 2024-02-18T11:33:03.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:33:03.676+0000 7fe8f13c5700 1 -- 172.21.15.12:0/4244100058 wait complete. 2024-02-18T11:33:03.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:33:03.677+0000 7fe8f13c5700 1 Processor -- start 2024-02-18T11:33:03.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:33:03.677+0000 7fe8f13c5700 1 -- start start 2024-02-18T11:33:03.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:33:03.677+0000 7fe8f13c5700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8ec102880 0x7fe8ec197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:33:03.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:33:03.677+0000 7fe8f13c5700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe8ec1051b0 0x7fe8ec198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:33:03.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:33:03.677+0000 7fe8f13c5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe8ec107ae0 0x7fe8ec19f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:33:03.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:33:03.677+0000 7fe8f13c5700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ec19f7a0 con 0x7fe8ec107ae0 2024-02-18T11:33:03.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:33:03.677+0000 7fe8f13c5700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ec19f8e0 con 0x7fe8ec1051b0 2024-02-18T11:33:03.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:33:03.677+0000 7fe8f13c5700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ec19faa0 con 0x7fe8ec102880 2024-02-18T11:33:03.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:33:03.677+0000 7fe8eaffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8ec102880 0x7fe8ec197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:33:03.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:33:03.677+0000 7fe8eb7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe8ec107ae0 0x7fe8ec19f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:33:03.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:33:03.677+0000 7fe8ea7fc700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe8ec1051b0 0x7fe8ec198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:33:03.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:33:03.677+0000 7fe8eaffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8ec102880 0x7fe8ec197c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.12:41550/0 (socket says 172.21.15.12:41550) 2024-02-18T11:33:03.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:33:03.677+0000 7fe8eaffd700 1 -- 172.21.15.12:0/899729893 learned_addr learned my addr 172.21.15.12:0/899729893 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:33:03.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:33:03.678+0000 7fe8eb7fe700 1 -- 172.21.15.12:0/899729893 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8ec102880 msgr2=0x7fe8ec197c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:33:03.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:33:03.678+0000 7fe8eb7fe700 1 --2- 172.21.15.12:0/899729893 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8ec102880 0x7fe8ec197c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:33:03.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:33:03.678+0000 7fe8eb7fe700 1 -- 172.21.15.12:0/899729893 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe8ec1051b0 msgr2=0x7fe8ec198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:33:03.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:33:03.678+0000 7fe8eb7fe700 1 --2- 172.21.15.12:0/899729893 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe8ec1051b0 0x7fe8ec198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:33:03.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:33:03.678+0000 7fe8eb7fe700 1 -- 172.21.15.12:0/899729893 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8d40095f0 con 0x7fe8ec107ae0 2024-02-18T11:33:03.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:33:03.678+0000 7fe8eb7fe700 1 --2- 172.21.15.12:0/899729893 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe8ec107ae0 0x7fe8ec19f1d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc007ee0 tx=0x7fe8dc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:33:03.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:33:03.678+0000 7fe8e3fff700 1 -- 172.21.15.12:0/899729893 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc009ea0 con 0x7fe8ec107ae0 2024-02-18T11:33:03.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:33:03.678+0000 7fe8e3fff700 1 -- 172.21.15.12:0/899729893 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe8dc00f040 con 0x7fe8ec107ae0 2024-02-18T11:33:03.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:33:03.678+0000 7fe8f13c5700 1 -- 172.21.15.12:0/899729893 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8ec19fd80 con 0x7fe8ec107ae0 2024-02-18T11:33:03.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:33:03.678+0000 7fe8e3fff700 1 -- 172.21.15.12:0/899729893 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc014600 con 0x7fe8ec107ae0 2024-02-18T11:33:03.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:33:03.678+0000 7fe8f13c5700 1 -- 172.21.15.12:0/899729893 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8ec1a0170 con 0x7fe8ec107ae0 2024-02-18T11:33:03.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:33:03.679+0000 7fe8e3fff700 1 -- 172.21.15.12:0/899729893 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) v1 ==== 46147+0+0 (secure 0 0 0) 0x7fe8dc014760 con 0x7fe8ec107ae0 2024-02-18T11:33:03.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:33:03.679+0000 7fe8f13c5700 1 -- 172.21.15.12:0/899729893 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8ec0fdd90 con 0x7fe8ec107ae0 2024-02-18T11:33:03.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:33:03.679+0000 7fe8e3fff700 1 --2- 172.21.15.12:0/899729893 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fe8d80393f0 0x7fe8d803b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:33:03.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:33:03.679+0000 7fe8e3fff700 1 -- 172.21.15.12:0/899729893 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe8dc04b8f0 con 0x7fe8ec107ae0 2024-02-18T11:33:03.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:33:03.679+0000 7fe8eaffd700 1 --2- 172.21.15.12:0/899729893 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fe8d80393f0 0x7fe8d803b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:33:03.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:33:03.679+0000 7fe8eaffd700 1 --2- 172.21.15.12:0/899729893 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fe8d80393f0 0x7fe8d803b8b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec199130 tx=0x7fe8e4007480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:33:03.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:33:03.684+0000 7fe8e3fff700 1 -- 172.21.15.12:0/899729893 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe8dc013e00 con 0x7fe8ec107ae0 2024-02-18T11:33:03.741 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Feb 18 11:33:03 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-z[38795]: 2024-02-18T11:33:03.469+0000 7fd890244200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-18T11:33:03.741 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Feb 18 11:33:03 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-z[38795]: 2024-02-18T11:33:03.599+0000 7fd890244200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-18T11:33:03.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:33:03.896+0000 7fe8f13c5700 1 -- 172.21.15.12:0/899729893 --> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fe8ec02d8b0 con 0x7fe8d80393f0 2024-02-18T11:33:03.954 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:03 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:33:03.955 INFO:journalctl@ceph.mgr.y.smithi158.stdout:Feb 18 11:33:03 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-y[38014]: 2024-02-18T11:33:03.865+0000 7f881ff14200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-18T11:33:03.955 INFO:journalctl@ceph.mgr.y.smithi158.stdout:Feb 18 11:33:03 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-y[38014]: 2024-02-18T11:33:03.959+0000 7f881ff14200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-18T11:33:04.014 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:03 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:33:04.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:03 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:33:04.246 INFO:journalctl@ceph.mgr.y.smithi158.stdout:Feb 18 11:33:04 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-y[38014]: 2024-02-18T11:33:04.111+0000 7f881ff14200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-18T11:33:04.246 INFO:journalctl@ceph.mgr.y.smithi158.stdout:Feb 18 11:33:04 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-y[38014]: 2024-02-18T11:33:04.207+0000 7f881ff14200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-18T11:33:04.355 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Feb 18 11:33:04 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-z[38795]: 2024-02-18T11:33:03.951+0000 7fd890244200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-18T11:33:04.355 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Feb 18 11:33:04 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-z[38795]: 2024-02-18T11:33:04.040+0000 7fd890244200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-18T11:33:04.355 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Feb 18 11:33:04 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-z[38795]: 2024-02-18T11:33:04.244+0000 7fd890244200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-18T11:33:04.555 INFO:journalctl@ceph.mgr.y.smithi158.stdout:Feb 18 11:33:04 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-y[38014]: 2024-02-18T11:33:04.295+0000 7f881ff14200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-18T11:33:04.555 INFO:journalctl@ceph.mgr.y.smithi158.stdout:Feb 18 11:33:04 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-y[38014]: 2024-02-18T11:33:04.559+0000 7f881ff14200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-18T11:33:04.707 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Feb 18 11:33:04 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-z[38795]: 2024-02-18T11:33:04.432+0000 7fd890244200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-18T11:33:04.846 INFO:journalctl@ceph.mgr.y.smithi158.stdout:Feb 18 11:33:04 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-y[38014]: 2024-02-18T11:33:04.669+0000 7f881ff14200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-18T11:33:04.846 INFO:journalctl@ceph.mgr.y.smithi158.stdout:Feb 18 11:33:04 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-y[38014]: 2024-02-18T11:33:04.754+0000 7f881ff14200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-18T11:33:04.846 INFO:journalctl@ceph.mgr.y.smithi158.stdout:Feb 18 11:33:04 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-y[38014]: 2024-02-18T11:33:04.850+0000 7f881ff14200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-18T11:33:04.846 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:04 smithi158 ceph-mon[36626]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:33:04.846 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:04 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T11:33:04.846 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:04 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T11:33:04.846 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:04 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:33:04.846 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:04 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:04.846 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:04 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:04.846 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:04 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:04.847 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:04 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:04.847 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:04 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:33:04.847 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:04 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:33:04.847 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:04 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:04.847 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:04 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-18T11:33:04.847 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:04 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-18T11:33:04.847 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:04 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:33:04.969 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Feb 18 11:33:04 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-z[38795]: 2024-02-18T11:33:04.712+0000 7fd890244200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-18T11:33:04.969 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Feb 18 11:33:04 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-z[38795]: 2024-02-18T11:33:04.974+0000 7fd890244200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-18T11:33:04.969 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:04 smithi188 ceph-mon[36925]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:33:04.970 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:04 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T11:33:04.970 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:04 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T11:33:04.970 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:04 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:33:04.970 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:04 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:04.970 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:04 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:04.970 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:04 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:04.970 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:04 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:04.970 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:04 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:33:04.970 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:04 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:33:04.970 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:04 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:04.970 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:04 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-18T11:33:04.970 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:04 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-18T11:33:04.970 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:04 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:33:05.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:04 smithi012 ceph-mon[32460]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:33:05.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:04 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T11:33:05.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:04 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T11:33:05.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:04 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:33:05.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:04 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:05.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:04 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:05.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:04 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:05.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:04 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:05.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:04 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:33:05.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:04 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:33:05.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:04 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:05.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:04 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-18T11:33:05.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:04 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-18T11:33:05.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:04 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:33:05.245 INFO:journalctl@ceph.mgr.y.smithi158.stdout:Feb 18 11:33:05 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-y[38014]: 2024-02-18T11:33:05.025+0000 7f881ff14200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-18T11:33:05.355 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Feb 18 11:33:05 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-z[38795]: 2024-02-18T11:33:05.210+0000 7fd890244200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-18T11:33:05.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:05 smithi012 ceph-mon[32460]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:33:05.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:05 smithi012 ceph-mon[32460]: Reconfiguring mgr.x (unknown last config time)... 2024-02-18T11:33:05.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:05 smithi012 ceph-mon[32460]: Reconfiguring daemon mgr.x on smithi012 2024-02-18T11:33:06.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:05 smithi188 ceph-mon[36925]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:33:06.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:05 smithi188 ceph-mon[36925]: Reconfiguring mgr.x (unknown last config time)... 2024-02-18T11:33:06.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:05 smithi188 ceph-mon[36925]: Reconfiguring daemon mgr.x on smithi012 2024-02-18T11:33:06.105 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Feb 18 11:33:06 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-z[38795]: 2024-02-18T11:33:06.016+0000 7fd890244200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-18T11:33:06.120 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:05 smithi158 ceph-mon[36626]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:33:06.120 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:05 smithi158 ceph-mon[36626]: Reconfiguring mgr.x (unknown last config time)... 2024-02-18T11:33:06.120 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:05 smithi158 ceph-mon[36626]: Reconfiguring daemon mgr.x on smithi012 2024-02-18T11:33:06.120 INFO:journalctl@ceph.mgr.y.smithi158.stdout:Feb 18 11:33:05 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-y[38014]: 2024-02-18T11:33:05.997+0000 7f881ff14200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-18T11:33:06.120 INFO:journalctl@ceph.mgr.y.smithi158.stdout:Feb 18 11:33:06 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-y[38014]: 2024-02-18T11:33:06.124+0000 7f881ff14200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-18T11:33:06.728 INFO:journalctl@ceph.mgr.y.smithi158.stdout:Feb 18 11:33:06 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-y[38014]: 2024-02-18T11:33:06.455+0000 7f881ff14200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-18T11:33:06.728 INFO:journalctl@ceph.mgr.y.smithi158.stdout:Feb 18 11:33:06 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-y[38014]: 2024-02-18T11:33:06.543+0000 7f881ff14200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-18T11:33:06.728 INFO:journalctl@ceph.mgr.y.smithi158.stdout:Feb 18 11:33:06 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-y[38014]: 2024-02-18T11:33:06.733+0000 7f881ff14200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-18T11:33:06.855 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Feb 18 11:33:06 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-z[38795]: 2024-02-18T11:33:06.588+0000 7fd890244200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-18T11:33:06.855 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Feb 18 11:33:06 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-z[38795]: 2024-02-18T11:33:06.686+0000 7fd890244200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-18T11:33:06.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:06 smithi188 ceph-mon[36925]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:33:06.856 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:06 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:06.856 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:06 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/4133447862' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35f15ea6-deb8-48a1-a7dd-ab649ec13533"}]: dispatch 2024-02-18T11:33:06.856 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:06 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/4133447862' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "35f15ea6-deb8-48a1-a7dd-ab649ec13533"}]': finished 2024-02-18T11:33:06.856 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:06 smithi188 ceph-mon[36925]: osdmap e5: 1 total, 0 up, 1 in 2024-02-18T11:33:06.856 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:06 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-18T11:33:06.995 INFO:journalctl@ceph.mgr.y.smithi158.stdout:Feb 18 11:33:06 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-y[38014]: 2024-02-18T11:33:06.915+0000 7f881ff14200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-18T11:33:06.996 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:06 smithi158 ceph-mon[36626]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:33:06.996 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:06 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:06.996 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:06 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/4133447862' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35f15ea6-deb8-48a1-a7dd-ab649ec13533"}]: dispatch 2024-02-18T11:33:06.996 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:06 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/4133447862' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "35f15ea6-deb8-48a1-a7dd-ab649ec13533"}]': finished 2024-02-18T11:33:06.996 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:06 smithi158 ceph-mon[36626]: osdmap e5: 1 total, 0 up, 1 in 2024-02-18T11:33:06.996 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:06 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-18T11:33:07.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:06 smithi012 ceph-mon[32460]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:33:07.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:06 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:07.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:06 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/4133447862' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35f15ea6-deb8-48a1-a7dd-ab649ec13533"}]: dispatch 2024-02-18T11:33:07.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:06 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/4133447862' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "35f15ea6-deb8-48a1-a7dd-ab649ec13533"}]': finished 2024-02-18T11:33:07.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:06 smithi012 ceph-mon[32460]: osdmap e5: 1 total, 0 up, 1 in 2024-02-18T11:33:07.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:06 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-18T11:33:07.355 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Feb 18 11:33:06 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-z[38795]: 2024-02-18T11:33:06.902+0000 7fd890244200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-18T11:33:07.445 INFO:journalctl@ceph.mgr.y.smithi158.stdout:Feb 18 11:33:07 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-y[38014]: 2024-02-18T11:33:07.191+0000 7f881ff14200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-18T11:33:07.445 INFO:journalctl@ceph.mgr.y.smithi158.stdout:Feb 18 11:33:07 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-y[38014]: 2024-02-18T11:33:07.449+0000 7f881ff14200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-18T11:33:07.745 INFO:journalctl@ceph.mgr.y.smithi158.stdout:Feb 18 11:33:07 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-y[38014]: 2024-02-18T11:33:07.682+0000 7f881ff14200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-18T11:33:07.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:33:07.817+0000 7fe8e3fff700 1 -- 172.21.15.12:0/899729893 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mgrmap(e 14) v1 ==== 91668+0+0 (secure 0 0 0) 0x7fe8dc01f7c0 con 0x7fe8ec107ae0 2024-02-18T11:33:08.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:07 smithi188 ceph-mon[36925]: Standby manager daemon z started 2024-02-18T11:33:08.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:07 smithi188 ceph-mon[36925]: from='mgr.? 172.21.15.188:0/3227236777' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-18T11:33:08.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:07 smithi188 ceph-mon[36925]: from='mgr.? 172.21.15.188:0/3227236777' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-18T11:33:08.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:07 smithi188 ceph-mon[36925]: from='mgr.? 172.21.15.188:0/3227236777' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-18T11:33:08.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:07 smithi188 ceph-mon[36925]: from='mgr.? 172.21.15.188:0/3227236777' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-18T11:33:08.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:07 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/2515668397' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T11:33:08.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:07 smithi012 ceph-mon[32460]: Standby manager daemon z started 2024-02-18T11:33:08.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:07 smithi012 ceph-mon[32460]: from='mgr.? 172.21.15.188:0/3227236777' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-18T11:33:08.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:07 smithi012 ceph-mon[32460]: from='mgr.? 172.21.15.188:0/3227236777' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-18T11:33:08.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:07 smithi012 ceph-mon[32460]: from='mgr.? 172.21.15.188:0/3227236777' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-18T11:33:08.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:07 smithi012 ceph-mon[32460]: from='mgr.? 172.21.15.188:0/3227236777' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-18T11:33:08.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:07 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/2515668397' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T11:33:08.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:07 smithi158 ceph-mon[36626]: Standby manager daemon z started 2024-02-18T11:33:08.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:07 smithi158 ceph-mon[36626]: from='mgr.? 172.21.15.188:0/3227236777' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-18T11:33:08.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:07 smithi158 ceph-mon[36626]: from='mgr.? 172.21.15.188:0/3227236777' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-18T11:33:08.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:07 smithi158 ceph-mon[36626]: from='mgr.? 172.21.15.188:0/3227236777' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-18T11:33:08.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:07 smithi158 ceph-mon[36626]: from='mgr.? 172.21.15.188:0/3227236777' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-18T11:33:08.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:07 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/2515668397' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T11:33:08.745 INFO:journalctl@ceph.mgr.y.smithi158.stdout:Feb 18 11:33:08 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-y[38014]: 2024-02-18T11:33:08.481+0000 7f881ff14200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-18T11:33:09.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:08 smithi188 ceph-mon[36925]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:33:09.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:08 smithi188 ceph-mon[36925]: mgrmap e14: x(active, since 76s), standbys: z 2024-02-18T11:33:09.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:08 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-18T11:33:09.144 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:08 smithi158 ceph-mon[36626]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:33:09.144 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:08 smithi158 ceph-mon[36626]: mgrmap e14: x(active, since 76s), standbys: z 2024-02-18T11:33:09.144 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:08 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-18T11:33:09.144 INFO:journalctl@ceph.mgr.y.smithi158.stdout:Feb 18 11:33:09 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-y[38014]: 2024-02-18T11:33:09.052+0000 7f881ff14200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-18T11:33:09.144 INFO:journalctl@ceph.mgr.y.smithi158.stdout:Feb 18 11:33:09 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-y[38014]: 2024-02-18T11:33:09.148+0000 7f881ff14200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-18T11:33:09.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:08 smithi012 ceph-mon[32460]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:33:09.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:08 smithi012 ceph-mon[32460]: mgrmap e14: x(active, since 76s), standbys: z 2024-02-18T11:33:09.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:08 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-18T11:33:09.495 INFO:journalctl@ceph.mgr.y.smithi158.stdout:Feb 18 11:33:09 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-y[38014]: 2024-02-18T11:33:09.363+0000 7f881ff14200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-18T11:33:09.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:33:09.825+0000 7fe8e3fff700 1 -- 172.21.15.12:0/899729893 <== mon.0 v2:172.21.15.12:3300/0 8 ==== mgrmap(e 15) v1 ==== 137189+0+0 (secure 0 0 0) 0x7fe8dc01eac0 con 0x7fe8ec107ae0 2024-02-18T11:33:10.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:09 smithi188 ceph-mon[36925]: Standby manager daemon y started 2024-02-18T11:33:10.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:09 smithi188 ceph-mon[36925]: from='mgr.? 172.21.15.158:0/808484764' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-02-18T11:33:10.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:09 smithi188 ceph-mon[36925]: from='mgr.? 172.21.15.158:0/808484764' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-18T11:33:10.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:09 smithi188 ceph-mon[36925]: from='mgr.? 172.21.15.158:0/808484764' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-02-18T11:33:10.106 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:09 smithi188 ceph-mon[36925]: from='mgr.? 172.21.15.158:0/808484764' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-18T11:33:10.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:09 smithi012 ceph-mon[32460]: Standby manager daemon y started 2024-02-18T11:33:10.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:09 smithi012 ceph-mon[32460]: from='mgr.? 172.21.15.158:0/808484764' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-02-18T11:33:10.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:09 smithi012 ceph-mon[32460]: from='mgr.? 172.21.15.158:0/808484764' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-18T11:33:10.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:09 smithi012 ceph-mon[32460]: from='mgr.? 172.21.15.158:0/808484764' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-02-18T11:33:10.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:09 smithi012 ceph-mon[32460]: from='mgr.? 172.21.15.158:0/808484764' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-18T11:33:10.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:09 smithi158 ceph-mon[36626]: Standby manager daemon y started 2024-02-18T11:33:10.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:09 smithi158 ceph-mon[36626]: from='mgr.? 172.21.15.158:0/808484764' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-02-18T11:33:10.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:09 smithi158 ceph-mon[36626]: from='mgr.? 172.21.15.158:0/808484764' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-18T11:33:10.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:09 smithi158 ceph-mon[36626]: from='mgr.? 172.21.15.158:0/808484764' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-02-18T11:33:10.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:09 smithi158 ceph-mon[36626]: from='mgr.? 172.21.15.158:0/808484764' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-18T11:33:10.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:10 smithi012 ceph-mon[32460]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:33:10.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:10 smithi012 ceph-mon[32460]: mgrmap e15: x(active, since 78s), standbys: z, y 2024-02-18T11:33:10.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:10 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-18T11:33:11.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:10 smithi188 ceph-mon[36925]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:33:11.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:10 smithi188 ceph-mon[36925]: mgrmap e15: x(active, since 78s), standbys: z, y 2024-02-18T11:33:11.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:10 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-18T11:33:11.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:10 smithi158 ceph-mon[36626]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:33:11.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:10 smithi158 ceph-mon[36626]: mgrmap e15: x(active, since 78s), standbys: z, y 2024-02-18T11:33:11.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:10 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-18T11:33:12.968 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:12 smithi012 ceph-mon[32460]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:33:12.968 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:12 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:12.968 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:12 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:12.968 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:12 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:33:12.968 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:12 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:33:12.968 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:12 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:33:12.969 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:12 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:13.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:12 smithi188 ceph-mon[36925]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:33:13.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:12 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:13.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:12 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:13.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:12 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:33:13.106 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:12 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:33:13.106 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:12 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:33:13.106 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:12 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:13.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:12 smithi158 ceph-mon[36626]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:33:13.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:12 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:13.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:12 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:13.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:12 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:33:13.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:12 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:33:13.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:12 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:33:13.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:12 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:15.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:14 smithi188 ceph-mon[36925]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:33:15.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:14 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-18T11:33:15.106 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:14 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:33:15.169 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:14 smithi012 ceph-mon[32460]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:33:15.169 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:14 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-18T11:33:15.169 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:14 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:33:15.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:14 smithi158 ceph-mon[36626]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:33:15.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:14 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-18T11:33:15.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:14 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:33:16.042 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:15 smithi012 ceph-mon[32460]: Deploying daemon osd.0 on smithi012 2024-02-18T11:33:16.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:15 smithi188 ceph-mon[36925]: Deploying daemon osd.0 on smithi012 2024-02-18T11:33:16.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:15 smithi158 ceph-mon[36626]: Deploying daemon osd.0 on smithi012 2024-02-18T11:33:17.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:16 smithi188 ceph-mon[36925]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:33:17.135 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:16 smithi012 ceph-mon[32460]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:33:17.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:16 smithi158 ceph-mon[36626]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:33:18.862 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:18 smithi012 ceph-mon[32460]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:33:19.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:18 smithi188 ceph-mon[36925]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:33:19.247 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:18 smithi158 ceph-mon[36626]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:33:20.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:20 smithi012 ceph-mon[32460]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:33:20.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:20 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:20.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:20 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:20.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:20 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:33:20.745 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:20 smithi158 ceph-mon[36626]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:33:20.745 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:20 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:20.745 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:20 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:20.746 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:20 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:33:20.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:20 smithi188 ceph-mon[36925]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:33:20.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:20 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:20.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:20 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:20.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:20 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:33:22.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:22 smithi012 ceph-mon[32460]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:33:22.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:22 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:22.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:22 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:22.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:22 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:33:22.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:22 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:33:22.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:22 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:22.745 INFO:teuthology.orchestra.run.smithi012.stdout:Created osd(s) 0 on host 'smithi012' 2024-02-18T11:33:22.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:33:22.750+0000 7fe8e3fff700 1 -- 172.21.15.12:0/899729893 <== mgr.14152 v2:172.21.15.12:6800/1153736685 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fe8ec02d8b0 con 0x7fe8d80393f0 2024-02-18T11:33:22.746 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:22 smithi158 ceph-mon[36626]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:33:22.746 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:22 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:22.746 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:22 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:22.746 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:22 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:33:22.746 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:22 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:33:22.746 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:22 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:22.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:33:22.752+0000 7fe8f13c5700 1 -- 172.21.15.12:0/899729893 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fe8d80393f0 msgr2=0x7fe8d803b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:33:22.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:33:22.752+0000 7fe8f13c5700 1 --2- 172.21.15.12:0/899729893 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fe8d80393f0 0x7fe8d803b8b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec199130 tx=0x7fe8e4007480 comp rx=0 tx=0).stop 2024-02-18T11:33:22.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:33:22.752+0000 7fe8f13c5700 1 -- 172.21.15.12:0/899729893 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe8ec107ae0 msgr2=0x7fe8ec19f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:33:22.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:33:22.752+0000 7fe8f13c5700 1 --2- 172.21.15.12:0/899729893 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe8ec107ae0 0x7fe8ec19f1d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc007ee0 tx=0x7fe8dc00d3b0 comp rx=0 tx=0).stop 2024-02-18T11:33:22.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:33:22.752+0000 7fe8f13c5700 1 -- 172.21.15.12:0/899729893 shutdown_connections 2024-02-18T11:33:22.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:33:22.752+0000 7fe8f13c5700 1 --2- 172.21.15.12:0/899729893 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fe8d80393f0 0x7fe8d803b8b0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:33:22.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:33:22.752+0000 7fe8f13c5700 1 --2- 172.21.15.12:0/899729893 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe8ec107ae0 0x7fe8ec19f1d0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:33:22.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:33:22.752+0000 7fe8f13c5700 1 --2- 172.21.15.12:0/899729893 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8ec102880 0x7fe8ec197c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:33:22.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:33:22.752+0000 7fe8f13c5700 1 --2- 172.21.15.12:0/899729893 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe8ec1051b0 0x7fe8ec198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:33:22.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:33:22.752+0000 7fe8f13c5700 1 -- 172.21.15.12:0/899729893 >> 172.21.15.12:0/899729893 conn(0x7fe8ec0fc210 msgr2=0x7fe8ec1031c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:33:22.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:33:22.752+0000 7fe8f13c5700 1 -- 172.21.15.12:0/899729893 shutdown_connections 2024-02-18T11:33:22.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:33:22.752+0000 7fe8f13c5700 1 -- 172.21.15.12:0/899729893 wait complete. 2024-02-18T11:33:22.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:22 smithi188 ceph-mon[36925]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:33:22.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:22 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:22.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:22 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:22.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:22 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:33:22.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:22 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:33:22.856 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:22 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:23.382 DEBUG:teuthology.orchestra.run.smithi012:osd.0> sudo journalctl -f -n 0 -u ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@osd.0.service 2024-02-18T11:33:23.384 INFO:tasks.cephadm:Deploying osd.1 on smithi158 with /dev/vg_nvme/lv_4... 2024-02-18T11:33:23.384 DEBUG:teuthology.orchestra.run.smithi158:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-02-18T11:33:23.628 INFO:teuthology.orchestra.run.smithi158.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T11:33:23.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:23 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:23.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:23 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:23.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:23 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:33:23.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:23 smithi012 ceph-mon[32460]: from='osd.0 [v2:172.21.15.12:6802/536522426,v1:172.21.15.12:6803/536522426]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-18T11:33:23.966 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:23 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:23.966 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:23 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:23.966 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:23 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:33:23.966 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:23 smithi158 ceph-mon[36626]: from='osd.0 [v2:172.21.15.12:6802/536522426,v1:172.21.15.12:6803/536522426]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-18T11:33:24.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:23 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:24.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:23 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:24.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:23 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:33:24.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:23 smithi188 ceph-mon[36925]: from='osd.0 [v2:172.21.15.12:6802/536522426,v1:172.21.15.12:6803/536522426]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-18T11:33:24.846 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:24 smithi158 ceph-mon[36626]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:33:24.846 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:24 smithi158 ceph-mon[36626]: from='osd.0 [v2:172.21.15.12:6802/536522426,v1:172.21.15.12:6803/536522426]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-18T11:33:24.846 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:24 smithi158 ceph-mon[36626]: osdmap e6: 1 total, 0 up, 1 in 2024-02-18T11:33:24.846 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:24 smithi158 ceph-mon[36626]: from='osd.0 [v2:172.21.15.12:6802/536522426,v1:172.21.15.12:6803/536522426]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-02-18T11:33:24.846 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:24 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-18T11:33:25.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:24 smithi188 ceph-mon[36925]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:33:25.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:24 smithi188 ceph-mon[36925]: from='osd.0 [v2:172.21.15.12:6802/536522426,v1:172.21.15.12:6803/536522426]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-18T11:33:25.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:24 smithi188 ceph-mon[36925]: osdmap e6: 1 total, 0 up, 1 in 2024-02-18T11:33:25.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:24 smithi188 ceph-mon[36925]: from='osd.0 [v2:172.21.15.12:6802/536522426,v1:172.21.15.12:6803/536522426]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-02-18T11:33:25.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:24 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-18T11:33:25.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:24 smithi012 ceph-mon[32460]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:33:25.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:24 smithi012 ceph-mon[32460]: from='osd.0 [v2:172.21.15.12:6802/536522426,v1:172.21.15.12:6803/536522426]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-18T11:33:25.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:24 smithi012 ceph-mon[32460]: osdmap e6: 1 total, 0 up, 1 in 2024-02-18T11:33:25.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:24 smithi012 ceph-mon[32460]: from='osd.0 [v2:172.21.15.12:6802/536522426,v1:172.21.15.12:6803/536522426]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-02-18T11:33:25.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:24 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-18T11:33:25.413 INFO:teuthology.orchestra.run.smithi158.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-02-18T11:33:25.413 INFO:teuthology.orchestra.run.smithi158.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-02-18T11:33:25.413 INFO:teuthology.orchestra.run.smithi158.stderr: stderr: 10+0 records in 2024-02-18T11:33:25.413 INFO:teuthology.orchestra.run.smithi158.stderr:10+0 records out 2024-02-18T11:33:25.413 INFO:teuthology.orchestra.run.smithi158.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0126144 s, 831 MB/s 2024-02-18T11:33:25.413 INFO:teuthology.orchestra.run.smithi158.stderr:--> Zapping successful for: 2024-02-18T11:33:25.785 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:25 smithi158 ceph-mon[36626]: from='osd.0 [v2:172.21.15.12:6802/536522426,v1:172.21.15.12:6803/536522426]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-02-18T11:33:25.785 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:25 smithi158 ceph-mon[36626]: osdmap e7: 1 total, 0 up, 1 in 2024-02-18T11:33:25.785 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:25 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-18T11:33:25.785 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:25 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-18T11:33:25.786 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:25 smithi158 ceph-mon[36626]: from='osd.0 [v2:172.21.15.12:6802/536522426,v1:172.21.15.12:6803/536522426]' entity='osd.0' 2024-02-18T11:33:25.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:25 smithi012 ceph-mon[32460]: from='osd.0 [v2:172.21.15.12:6802/536522426,v1:172.21.15.12:6803/536522426]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-02-18T11:33:25.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:25 smithi012 ceph-mon[32460]: osdmap e7: 1 total, 0 up, 1 in 2024-02-18T11:33:25.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:25 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-18T11:33:25.925 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:25 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-18T11:33:25.925 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:25 smithi012 ceph-mon[32460]: from='osd.0 [v2:172.21.15.12:6802/536522426,v1:172.21.15.12:6803/536522426]' entity='osd.0' 2024-02-18T11:33:25.925 INFO:journalctl@ceph.osd.0.smithi012.stdout:Feb 18 11:33:25 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-0[44875]: 2024-02-18T11:33:25.550+0000 7f86dbbf6700 -1 osd.0 0 waiting for initial osdmap 2024-02-18T11:33:25.925 INFO:journalctl@ceph.osd.0.smithi012.stdout:Feb 18 11:33:25 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-0[44875]: 2024-02-18T11:33:25.563+0000 7f86d29df700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-18T11:33:26.071 DEBUG:teuthology.orchestra.run.smithi158:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi158:vg_nvme/lv_4 2024-02-18T11:33:26.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:25 smithi188 ceph-mon[36925]: from='osd.0 [v2:172.21.15.12:6802/536522426,v1:172.21.15.12:6803/536522426]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-02-18T11:33:26.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:25 smithi188 ceph-mon[36925]: osdmap e7: 1 total, 0 up, 1 in 2024-02-18T11:33:26.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:25 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-18T11:33:26.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:25 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-18T11:33:26.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:25 smithi188 ceph-mon[36925]: from='osd.0 [v2:172.21.15.12:6802/536522426,v1:172.21.15.12:6803/536522426]' entity='osd.0' 2024-02-18T11:33:26.310 INFO:teuthology.orchestra.run.smithi158.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T11:33:26.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:26 smithi158 ceph-mon[36626]: purged_snaps scrub starts 2024-02-18T11:33:26.996 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:26 smithi158 ceph-mon[36626]: purged_snaps scrub ok 2024-02-18T11:33:26.996 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:26 smithi158 ceph-mon[36626]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:33:26.996 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:26 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-18T11:33:26.996 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:26 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:26.996 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:26 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:26.996 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:26 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:33:26.996 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:26 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:26.996 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:26 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:33:26.996 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:26 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:33:26.996 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:26 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:26.996 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:26 smithi158 ceph-mon[36626]: osd.0 [v2:172.21.15.12:6802/536522426,v1:172.21.15.12:6803/536522426] boot 2024-02-18T11:33:26.996 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:26 smithi158 ceph-mon[36626]: osdmap e8: 1 total, 1 up, 1 in 2024-02-18T11:33:26.996 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:26 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-18T11:33:27.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:26 smithi188 ceph-mon[36925]: purged_snaps scrub starts 2024-02-18T11:33:27.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:26 smithi188 ceph-mon[36925]: purged_snaps scrub ok 2024-02-18T11:33:27.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:26 smithi188 ceph-mon[36925]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:33:27.106 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:26 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-18T11:33:27.106 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:26 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:27.106 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:26 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:27.106 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:26 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:33:27.106 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:26 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:27.106 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:26 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:33:27.106 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:26 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:33:27.106 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:26 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:27.106 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:26 smithi188 ceph-mon[36925]: osd.0 [v2:172.21.15.12:6802/536522426,v1:172.21.15.12:6803/536522426] boot 2024-02-18T11:33:27.106 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:26 smithi188 ceph-mon[36925]: osdmap e8: 1 total, 1 up, 1 in 2024-02-18T11:33:27.106 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:26 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-18T11:33:27.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:26 smithi012 ceph-mon[32460]: purged_snaps scrub starts 2024-02-18T11:33:27.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:26 smithi012 ceph-mon[32460]: purged_snaps scrub ok 2024-02-18T11:33:27.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:26 smithi012 ceph-mon[32460]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T11:33:27.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:26 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-18T11:33:27.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:26 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:27.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:26 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:27.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:26 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:33:27.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:26 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:27.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:26 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:33:27.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:26 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:33:27.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:26 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:27.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:26 smithi012 ceph-mon[32460]: osd.0 [v2:172.21.15.12:6802/536522426,v1:172.21.15.12:6803/536522426] boot 2024-02-18T11:33:27.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:26 smithi012 ceph-mon[32460]: osdmap e8: 1 total, 1 up, 1 in 2024-02-18T11:33:27.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:26 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-18T11:33:27.821 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:27 smithi158 ceph-mon[36626]: Detected new or changed devices on smithi012 2024-02-18T11:33:27.821 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:27 smithi158 ceph-mon[36626]: Adjusting osd_memory_target on smithi012 to 17093M 2024-02-18T11:33:27.828 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:33:27.832+0000 7f61ae709700 1 -- 172.21.15.158:0/3686045076 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f61a8102790 msgr2=0x7f61a8109040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:33:27.828 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:33:27.832+0000 7f61ae709700 1 --2- 172.21.15.158:0/3686045076 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f61a8102790 0x7f61a8109040 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f61a000b0d0 tx=0x7f61a000b3e0 comp rx=0 tx=0).stop 2024-02-18T11:33:27.828 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:33:27.833+0000 7f61ae709700 1 -- 172.21.15.158:0/3686045076 shutdown_connections 2024-02-18T11:33:27.828 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:33:27.833+0000 7f61ae709700 1 --2- 172.21.15.158:0/3686045076 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f61a8102790 0x7f61a8109040 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:33:27.829 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:33:27.833+0000 7f61ae709700 1 --2- 172.21.15.158:0/3686045076 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f61a8100bf0 0x7f61a8101010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:33:27.829 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:33:27.833+0000 7f61ae709700 1 --2- 172.21.15.158:0/3686045076 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61a8101df0 0x7f61a8102250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:33:27.829 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:33:27.833+0000 7f61ae709700 1 -- 172.21.15.158:0/3686045076 >> 172.21.15.158:0/3686045076 conn(0x7f61a80fc1b0 msgr2=0x7f61a80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:33:27.829 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:33:27.833+0000 7f61ae709700 1 -- 172.21.15.158:0/3686045076 shutdown_connections 2024-02-18T11:33:27.829 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:33:27.833+0000 7f61ae709700 1 -- 172.21.15.158:0/3686045076 wait complete. 2024-02-18T11:33:27.829 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:33:27.833+0000 7f61ae709700 1 Processor -- start 2024-02-18T11:33:27.829 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:33:27.833+0000 7f61ae709700 1 -- start start 2024-02-18T11:33:27.830 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:33:27.833+0000 7f61ae709700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61a8100bf0 0x7f61a810b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:33:27.831 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:33:27.834+0000 7f61ae709700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f61a8101df0 0x7f61a8109b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:33:27.831 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:33:27.834+0000 7f61ae709700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f61a8102790 0x7f61a810a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:33:27.831 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:33:27.834+0000 7f61ae709700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a810a810 con 0x7f61a8100bf0 2024-02-18T11:33:27.831 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:33:27.834+0000 7f61ae709700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a810a980 con 0x7f61a8102790 2024-02-18T11:33:27.831 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:33:27.834+0000 7f61ae709700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a81a1e70 con 0x7f61a8101df0 2024-02-18T11:33:27.831 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:33:27.834+0000 7f61acca6700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f61a8102790 0x7f61a810a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:33:27.831 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:33:27.834+0000 7f61a7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61a8100bf0 0x7f61a810b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:33:27.831 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:33:27.834+0000 7f61acca6700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f61a8102790 0x7f61a810a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.158:3300/0 says I am v2:172.21.15.158:48622/0 (socket says 172.21.15.158:48622) 2024-02-18T11:33:27.831 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:33:27.834+0000 7f61acca6700 1 -- 172.21.15.158:0/4171091661 learned_addr learned my addr 172.21.15.158:0/4171091661 (peer_addr_for_me v2:172.21.15.158:0/0) 2024-02-18T11:33:27.831 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:33:27.834+0000 7f61acca6700 1 -- 172.21.15.158:0/4171091661 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f61a8101df0 msgr2=0x7f61a8109b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:33:27.831 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:33:27.834+0000 7f61acca6700 1 --2- 172.21.15.158:0/4171091661 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f61a8101df0 0x7f61a8109b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:33:27.831 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:33:27.834+0000 7f61acca6700 1 -- 172.21.15.158:0/4171091661 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61a8100bf0 msgr2=0x7f61a810b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:33:27.832 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:33:27.834+0000 7f61acca6700 1 --2- 172.21.15.158:0/4171091661 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61a8100bf0 0x7f61a810b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:33:27.832 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:33:27.834+0000 7f61acca6700 1 -- 172.21.15.158:0/4171091661 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61a0009d00 con 0x7f61a8102790 2024-02-18T11:33:27.832 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:33:27.834+0000 7f61acca6700 1 --2- 172.21.15.158:0/4171091661 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f61a8102790 0x7f61a810a090 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f61a0001220 tx=0x7f61a0007930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:33:27.832 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:33:27.834+0000 7f61a57fa700 1 -- 172.21.15.158:0/4171091661 <== mon.2 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a000e040 con 0x7f61a8102790 2024-02-18T11:33:27.832 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:33:27.834+0000 7f61ae709700 1 -- 172.21.15.158:0/4171091661 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61a81a20f0 con 0x7f61a8102790 2024-02-18T11:33:27.832 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:33:27.834+0000 7f61a57fa700 1 -- 172.21.15.158:0/4171091661 <== mon.2 v2:172.21.15.158:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f61a0009460 con 0x7f61a8102790 2024-02-18T11:33:27.832 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:33:27.834+0000 7f61a57fa700 1 -- 172.21.15.158:0/4171091661 <== mon.2 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a001d6e0 con 0x7f61a8102790 2024-02-18T11:33:27.832 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:33:27.834+0000 7f61ae709700 1 -- 172.21.15.158:0/4171091661 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61a81a25e0 con 0x7f61a8102790 2024-02-18T11:33:27.832 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:33:27.835+0000 7f61ae709700 1 -- 172.21.15.158:0/4171091661 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61a8107ee0 con 0x7f61a8102790 2024-02-18T11:33:27.833 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:33:27.836+0000 7f61a57fa700 1 -- 172.21.15.158:0/4171091661 <== mon.2 v2:172.21.15.158:3300/0 4 ==== mgrmap(e 15) v1 ==== 137189+0+0 (secure 0 0 0) 0x7f61a0004aa0 con 0x7f61a8102790 2024-02-18T11:33:27.833 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:33:27.837+0000 7f61a57fa700 1 --2- 172.21.15.158:0/4171091661 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f61900a2750 0x7f61900a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:33:27.833 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:33:27.837+0000 7f61a57fa700 1 -- 172.21.15.158:0/4171091661 <== mon.2 v2:172.21.15.158:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1767+0+0 (secure 0 0 0) 0x7f61a00ce550 con 0x7f61a8102790 2024-02-18T11:33:27.833 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:33:27.837+0000 7f61a7fff700 1 --2- 172.21.15.158:0/4171091661 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f61900a2750 0x7f61900a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:33:27.833 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:33:27.838+0000 7f61a7fff700 1 --2- 172.21.15.158:0/4171091661 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f61900a2750 0x7f61900a4c10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f61a8101c50 tx=0x7f619800a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:33:27.836 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:33:27.841+0000 7f61a57fa700 1 -- 172.21.15.158:0/4171091661 <== mon.2 v2:172.21.15.158:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f61a0098de0 con 0x7f61a8102790 2024-02-18T11:33:28.039 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:33:28.043+0000 7f61ae709700 1 -- 172.21.15.158:0/4171091661 --> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi158:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f61a802d8b0 con 0x7f61900a2750 2024-02-18T11:33:28.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:27 smithi188 ceph-mon[36925]: Detected new or changed devices on smithi012 2024-02-18T11:33:28.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:27 smithi188 ceph-mon[36925]: Adjusting osd_memory_target on smithi012 to 17093M 2024-02-18T11:33:28.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:27 smithi012 ceph-mon[32460]: Detected new or changed devices on smithi012 2024-02-18T11:33:28.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:27 smithi012 ceph-mon[32460]: Adjusting osd_memory_target on smithi012 to 17093M 2024-02-18T11:33:28.941 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:28 smithi158 ceph-mon[36626]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-18T11:33:28.941 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:28 smithi158 ceph-mon[36626]: osdmap e9: 1 total, 1 up, 1 in 2024-02-18T11:33:28.941 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:28 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T11:33:28.941 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:28 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T11:33:28.941 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:28 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:33:29.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:28 smithi188 ceph-mon[36925]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-18T11:33:29.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:28 smithi188 ceph-mon[36925]: osdmap e9: 1 total, 1 up, 1 in 2024-02-18T11:33:29.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:28 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T11:33:29.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:28 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T11:33:29.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:28 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:33:29.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:28 smithi012 ceph-mon[32460]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-18T11:33:29.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:28 smithi012 ceph-mon[32460]: osdmap e9: 1 total, 1 up, 1 in 2024-02-18T11:33:29.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:28 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T11:33:29.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:28 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T11:33:29.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:28 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:33:29.847 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:29 smithi158 ceph-mon[36626]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi158:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:33:30.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:29 smithi188 ceph-mon[36925]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi158:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:33:30.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:29 smithi012 ceph-mon[32460]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi158:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:33:31.006 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:30 smithi188 ceph-mon[36925]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-18T11:33:31.007 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:30 smithi188 ceph-mon[36925]: from='client.? 172.21.15.158:0/3214750196' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2af937ef-d827-44e6-b805-70cffbfe1126"}]: dispatch 2024-02-18T11:33:31.007 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:30 smithi188 ceph-mon[36925]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2af937ef-d827-44e6-b805-70cffbfe1126"}]: dispatch 2024-02-18T11:33:31.007 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:30 smithi188 ceph-mon[36925]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2af937ef-d827-44e6-b805-70cffbfe1126"}]': finished 2024-02-18T11:33:31.007 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:30 smithi188 ceph-mon[36925]: osdmap e10: 2 total, 1 up, 2 in 2024-02-18T11:33:31.007 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:30 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-18T11:33:31.030 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:30 smithi158 ceph-mon[36626]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-18T11:33:31.030 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:30 smithi158 ceph-mon[36626]: from='client.? 172.21.15.158:0/3214750196' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2af937ef-d827-44e6-b805-70cffbfe1126"}]: dispatch 2024-02-18T11:33:31.030 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:30 smithi158 ceph-mon[36626]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2af937ef-d827-44e6-b805-70cffbfe1126"}]: dispatch 2024-02-18T11:33:31.030 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:30 smithi158 ceph-mon[36626]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2af937ef-d827-44e6-b805-70cffbfe1126"}]': finished 2024-02-18T11:33:31.031 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:30 smithi158 ceph-mon[36626]: osdmap e10: 2 total, 1 up, 2 in 2024-02-18T11:33:31.031 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:30 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-18T11:33:31.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:30 smithi012 ceph-mon[32460]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-18T11:33:31.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:30 smithi012 ceph-mon[32460]: from='client.? 172.21.15.158:0/3214750196' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2af937ef-d827-44e6-b805-70cffbfe1126"}]: dispatch 2024-02-18T11:33:31.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:30 smithi012 ceph-mon[32460]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2af937ef-d827-44e6-b805-70cffbfe1126"}]: dispatch 2024-02-18T11:33:31.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:30 smithi012 ceph-mon[32460]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2af937ef-d827-44e6-b805-70cffbfe1126"}]': finished 2024-02-18T11:33:31.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:30 smithi012 ceph-mon[32460]: osdmap e10: 2 total, 1 up, 2 in 2024-02-18T11:33:31.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:30 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-18T11:33:32.040 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:31 smithi158 ceph-mon[36626]: from='client.? 172.21.15.158:0/1503646138' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T11:33:32.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:31 smithi188 ceph-mon[36925]: from='client.? 172.21.15.158:0/1503646138' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T11:33:32.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:31 smithi012 ceph-mon[32460]: from='client.? 172.21.15.158:0/1503646138' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T11:33:33.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:32 smithi188 ceph-mon[36925]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-18T11:33:33.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:32 smithi012 ceph-mon[32460]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-18T11:33:33.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:32 smithi158 ceph-mon[36626]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-18T11:33:34.953 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:34 smithi158 ceph-mon[36626]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-18T11:33:35.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:34 smithi188 ceph-mon[36925]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-18T11:33:35.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:34 smithi012 ceph-mon[32460]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-18T11:33:37.055 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:36 smithi158 ceph-mon[36626]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-18T11:33:37.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:36 smithi188 ceph-mon[36925]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-18T11:33:37.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:36 smithi012 ceph-mon[32460]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-18T11:33:38.965 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:38 smithi158 ceph-mon[36626]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-18T11:33:38.965 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:38 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-18T11:33:38.965 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:38 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:33:38.965 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:38 smithi158 ceph-mon[36626]: Deploying daemon osd.1 on smithi158 2024-02-18T11:33:39.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:38 smithi188 ceph-mon[36925]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-18T11:33:39.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:38 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-18T11:33:39.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:38 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:33:39.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:38 smithi188 ceph-mon[36925]: Deploying daemon osd.1 on smithi158 2024-02-18T11:33:39.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:38 smithi012 ceph-mon[32460]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-18T11:33:39.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:38 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-18T11:33:39.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:38 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:33:39.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:38 smithi012 ceph-mon[32460]: Deploying daemon osd.1 on smithi158 2024-02-18T11:33:41.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:40 smithi188 ceph-mon[36925]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-18T11:33:41.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:40 smithi012 ceph-mon[32460]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-18T11:33:41.209 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:40 smithi158 ceph-mon[36626]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-18T11:33:42.893 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:42 smithi158 ceph-mon[36626]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-18T11:33:43.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:42 smithi188 ceph-mon[36925]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-18T11:33:43.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:42 smithi012 ceph-mon[32460]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-18T11:33:44.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:44 smithi158 ceph-mon[36626]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-18T11:33:44.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:44 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:44.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:44 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:44.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:44 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:33:44.355 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:44 smithi188 ceph-mon[36925]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-18T11:33:44.355 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:44 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:44.355 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:44 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:44.355 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:44 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:33:44.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:44 smithi012 ceph-mon[32460]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-18T11:33:44.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:44 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:44.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:44 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:44.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:44 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:33:46.206 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:33:46.210+0000 7f61a57fa700 1 -- 172.21.15.158:0/4171091661 <== mgr.14152 v2:172.21.15.12:6800/1153736685 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f61a802d8b0 con 0x7f61900a2750 2024-02-18T11:33:46.207 INFO:teuthology.orchestra.run.smithi158.stdout:Created osd(s) 1 on host 'smithi158' 2024-02-18T11:33:46.209 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:33:46.213+0000 7f61ae709700 1 -- 172.21.15.158:0/4171091661 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f61900a2750 msgr2=0x7f61900a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:33:46.209 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:33:46.213+0000 7f61ae709700 1 --2- 172.21.15.158:0/4171091661 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f61900a2750 0x7f61900a4c10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f61a8101c50 tx=0x7f619800a3b0 comp rx=0 tx=0).stop 2024-02-18T11:33:46.209 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:33:46.213+0000 7f61ae709700 1 -- 172.21.15.158:0/4171091661 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f61a8102790 msgr2=0x7f61a810a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:33:46.209 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:33:46.213+0000 7f61ae709700 1 --2- 172.21.15.158:0/4171091661 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f61a8102790 0x7f61a810a090 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f61a0001220 tx=0x7f61a0007930 comp rx=0 tx=0).stop 2024-02-18T11:33:46.209 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:33:46.213+0000 7f61ae709700 1 -- 172.21.15.158:0/4171091661 shutdown_connections 2024-02-18T11:33:46.209 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:33:46.213+0000 7f61ae709700 1 --2- 172.21.15.158:0/4171091661 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f61a8102790 0x7f61a810a090 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:33:46.209 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:33:46.213+0000 7f61ae709700 1 --2- 172.21.15.158:0/4171091661 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f61900a2750 0x7f61900a4c10 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:33:46.209 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:33:46.213+0000 7f61ae709700 1 --2- 172.21.15.158:0/4171091661 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61a8100bf0 0x7f61a810b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:33:46.209 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:33:46.213+0000 7f61ae709700 1 --2- 172.21.15.158:0/4171091661 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f61a8101df0 0x7f61a8109b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:33:46.209 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:33:46.213+0000 7f61ae709700 1 -- 172.21.15.158:0/4171091661 >> 172.21.15.158:0/4171091661 conn(0x7f61a80fc1b0 msgr2=0x7f61a8105020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:33:46.209 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:33:46.213+0000 7f61ae709700 1 -- 172.21.15.158:0/4171091661 shutdown_connections 2024-02-18T11:33:46.210 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:33:46.213+0000 7f61ae709700 1 -- 172.21.15.158:0/4171091661 wait complete. 2024-02-18T11:33:46.903 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:46 smithi158 ceph-mon[36626]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-18T11:33:46.904 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:46 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:46.904 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:46 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:46.904 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:46 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:33:46.904 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:46 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:33:46.904 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:46 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:46.904 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:46 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:46.904 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:46 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:46.904 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:46 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:33:46.944 DEBUG:teuthology.orchestra.run.smithi158:osd.1> sudo journalctl -f -n 0 -u ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@osd.1.service 2024-02-18T11:33:46.947 INFO:tasks.cephadm:Deploying osd.2 on smithi188 with /dev/vg_nvme/lv_4... 2024-02-18T11:33:46.947 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-02-18T11:33:46.977 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:46 smithi188 ceph-mon[36925]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-18T11:33:46.977 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:46 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:46.978 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:46 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:46.978 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:46 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:33:46.978 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:46 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:33:46.978 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:46 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:46.978 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:46 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:46.978 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:46 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:46.978 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:46 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:33:47.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:46 smithi012 ceph-mon[32460]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-18T11:33:47.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:46 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:47.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:46 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:47.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:46 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:33:47.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:46 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:33:47.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:46 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:47.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:46 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:47.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:46 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:47.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:46 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:33:47.189 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.c/config 2024-02-18T11:33:47.745 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:47 smithi158 ceph-mon[36626]: from='osd.1 [v2:172.21.15.158:6800/4249111488,v1:172.21.15.158:6801/4249111488]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-18T11:33:47.745 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:47 smithi158 ceph-mon[36626]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-18T11:33:48.108 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:47 smithi188 ceph-mon[36925]: from='osd.1 [v2:172.21.15.158:6800/4249111488,v1:172.21.15.158:6801/4249111488]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-18T11:33:48.109 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:47 smithi188 ceph-mon[36925]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-18T11:33:48.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:47 smithi012 ceph-mon[32460]: from='osd.1 [v2:172.21.15.158:6800/4249111488,v1:172.21.15.158:6801/4249111488]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-18T11:33:48.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:47 smithi012 ceph-mon[32460]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-18T11:33:48.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:48 smithi188 ceph-mon[36925]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-18T11:33:48.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:48 smithi188 ceph-mon[36925]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-18T11:33:48.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:48 smithi188 ceph-mon[36925]: from='osd.1 [v2:172.21.15.158:6800/4249111488,v1:172.21.15.158:6801/4249111488]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi158", "root=default"]}]: dispatch 2024-02-18T11:33:48.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:48 smithi188 ceph-mon[36925]: osdmap e11: 2 total, 1 up, 2 in 2024-02-18T11:33:48.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:48 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-18T11:33:48.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:48 smithi188 ceph-mon[36925]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi158", "root=default"]}]: dispatch 2024-02-18T11:33:48.996 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:48 smithi158 ceph-mon[36626]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-18T11:33:48.996 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:48 smithi158 ceph-mon[36626]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-18T11:33:48.996 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:48 smithi158 ceph-mon[36626]: from='osd.1 [v2:172.21.15.158:6800/4249111488,v1:172.21.15.158:6801/4249111488]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi158", "root=default"]}]: dispatch 2024-02-18T11:33:48.996 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:48 smithi158 ceph-mon[36626]: osdmap e11: 2 total, 1 up, 2 in 2024-02-18T11:33:48.996 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:48 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-18T11:33:48.996 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:48 smithi158 ceph-mon[36626]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi158", "root=default"]}]: dispatch 2024-02-18T11:33:48.996 INFO:journalctl@ceph.osd.1.smithi158.stdout:Feb 18 11:33:48 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-1[41251]: 2024-02-18T11:33:48.693+0000 7f0a60159700 -1 osd.1 0 waiting for initial osdmap 2024-02-18T11:33:48.996 INFO:journalctl@ceph.osd.1.smithi158.stdout:Feb 18 11:33:48 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-1[41251]: 2024-02-18T11:33:48.697+0000 7f0a5c74d700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-18T11:33:49.052 INFO:teuthology.orchestra.run.smithi188.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-02-18T11:33:49.052 INFO:teuthology.orchestra.run.smithi188.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-02-18T11:33:49.052 INFO:teuthology.orchestra.run.smithi188.stderr: stderr: 10+0 records in 2024-02-18T11:33:49.052 INFO:teuthology.orchestra.run.smithi188.stderr:10+0 records out 2024-02-18T11:33:49.052 INFO:teuthology.orchestra.run.smithi188.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0110815 s, 946 MB/s 2024-02-18T11:33:49.052 INFO:teuthology.orchestra.run.smithi188.stderr:--> Zapping successful for: 2024-02-18T11:33:49.075 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:48 smithi012 ceph-mon[32460]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-18T11:33:49.075 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:48 smithi012 ceph-mon[32460]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-18T11:33:49.075 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:48 smithi012 ceph-mon[32460]: from='osd.1 [v2:172.21.15.158:6800/4249111488,v1:172.21.15.158:6801/4249111488]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi158", "root=default"]}]: dispatch 2024-02-18T11:33:49.075 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:48 smithi012 ceph-mon[32460]: osdmap e11: 2 total, 1 up, 2 in 2024-02-18T11:33:49.075 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:48 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-18T11:33:49.075 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:48 smithi012 ceph-mon[32460]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi158", "root=default"]}]: dispatch 2024-02-18T11:33:49.674 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi188:vg_nvme/lv_4 2024-02-18T11:33:49.958 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.c/config 2024-02-18T11:33:49.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:49 smithi158 ceph-mon[36626]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi158", "root=default"]}]': finished 2024-02-18T11:33:49.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:49 smithi158 ceph-mon[36626]: osdmap e12: 2 total, 1 up, 2 in 2024-02-18T11:33:49.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:49 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-18T11:33:49.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:49 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-18T11:33:50.080 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:49 smithi188 ceph-mon[36925]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi158", "root=default"]}]': finished 2024-02-18T11:33:50.080 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:49 smithi188 ceph-mon[36925]: osdmap e12: 2 total, 1 up, 2 in 2024-02-18T11:33:50.080 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:49 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-18T11:33:50.080 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:49 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-18T11:33:50.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:49 smithi012 ceph-mon[32460]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi158", "root=default"]}]': finished 2024-02-18T11:33:50.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:49 smithi012 ceph-mon[32460]: osdmap e12: 2 total, 1 up, 2 in 2024-02-18T11:33:50.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:49 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-18T11:33:50.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:49 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-18T11:33:50.970 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:50 smithi188 ceph-mon[36925]: purged_snaps scrub starts 2024-02-18T11:33:50.970 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:50 smithi188 ceph-mon[36925]: purged_snaps scrub ok 2024-02-18T11:33:50.970 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:50 smithi188 ceph-mon[36925]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-18T11:33:50.970 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:50 smithi188 ceph-mon[36925]: osd.1 [v2:172.21.15.158:6800/4249111488,v1:172.21.15.158:6801/4249111488] boot 2024-02-18T11:33:50.970 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:50 smithi188 ceph-mon[36925]: osdmap e13: 2 total, 2 up, 2 in 2024-02-18T11:33:50.970 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:50 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-18T11:33:50.970 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:50 smithi188 ceph-mon[36925]: Detected new or changed devices on smithi158 2024-02-18T11:33:50.970 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:50 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:50.970 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:50 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:50.970 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:50 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:33:50.970 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:50 smithi188 ceph-mon[36925]: Adjusting osd_memory_target on smithi158 to 17080M 2024-02-18T11:33:50.970 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:50 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:50.970 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:50 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:33:50.971 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:50 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:33:50.971 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:50 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:50.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:50 smithi158 ceph-mon[36626]: purged_snaps scrub starts 2024-02-18T11:33:50.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:50 smithi158 ceph-mon[36626]: purged_snaps scrub ok 2024-02-18T11:33:50.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:50 smithi158 ceph-mon[36626]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-18T11:33:50.996 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:50 smithi158 ceph-mon[36626]: osd.1 [v2:172.21.15.158:6800/4249111488,v1:172.21.15.158:6801/4249111488] boot 2024-02-18T11:33:50.996 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:50 smithi158 ceph-mon[36626]: osdmap e13: 2 total, 2 up, 2 in 2024-02-18T11:33:50.996 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:50 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-18T11:33:50.996 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:50 smithi158 ceph-mon[36626]: Detected new or changed devices on smithi158 2024-02-18T11:33:50.996 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:50 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:50.996 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:50 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:50.996 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:50 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:33:50.996 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:50 smithi158 ceph-mon[36626]: Adjusting osd_memory_target on smithi158 to 17080M 2024-02-18T11:33:50.996 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:50 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:50.996 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:50 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:33:50.996 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:50 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:33:50.996 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:50 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:51.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:50 smithi012 ceph-mon[32460]: purged_snaps scrub starts 2024-02-18T11:33:51.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:50 smithi012 ceph-mon[32460]: purged_snaps scrub ok 2024-02-18T11:33:51.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:50 smithi012 ceph-mon[32460]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-18T11:33:51.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:50 smithi012 ceph-mon[32460]: osd.1 [v2:172.21.15.158:6800/4249111488,v1:172.21.15.158:6801/4249111488] boot 2024-02-18T11:33:51.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:50 smithi012 ceph-mon[32460]: osdmap e13: 2 total, 2 up, 2 in 2024-02-18T11:33:51.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:50 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-18T11:33:51.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:50 smithi012 ceph-mon[32460]: Detected new or changed devices on smithi158 2024-02-18T11:33:51.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:50 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:51.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:50 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:51.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:50 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:33:51.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:50 smithi012 ceph-mon[32460]: Adjusting osd_memory_target on smithi158 to 17080M 2024-02-18T11:33:51.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:50 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:51.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:50 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:33:51.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:50 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:33:51.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:50 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:33:51.511 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:33:51.514+0000 7f638643e700 1 -- 172.21.15.188:0/354473312 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6380101e80 msgr2=0x7f63801022e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:33:51.511 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:33:51.514+0000 7f638643e700 1 --2- 172.21.15.188:0/354473312 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6380101e80 0x7f63801022e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6368009990 tx=0x7f6368009ca0 comp rx=0 tx=0).stop 2024-02-18T11:33:51.511 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:33:51.514+0000 7f638643e700 1 -- 172.21.15.188:0/354473312 shutdown_connections 2024-02-18T11:33:51.511 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:33:51.514+0000 7f638643e700 1 --2- 172.21.15.188:0/354473312 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6380102820 0x7f63801090d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:33:51.511 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:33:51.514+0000 7f638643e700 1 --2- 172.21.15.188:0/354473312 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6380100c80 0x7f63801010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:33:51.511 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:33:51.514+0000 7f638643e700 1 --2- 172.21.15.188:0/354473312 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6380101e80 0x7f63801022e0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:33:51.511 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:33:51.514+0000 7f638643e700 1 -- 172.21.15.188:0/354473312 >> 172.21.15.188:0/354473312 conn(0x7f63800fc240 msgr2=0x7f63800fe660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:33:51.511 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:33:51.514+0000 7f638643e700 1 -- 172.21.15.188:0/354473312 shutdown_connections 2024-02-18T11:33:51.511 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:33:51.514+0000 7f638643e700 1 -- 172.21.15.188:0/354473312 wait complete. 2024-02-18T11:33:51.511 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:33:51.514+0000 7f638643e700 1 Processor -- start 2024-02-18T11:33:51.511 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:33:51.514+0000 7f638643e700 1 -- start start 2024-02-18T11:33:51.512 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:33:51.515+0000 7f638643e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6380100c80 0x7f6380104ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:33:51.512 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:33:51.515+0000 7f638643e700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6380101e80 0x7f6380103260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:33:51.512 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:33:51.515+0000 7f638643e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6380102820 0x7f63801037a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:33:51.512 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:33:51.515+0000 7f638643e700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6380103eb0 con 0x7f6380102820 2024-02-18T11:33:51.512 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:33:51.515+0000 7f638643e700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6380104020 con 0x7f6380101e80 2024-02-18T11:33:51.512 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:33:51.515+0000 7f638643e700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63801a1d80 con 0x7f6380100c80 2024-02-18T11:33:51.512 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:33:51.515+0000 7f637ffff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6380100c80 0x7f6380104ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:33:51.512 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:33:51.515+0000 7f637ffff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6380100c80 0x7f6380104ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:57774/0 (socket says 172.21.15.188:57774) 2024-02-18T11:33:51.512 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:33:51.515+0000 7f637ffff700 1 -- 172.21.15.188:0/2792655071 learned_addr learned my addr 172.21.15.188:0/2792655071 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-02-18T11:33:51.512 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:33:51.515+0000 7f637f7fe700 1 --2- 172.21.15.188:0/2792655071 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6380101e80 0x7f6380103260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:33:51.512 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:33:51.515+0000 7f63849db700 1 --2- 172.21.15.188:0/2792655071 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6380102820 0x7f63801037a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:33:51.512 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:33:51.515+0000 7f637ffff700 1 -- 172.21.15.188:0/2792655071 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6380101e80 msgr2=0x7f6380103260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:33:51.512 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:33:51.515+0000 7f637ffff700 1 --2- 172.21.15.188:0/2792655071 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6380101e80 0x7f6380103260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:33:51.512 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:33:51.515+0000 7f637ffff700 1 -- 172.21.15.188:0/2792655071 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6380102820 msgr2=0x7f63801037a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:33:51.513 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:33:51.515+0000 7f637ffff700 1 --2- 172.21.15.188:0/2792655071 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6380102820 0x7f63801037a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:33:51.513 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:33:51.515+0000 7f637ffff700 1 -- 172.21.15.188:0/2792655071 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63680095f0 con 0x7f6380100c80 2024-02-18T11:33:51.513 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:33:51.515+0000 7f637ffff700 1 --2- 172.21.15.188:0/2792655071 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6380100c80 0x7f6380104ba0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f637000d980 tx=0x7f637000dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:33:51.513 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:33:51.515+0000 7f637d7fa700 1 -- 172.21.15.188:0/2792655071 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f637000b950 con 0x7f6380100c80 2024-02-18T11:33:51.513 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:33:51.516+0000 7f637d7fa700 1 -- 172.21.15.188:0/2792655071 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6370007620 con 0x7f6380100c80 2024-02-18T11:33:51.513 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:33:51.516+0000 7f638643e700 1 -- 172.21.15.188:0/2792655071 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63801a1fe0 con 0x7f6380100c80 2024-02-18T11:33:51.513 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:33:51.516+0000 7f637d7fa700 1 -- 172.21.15.188:0/2792655071 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f637000f620 con 0x7f6380100c80 2024-02-18T11:33:51.513 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:33:51.516+0000 7f638643e700 1 -- 172.21.15.188:0/2792655071 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63801a23f0 con 0x7f6380100c80 2024-02-18T11:33:51.514 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:33:51.517+0000 7f638643e700 1 -- 172.21.15.188:0/2792655071 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f638010c550 con 0x7f6380100c80 2024-02-18T11:33:51.514 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:33:51.518+0000 7f637d7fa700 1 -- 172.21.15.188:0/2792655071 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 15) v1 ==== 137189+0+0 (secure 0 0 0) 0x7f63700106f0 con 0x7f6380100c80 2024-02-18T11:33:51.514 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:33:51.518+0000 7f637d7fa700 1 --2- 172.21.15.188:0/2792655071 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f636c0a2750 0x7f636c0a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:33:51.514 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:33:51.518+0000 7f637d7fa700 1 -- 172.21.15.188:0/2792655071 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2309+0+0 (secure 0 0 0) 0x7f63700cd3e0 con 0x7f6380100c80 2024-02-18T11:33:51.514 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:33:51.518+0000 7f637f7fe700 1 --2- 172.21.15.188:0/2792655071 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f636c0a2750 0x7f636c0a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:33:51.514 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:33:51.519+0000 7f637f7fe700 1 --2- 172.21.15.188:0/2792655071 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f636c0a2750 0x7f636c0a4c10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f636800b5c0 tx=0x7f63680038d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:33:51.516 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:33:51.522+0000 7f637d7fa700 1 -- 172.21.15.188:0/2792655071 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f63700979c0 con 0x7f6380100c80 2024-02-18T11:33:51.717 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:33:51.723+0000 7f638643e700 1 -- 172.21.15.188:0/2792655071 --> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f63800fe400 con 0x7f636c0a2750 2024-02-18T11:33:52.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:51 smithi188 ceph-mon[36925]: osdmap e14: 2 total, 2 up, 2 in 2024-02-18T11:33:52.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:51 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T11:33:52.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:51 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T11:33:52.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:51 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:33:52.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:51 smithi012 ceph-mon[32460]: osdmap e14: 2 total, 2 up, 2 in 2024-02-18T11:33:52.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:51 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T11:33:52.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:51 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T11:33:52.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:51 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:33:52.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:51 smithi158 ceph-mon[36626]: osdmap e14: 2 total, 2 up, 2 in 2024-02-18T11:33:52.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:51 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T11:33:52.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:51 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T11:33:52.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:51 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:33:52.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:52 smithi188 ceph-mon[36925]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-18T11:33:52.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:52 smithi188 ceph-mon[36925]: from='client.24154 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:33:53.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:52 smithi012 ceph-mon[32460]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-18T11:33:53.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:52 smithi012 ceph-mon[32460]: from='client.24154 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:33:53.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:52 smithi158 ceph-mon[36626]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-18T11:33:53.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:52 smithi158 ceph-mon[36626]: from='client.24154 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:33:55.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:54 smithi188 ceph-mon[36925]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-18T11:33:55.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:54 smithi188 ceph-mon[36925]: from='client.? 172.21.15.188:0/1099021226' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17b804dd-31df-45ae-a637-4a5f65bbaf26"}]: dispatch 2024-02-18T11:33:55.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:54 smithi188 ceph-mon[36925]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17b804dd-31df-45ae-a637-4a5f65bbaf26"}]: dispatch 2024-02-18T11:33:55.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:54 smithi188 ceph-mon[36925]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "17b804dd-31df-45ae-a637-4a5f65bbaf26"}]': finished 2024-02-18T11:33:55.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:54 smithi188 ceph-mon[36925]: osdmap e15: 3 total, 2 up, 3 in 2024-02-18T11:33:55.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:54 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-18T11:33:55.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:54 smithi188 ceph-mon[36925]: from='client.? 172.21.15.188:0/4174391987' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T11:33:55.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:54 smithi012 ceph-mon[32460]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-18T11:33:55.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:54 smithi012 ceph-mon[32460]: from='client.? 172.21.15.188:0/1099021226' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17b804dd-31df-45ae-a637-4a5f65bbaf26"}]: dispatch 2024-02-18T11:33:55.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:54 smithi012 ceph-mon[32460]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17b804dd-31df-45ae-a637-4a5f65bbaf26"}]: dispatch 2024-02-18T11:33:55.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:54 smithi012 ceph-mon[32460]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "17b804dd-31df-45ae-a637-4a5f65bbaf26"}]': finished 2024-02-18T11:33:55.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:54 smithi012 ceph-mon[32460]: osdmap e15: 3 total, 2 up, 3 in 2024-02-18T11:33:55.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:54 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-18T11:33:55.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:54 smithi012 ceph-mon[32460]: from='client.? 172.21.15.188:0/4174391987' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T11:33:55.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:54 smithi158 ceph-mon[36626]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-18T11:33:55.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:54 smithi158 ceph-mon[36626]: from='client.? 172.21.15.188:0/1099021226' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17b804dd-31df-45ae-a637-4a5f65bbaf26"}]: dispatch 2024-02-18T11:33:55.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:54 smithi158 ceph-mon[36626]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17b804dd-31df-45ae-a637-4a5f65bbaf26"}]: dispatch 2024-02-18T11:33:55.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:54 smithi158 ceph-mon[36626]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "17b804dd-31df-45ae-a637-4a5f65bbaf26"}]': finished 2024-02-18T11:33:55.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:54 smithi158 ceph-mon[36626]: osdmap e15: 3 total, 2 up, 3 in 2024-02-18T11:33:55.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:54 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-18T11:33:55.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:54 smithi158 ceph-mon[36626]: from='client.? 172.21.15.188:0/4174391987' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T11:33:57.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:56 smithi188 ceph-mon[36925]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-18T11:33:57.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:56 smithi012 ceph-mon[32460]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-18T11:33:57.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:56 smithi158 ceph-mon[36626]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-18T11:33:59.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:33:58 smithi188 ceph-mon[36925]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-18T11:33:59.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:33:58 smithi012 ceph-mon[32460]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-18T11:33:59.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:33:58 smithi158 ceph-mon[36626]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-18T11:34:00.831 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:00 smithi188 ceph-mon[36925]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-18T11:34:01.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:00 smithi012 ceph-mon[32460]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-18T11:34:01.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:00 smithi158 ceph-mon[36626]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-18T11:34:02.042 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:01 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-18T11:34:02.043 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:01 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:34:02.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:01 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-18T11:34:02.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:01 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:34:02.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:01 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-18T11:34:02.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:01 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:34:03.004 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:02 smithi188 ceph-mon[36925]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-18T11:34:03.004 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:02 smithi188 ceph-mon[36925]: Deploying daemon osd.2 on smithi188 2024-02-18T11:34:03.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:02 smithi012 ceph-mon[32460]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-18T11:34:03.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:02 smithi012 ceph-mon[32460]: Deploying daemon osd.2 on smithi188 2024-02-18T11:34:03.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:02 smithi158 ceph-mon[36626]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-18T11:34:03.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:02 smithi158 ceph-mon[36626]: Deploying daemon osd.2 on smithi188 2024-02-18T11:34:05.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:04 smithi188 ceph-mon[36925]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-18T11:34:05.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:04 smithi012 ceph-mon[32460]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-18T11:34:05.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:04 smithi158 ceph-mon[36626]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-18T11:34:07.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:06 smithi188 ceph-mon[36925]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-18T11:34:07.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:06 smithi012 ceph-mon[32460]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-18T11:34:07.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:06 smithi158 ceph-mon[36626]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-18T11:34:08.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:07 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:08.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:07 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:08.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:07 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:34:08.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:07 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:08.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:07 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:08.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:07 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:34:08.355 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:07 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:08.355 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:07 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:08.355 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:07 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:34:08.998 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:08 smithi188 ceph-mon[36925]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-18T11:34:09.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:08 smithi012 ceph-mon[32460]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-18T11:34:09.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:08 smithi158 ceph-mon[36626]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-18T11:34:10.024 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:09 smithi188 ceph-mon[36925]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-18T11:34:10.086 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:34:10.091+0000 7f637d7fa700 1 -- 172.21.15.188:0/2792655071 <== mgr.14152 v2:172.21.15.12:6800/1153736685 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f63800fe400 con 0x7f636c0a2750 2024-02-18T11:34:10.087 INFO:teuthology.orchestra.run.smithi188.stdout:Created osd(s) 2 on host 'smithi188' 2024-02-18T11:34:10.089 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:34:10.093+0000 7f638643e700 1 -- 172.21.15.188:0/2792655071 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f636c0a2750 msgr2=0x7f636c0a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:34:10.089 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:34:10.093+0000 7f638643e700 1 --2- 172.21.15.188:0/2792655071 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f636c0a2750 0x7f636c0a4c10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f636800b5c0 tx=0x7f63680038d0 comp rx=0 tx=0).stop 2024-02-18T11:34:10.089 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:34:10.093+0000 7f638643e700 1 -- 172.21.15.188:0/2792655071 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6380100c80 msgr2=0x7f6380104ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:34:10.089 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:34:10.093+0000 7f638643e700 1 --2- 172.21.15.188:0/2792655071 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6380100c80 0x7f6380104ba0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f637000d980 tx=0x7f637000dc90 comp rx=0 tx=0).stop 2024-02-18T11:34:10.089 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:34:10.093+0000 7f638643e700 1 -- 172.21.15.188:0/2792655071 shutdown_connections 2024-02-18T11:34:10.089 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:34:10.093+0000 7f638643e700 1 --2- 172.21.15.188:0/2792655071 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f636c0a2750 0x7f636c0a4c10 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:34:10.089 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:34:10.093+0000 7f638643e700 1 --2- 172.21.15.188:0/2792655071 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6380102820 0x7f63801037a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:34:10.089 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:34:10.093+0000 7f638643e700 1 --2- 172.21.15.188:0/2792655071 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6380100c80 0x7f6380104ba0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:34:10.089 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:34:10.093+0000 7f638643e700 1 --2- 172.21.15.188:0/2792655071 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6380101e80 0x7f6380103260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:34:10.089 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:34:10.093+0000 7f638643e700 1 -- 172.21.15.188:0/2792655071 >> 172.21.15.188:0/2792655071 conn(0x7f63800fc240 msgr2=0x7f63800fdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:34:10.089 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:34:10.093+0000 7f638643e700 1 -- 172.21.15.188:0/2792655071 shutdown_connections 2024-02-18T11:34:10.090 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:34:10.093+0000 7f638643e700 1 -- 172.21.15.188:0/2792655071 wait complete. 2024-02-18T11:34:10.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:09 smithi158 ceph-mon[36626]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-18T11:34:10.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:09 smithi012 ceph-mon[32460]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-18T11:34:10.769 DEBUG:teuthology.orchestra.run.smithi188:osd.2> sudo journalctl -f -n 0 -u ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@osd.2.service 2024-02-18T11:34:10.772 INFO:tasks.cephadm:Deploying osd.3 on smithi012 with /dev/vg_nvme/lv_3... 2024-02-18T11:34:10.773 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-02-18T11:34:11.024 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T11:34:11.347 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:11 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:11.347 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:11 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:11.348 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:11 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:11.348 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:11 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:11.348 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:11 smithi012 ceph-mon[32460]: from='osd.2 [v2:172.21.15.188:6800/627976150,v1:172.21.15.188:6801/627976150]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-18T11:34:11.349 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:11 smithi012 ceph-mon[32460]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-18T11:34:11.355 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:11 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:11.355 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:11 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:11.355 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:11 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:11.355 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:11 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:11.355 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:11 smithi188 ceph-mon[36925]: from='osd.2 [v2:172.21.15.188:6800/627976150,v1:172.21.15.188:6801/627976150]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-18T11:34:11.356 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:11 smithi188 ceph-mon[36925]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-18T11:34:11.496 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:11 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:11.496 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:11 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:11.496 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:11 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:11.496 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:11 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:11.496 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:11 smithi158 ceph-mon[36626]: from='osd.2 [v2:172.21.15.188:6800/627976150,v1:172.21.15.188:6801/627976150]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-18T11:34:11.496 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:11 smithi158 ceph-mon[36626]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-18T11:34:12.319 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:12 smithi012 ceph-mon[32460]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-18T11:34:12.319 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:12 smithi012 ceph-mon[32460]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-18T11:34:12.319 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:12 smithi012 ceph-mon[32460]: from='osd.2 [v2:172.21.15.188:6800/627976150,v1:172.21.15.188:6801/627976150]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-02-18T11:34:12.319 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:12 smithi012 ceph-mon[32460]: osdmap e16: 3 total, 2 up, 3 in 2024-02-18T11:34:12.319 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:12 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-18T11:34:12.319 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:12 smithi012 ceph-mon[32460]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-02-18T11:34:12.336 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:12 smithi188 ceph-mon[36925]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-18T11:34:12.336 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:12 smithi188 ceph-mon[36925]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-18T11:34:12.336 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:12 smithi188 ceph-mon[36925]: from='osd.2 [v2:172.21.15.188:6800/627976150,v1:172.21.15.188:6801/627976150]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-02-18T11:34:12.336 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:12 smithi188 ceph-mon[36925]: osdmap e16: 3 total, 2 up, 3 in 2024-02-18T11:34:12.336 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:12 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-18T11:34:12.336 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:12 smithi188 ceph-mon[36925]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-02-18T11:34:12.337 INFO:journalctl@ceph.osd.2.smithi188.stdout:Feb 18 11:34:12 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-2[42103]: 2024-02-18T11:34:12.163+0000 7fc426d34700 -1 osd.2 0 waiting for initial osdmap 2024-02-18T11:34:12.337 INFO:journalctl@ceph.osd.2.smithi188.stdout:Feb 18 11:34:12 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-2[42103]: 2024-02-18T11:34:12.167+0000 7fc423328700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-18T11:34:12.495 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:12 smithi158 ceph-mon[36626]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-18T11:34:12.495 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:12 smithi158 ceph-mon[36626]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-18T11:34:12.495 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:12 smithi158 ceph-mon[36626]: from='osd.2 [v2:172.21.15.188:6800/627976150,v1:172.21.15.188:6801/627976150]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-02-18T11:34:12.495 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:12 smithi158 ceph-mon[36626]: osdmap e16: 3 total, 2 up, 3 in 2024-02-18T11:34:12.495 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:12 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-18T11:34:12.496 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:12 smithi158 ceph-mon[36626]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-02-18T11:34:12.871 INFO:teuthology.orchestra.run.smithi012.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-02-18T11:34:12.871 INFO:teuthology.orchestra.run.smithi012.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-02-18T11:34:12.872 INFO:teuthology.orchestra.run.smithi012.stderr: stderr: 10+0 records in 2024-02-18T11:34:12.872 INFO:teuthology.orchestra.run.smithi012.stderr:10+0 records out 2024-02-18T11:34:12.872 INFO:teuthology.orchestra.run.smithi012.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113812 s, 921 MB/s 2024-02-18T11:34:12.872 INFO:teuthology.orchestra.run.smithi012.stderr:--> Zapping successful for: 2024-02-18T11:34:13.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:13 smithi012 ceph-mon[32460]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-02-18T11:34:13.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:13 smithi012 ceph-mon[32460]: osdmap e17: 3 total, 2 up, 3 in 2024-02-18T11:34:13.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:13 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-18T11:34:13.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:13 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-18T11:34:13.461 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:13 smithi188 ceph-mon[36925]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-02-18T11:34:13.461 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:13 smithi188 ceph-mon[36925]: osdmap e17: 3 total, 2 up, 3 in 2024-02-18T11:34:13.461 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:13 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-18T11:34:13.461 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:13 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-18T11:34:13.477 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi012:vg_nvme/lv_3 2024-02-18T11:34:13.495 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:13 smithi158 ceph-mon[36626]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-02-18T11:34:13.495 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:13 smithi158 ceph-mon[36626]: osdmap e17: 3 total, 2 up, 3 in 2024-02-18T11:34:13.495 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:13 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-18T11:34:13.495 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:13 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-18T11:34:13.735 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T11:34:14.396 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:14 smithi012 ceph-mon[32460]: purged_snaps scrub starts 2024-02-18T11:34:14.397 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:14 smithi012 ceph-mon[32460]: purged_snaps scrub ok 2024-02-18T11:34:14.397 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:14 smithi012 ceph-mon[32460]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-18T11:34:14.397 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:14 smithi012 ceph-mon[32460]: osd.2 [v2:172.21.15.188:6800/627976150,v1:172.21.15.188:6801/627976150] boot 2024-02-18T11:34:14.397 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:14 smithi012 ceph-mon[32460]: osdmap e18: 3 total, 3 up, 3 in 2024-02-18T11:34:14.397 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:14 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-18T11:34:14.397 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:14 smithi012 ceph-mon[32460]: Detected new or changed devices on smithi188 2024-02-18T11:34:14.397 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:14 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:14.397 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:14 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:14.397 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:14 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:34:14.397 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:14 smithi012 ceph-mon[32460]: Adjusting osd_memory_target on smithi188 to 17080M 2024-02-18T11:34:14.397 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:14 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:14.397 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:14 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:34:14.397 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:14 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:34:14.398 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:14 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:14.398 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:14 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:34:14.398 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:14 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:34:14.398 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:14 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:34:14.398 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:14 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:14.495 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:14 smithi158 ceph-mon[36626]: purged_snaps scrub starts 2024-02-18T11:34:14.495 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:14 smithi158 ceph-mon[36626]: purged_snaps scrub ok 2024-02-18T11:34:14.495 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:14 smithi158 ceph-mon[36626]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-18T11:34:14.495 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:14 smithi158 ceph-mon[36626]: osd.2 [v2:172.21.15.188:6800/627976150,v1:172.21.15.188:6801/627976150] boot 2024-02-18T11:34:14.496 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:14 smithi158 ceph-mon[36626]: osdmap e18: 3 total, 3 up, 3 in 2024-02-18T11:34:14.496 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:14 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-18T11:34:14.496 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:14 smithi158 ceph-mon[36626]: Detected new or changed devices on smithi188 2024-02-18T11:34:14.496 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:14 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:14.496 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:14 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:14.496 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:14 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:34:14.496 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:14 smithi158 ceph-mon[36626]: Adjusting osd_memory_target on smithi188 to 17080M 2024-02-18T11:34:14.496 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:14 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:14.496 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:14 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:34:14.496 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:14 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:34:14.496 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:14 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:14.496 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:14 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:34:14.496 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:14 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:34:14.496 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:14 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:34:14.496 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:14 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:14.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:14 smithi188 ceph-mon[36925]: purged_snaps scrub starts 2024-02-18T11:34:14.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:14 smithi188 ceph-mon[36925]: purged_snaps scrub ok 2024-02-18T11:34:14.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:14 smithi188 ceph-mon[36925]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-18T11:34:14.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:14 smithi188 ceph-mon[36925]: osd.2 [v2:172.21.15.188:6800/627976150,v1:172.21.15.188:6801/627976150] boot 2024-02-18T11:34:14.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:14 smithi188 ceph-mon[36925]: osdmap e18: 3 total, 3 up, 3 in 2024-02-18T11:34:14.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:14 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-18T11:34:14.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:14 smithi188 ceph-mon[36925]: Detected new or changed devices on smithi188 2024-02-18T11:34:14.606 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:14 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:14.606 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:14 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:14.606 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:14 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:34:14.606 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:14 smithi188 ceph-mon[36925]: Adjusting osd_memory_target on smithi188 to 17080M 2024-02-18T11:34:14.606 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:14 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:14.606 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:14 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:34:14.606 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:14 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:34:14.606 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:14 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:14.606 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:14 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:34:14.606 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:14 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:34:14.606 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:14 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:34:14.606 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:14 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:15.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:34:15.218+0000 7f9003915700 1 -- 172.21.15.12:0/2835443413 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8ffc1079c0 msgr2=0x7f8ffc109e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:34:15.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:34:15.218+0000 7f9003915700 1 --2- 172.21.15.12:0/2835443413 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8ffc1079c0 0x7f8ffc109e30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8ff4009f90 tx=0x7f8ff400b330 comp rx=0 tx=0).stop 2024-02-18T11:34:15.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:34:15.218+0000 7f9003915700 1 -- 172.21.15.12:0/2835443413 shutdown_connections 2024-02-18T11:34:15.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:34:15.218+0000 7f9003915700 1 --2- 172.21.15.12:0/2835443413 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8ffc1079c0 0x7f8ffc109e30 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:34:15.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:34:15.218+0000 7f9003915700 1 --2- 172.21.15.12:0/2835443413 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ffc102760 0x7f8ffc104b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:34:15.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:34:15.218+0000 7f9003915700 1 --2- 172.21.15.12:0/2835443413 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ffc105090 0x7f8ffc107480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:34:15.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:34:15.218+0000 7f9003915700 1 -- 172.21.15.12:0/2835443413 >> 172.21.15.12:0/2835443413 conn(0x7f8ffc0fc170 msgr2=0x7f8ffc0fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:34:15.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:34:15.218+0000 7f9003915700 1 -- 172.21.15.12:0/2835443413 shutdown_connections 2024-02-18T11:34:15.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:34:15.218+0000 7f9003915700 1 -- 172.21.15.12:0/2835443413 wait complete. 2024-02-18T11:34:15.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:34:15.219+0000 7f9003915700 1 Processor -- start 2024-02-18T11:34:15.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:34:15.219+0000 7f9003915700 1 -- start start 2024-02-18T11:34:15.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:34:15.219+0000 7f9003915700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ffc102760 0x7f8ffc101fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:34:15.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:34:15.219+0000 7f9003915700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ffc105090 0x7f8ffc100710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:34:15.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:34:15.219+0000 7f9003915700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8ffc1079c0 0x7f8ffc100c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:34:15.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:34:15.219+0000 7f9003915700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ffc1012c0 con 0x7f8ffc102760 2024-02-18T11:34:15.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:34:15.219+0000 7f9003915700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ffc101430 con 0x7f8ffc1079c0 2024-02-18T11:34:15.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:34:15.219+0000 7f9003915700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ffc1a1ed0 con 0x7f8ffc105090 2024-02-18T11:34:15.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:34:15.219+0000 7f90016b1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ffc102760 0x7f8ffc101fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:34:15.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:34:15.219+0000 7f90016b1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ffc102760 0x7f8ffc101fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54692/0 (socket says 172.21.15.12:54692) 2024-02-18T11:34:15.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:34:15.219+0000 7f90016b1700 1 -- 172.21.15.12:0/765389831 learned_addr learned my addr 172.21.15.12:0/765389831 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:34:15.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:34:15.219+0000 7f9001eb2700 1 --2- 172.21.15.12:0/765389831 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8ffc1079c0 0x7f8ffc100c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:34:15.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:34:15.219+0000 7f9000eb0700 1 --2- 172.21.15.12:0/765389831 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ffc105090 0x7f8ffc100710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:34:15.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:34:15.219+0000 7f90016b1700 1 -- 172.21.15.12:0/765389831 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ffc105090 msgr2=0x7f8ffc100710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:34:15.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:34:15.219+0000 7f90016b1700 1 --2- 172.21.15.12:0/765389831 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ffc105090 0x7f8ffc100710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:34:15.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:34:15.219+0000 7f90016b1700 1 -- 172.21.15.12:0/765389831 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8ffc1079c0 msgr2=0x7f8ffc100c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:34:15.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:34:15.219+0000 7f90016b1700 1 --2- 172.21.15.12:0/765389831 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8ffc1079c0 0x7f8ffc100c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:34:15.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:34:15.219+0000 7f90016b1700 1 -- 172.21.15.12:0/765389831 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ff4009c40 con 0x7f8ffc102760 2024-02-18T11:34:15.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:34:15.219+0000 7f9001eb2700 1 --2- 172.21.15.12:0/765389831 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8ffc1079c0 0x7f8ffc100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-18T11:34:15.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:34:15.220+0000 7f90016b1700 1 --2- 172.21.15.12:0/765389831 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ffc102760 0x7f8ffc101fb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8ff800e9a0 tx=0x7f8ff800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:34:15.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:34:15.280+0000 7f8ff27fc700 1 -- 172.21.15.12:0/765389831 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff8009790 con 0x7f8ffc102760 2024-02-18T11:34:15.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:34:15.280+0000 7f8ff27fc700 1 -- 172.21.15.12:0/765389831 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8ff8010430 con 0x7f8ffc102760 2024-02-18T11:34:15.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:34:15.280+0000 7f8ff27fc700 1 -- 172.21.15.12:0/765389831 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff800cea0 con 0x7f8ffc102760 2024-02-18T11:34:15.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:34:15.280+0000 7f9003915700 1 -- 172.21.15.12:0/765389831 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ffc1a20d0 con 0x7f8ffc102760 2024-02-18T11:34:15.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:34:15.280+0000 7f9003915700 1 -- 172.21.15.12:0/765389831 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ffc1a2550 con 0x7f8ffc102760 2024-02-18T11:34:15.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:34:15.281+0000 7f9003915700 1 -- 172.21.15.12:0/765389831 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ffc0fdcf0 con 0x7f8ffc102760 2024-02-18T11:34:15.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:34:15.283+0000 7f8ff27fc700 1 -- 172.21.15.12:0/765389831 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 15) v1 ==== 137189+0+0 (secure 0 0 0) 0x7f8ff8010910 con 0x7f8ffc102760 2024-02-18T11:34:15.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:34:15.283+0000 7f8ff27fc700 1 --2- 172.21.15.12:0/765389831 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f8fe80a27d0 0x7f8fe80a4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:34:15.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:34:15.284+0000 7f9000eb0700 1 --2- 172.21.15.12:0/765389831 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f8fe80a27d0 0x7f8fe80a4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:34:15.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:34:15.284+0000 7f8ff27fc700 1 -- 172.21.15.12:0/765389831 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 2851+0+0 (secure 0 0 0) 0x7f8ff8014070 con 0x7f8ffc102760 2024-02-18T11:34:15.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:34:15.284+0000 7f9000eb0700 1 --2- 172.21.15.12:0/765389831 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f8fe80a27d0 0x7f8fe80a4c90 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8fec005fd0 tx=0x7f8fec009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:34:15.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:34:15.287+0000 7f8ff27fc700 1 -- 172.21.15.12:0/765389831 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8ff8097900 con 0x7f8ffc102760 2024-02-18T11:34:15.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:34:15.499+0000 7f9003915700 1 -- 172.21.15.12:0/765389831 --> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f8ffc02d810 con 0x7f8fe80a27d0 2024-02-18T11:34:15.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:15 smithi012 ceph-mon[32460]: osdmap e19: 3 total, 3 up, 3 in 2024-02-18T11:34:15.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:15 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-18T11:34:15.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:15 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T11:34:15.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:15 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T11:34:15.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:15 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:34:15.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:15 smithi158 ceph-mon[36626]: osdmap e19: 3 total, 3 up, 3 in 2024-02-18T11:34:15.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:15 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-18T11:34:15.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:15 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T11:34:15.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:15 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T11:34:15.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:15 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:34:16.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:15 smithi188 ceph-mon[36925]: osdmap e19: 3 total, 3 up, 3 in 2024-02-18T11:34:16.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:15 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-18T11:34:16.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:15 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T11:34:16.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:15 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T11:34:16.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:15 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:34:16.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:16 smithi012 ceph-mon[32460]: pgmap v81: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-18T11:34:16.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:16 smithi012 ceph-mon[32460]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:34:16.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:16 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-18T11:34:16.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:16 smithi012 ceph-mon[32460]: osdmap e20: 3 total, 3 up, 3 in 2024-02-18T11:34:16.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:16 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-18T11:34:16.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:16 smithi158 ceph-mon[36626]: pgmap v81: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-18T11:34:16.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:16 smithi158 ceph-mon[36626]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:34:16.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:16 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-18T11:34:16.996 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:16 smithi158 ceph-mon[36626]: osdmap e20: 3 total, 3 up, 3 in 2024-02-18T11:34:16.996 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:16 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-18T11:34:17.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:16 smithi188 ceph-mon[36925]: pgmap v81: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-18T11:34:17.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:16 smithi188 ceph-mon[36925]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:34:17.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:16 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-18T11:34:17.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:16 smithi188 ceph-mon[36925]: osdmap e20: 3 total, 3 up, 3 in 2024-02-18T11:34:17.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:16 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-18T11:34:17.140 INFO:journalctl@ceph.osd.0.smithi012.stdout:Feb 18 11:34:16 smithi012 sudo[49294]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-18T11:34:17.140 INFO:journalctl@ceph.osd.0.smithi012.stdout:Feb 18 11:34:16 smithi012 sudo[49294]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-18T11:34:17.140 INFO:journalctl@ceph.osd.0.smithi012.stdout:Feb 18 11:34:16 smithi012 sudo[49294]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-18T11:34:17.140 INFO:journalctl@ceph.osd.0.smithi012.stdout:Feb 18 11:34:16 smithi012 sudo[49294]: pam_unix(sudo:session): session closed for user root 2024-02-18T11:34:17.423 INFO:journalctl@ceph.osd.0.smithi012.stdout:Feb 18 11:34:17 smithi012 sudo[49319]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-18T11:34:17.424 INFO:journalctl@ceph.osd.0.smithi012.stdout:Feb 18 11:34:17 smithi012 sudo[49319]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-18T11:34:17.424 INFO:journalctl@ceph.osd.0.smithi012.stdout:Feb 18 11:34:17 smithi012 sudo[49319]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-18T11:34:17.424 INFO:journalctl@ceph.osd.0.smithi012.stdout:Feb 18 11:34:17 smithi012 sudo[49319]: pam_unix(sudo:session): session closed for user root 2024-02-18T11:34:17.745 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:17 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-18T11:34:17.745 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:17 smithi158 ceph-mon[36626]: osdmap e21: 3 total, 3 up, 3 in 2024-02-18T11:34:17.745 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:17 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:17.745 INFO:journalctl@ceph.osd.1.smithi158.stdout:Feb 18 11:34:17 smithi158 sudo[44979]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-18T11:34:17.746 INFO:journalctl@ceph.osd.1.smithi158.stdout:Feb 18 11:34:17 smithi158 sudo[44979]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-18T11:34:17.746 INFO:journalctl@ceph.osd.1.smithi158.stdout:Feb 18 11:34:17 smithi158 sudo[44979]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-18T11:34:17.746 INFO:journalctl@ceph.osd.1.smithi158.stdout:Feb 18 11:34:17 smithi158 sudo[44979]: pam_unix(sudo:session): session closed for user root 2024-02-18T11:34:17.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:17 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-18T11:34:17.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:17 smithi012 ceph-mon[32460]: osdmap e21: 3 total, 3 up, 3 in 2024-02-18T11:34:17.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:17 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:18.014 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:17 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-18T11:34:18.014 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:17 smithi188 ceph-mon[36925]: osdmap e21: 3 total, 3 up, 3 in 2024-02-18T11:34:18.014 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:17 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:18.245 INFO:journalctl@ceph.osd.1.smithi158.stdout:Feb 18 11:34:17 smithi158 sudo[44982]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-18T11:34:18.245 INFO:journalctl@ceph.osd.1.smithi158.stdout:Feb 18 11:34:17 smithi158 sudo[44982]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-18T11:34:18.245 INFO:journalctl@ceph.osd.1.smithi158.stdout:Feb 18 11:34:17 smithi158 sudo[44982]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-18T11:34:18.245 INFO:journalctl@ceph.osd.1.smithi158.stdout:Feb 18 11:34:17 smithi158 sudo[44982]: pam_unix(sudo:session): session closed for user root 2024-02-18T11:34:18.265 INFO:journalctl@ceph.osd.2.smithi188.stdout:Feb 18 11:34:18 smithi188 sudo[45750]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-18T11:34:18.265 INFO:journalctl@ceph.osd.2.smithi188.stdout:Feb 18 11:34:18 smithi188 sudo[45750]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-18T11:34:18.266 INFO:journalctl@ceph.osd.2.smithi188.stdout:Feb 18 11:34:18 smithi188 sudo[45750]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-18T11:34:18.266 INFO:journalctl@ceph.osd.2.smithi188.stdout:Feb 18 11:34:18 smithi188 sudo[45750]: pam_unix(sudo:session): session closed for user root 2024-02-18T11:34:18.605 INFO:journalctl@ceph.osd.2.smithi188.stdout:Feb 18 11:34:18 smithi188 sudo[45753]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-18T11:34:18.605 INFO:journalctl@ceph.osd.2.smithi188.stdout:Feb 18 11:34:18 smithi188 sudo[45753]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-18T11:34:18.605 INFO:journalctl@ceph.osd.2.smithi188.stdout:Feb 18 11:34:18 smithi188 sudo[45753]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-18T11:34:18.605 INFO:journalctl@ceph.osd.2.smithi188.stdout:Feb 18 11:34:18 smithi188 sudo[45753]: pam_unix(sudo:session): session closed for user root 2024-02-18T11:34:18.807 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:18 smithi012 sudo[49404]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-18T11:34:18.807 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:18 smithi012 sudo[49404]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-18T11:34:18.807 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:18 smithi012 sudo[49404]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-18T11:34:18.807 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:18 smithi012 sudo[49404]: pam_unix(sudo:session): session closed for user root 2024-02-18T11:34:18.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:34:18.833+0000 7f8ff27fc700 1 -- 172.21.15.12:0/765389831 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mgrmap(e 16) v1 ==== 137245+0+0 (secure 0 0 0) 0x7f8ff8097cd0 con 0x7f8ffc102760 2024-02-18T11:34:19.067 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:18 smithi188 ceph-mon[36925]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-18T11:34:19.067 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:18 smithi188 ceph-mon[36925]: osdmap e22: 3 total, 3 up, 3 in 2024-02-18T11:34:19.068 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:18 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:19.068 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:18 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/2977385264' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3f9cc423-cdfa-4acf-aaa2-b58b32ccb9cf"}]: dispatch 2024-02-18T11:34:19.068 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:18 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/2977385264' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3f9cc423-cdfa-4acf-aaa2-b58b32ccb9cf"}]': finished 2024-02-18T11:34:19.068 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:18 smithi188 ceph-mon[36925]: osdmap e23: 4 total, 3 up, 4 in 2024-02-18T11:34:19.068 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:18 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-18T11:34:19.068 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:18 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:19.068 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:18 smithi188 ceph-mon[36925]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-18T11:34:19.068 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:19 smithi188 sudo[45756]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-18T11:34:19.068 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:19 smithi188 sudo[45756]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-18T11:34:19.068 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:19 smithi188 sudo[45756]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-18T11:34:19.068 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:19 smithi188 sudo[45756]: pam_unix(sudo:session): session closed for user root 2024-02-18T11:34:19.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:18 smithi012 sudo[49407]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-18T11:34:19.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:18 smithi012 sudo[49407]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-18T11:34:19.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:18 smithi012 sudo[49407]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-18T11:34:19.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:18 smithi012 sudo[49407]: pam_unix(sudo:session): session closed for user root 2024-02-18T11:34:19.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:18 smithi012 ceph-mon[32460]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-18T11:34:19.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:18 smithi012 ceph-mon[32460]: osdmap e22: 3 total, 3 up, 3 in 2024-02-18T11:34:19.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:18 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:19.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:18 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/2977385264' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3f9cc423-cdfa-4acf-aaa2-b58b32ccb9cf"}]: dispatch 2024-02-18T11:34:19.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:18 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/2977385264' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3f9cc423-cdfa-4acf-aaa2-b58b32ccb9cf"}]': finished 2024-02-18T11:34:19.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:18 smithi012 ceph-mon[32460]: osdmap e23: 4 total, 3 up, 4 in 2024-02-18T11:34:19.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:18 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-18T11:34:19.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:18 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:19.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:18 smithi012 ceph-mon[32460]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-18T11:34:19.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:18 smithi158 ceph-mon[36626]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-18T11:34:19.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:18 smithi158 ceph-mon[36626]: osdmap e22: 3 total, 3 up, 3 in 2024-02-18T11:34:19.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:18 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:19.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:18 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/2977385264' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3f9cc423-cdfa-4acf-aaa2-b58b32ccb9cf"}]: dispatch 2024-02-18T11:34:19.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:18 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/2977385264' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3f9cc423-cdfa-4acf-aaa2-b58b32ccb9cf"}]': finished 2024-02-18T11:34:19.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:18 smithi158 ceph-mon[36626]: osdmap e23: 4 total, 3 up, 4 in 2024-02-18T11:34:19.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:18 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-18T11:34:19.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:18 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:19.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:18 smithi158 ceph-mon[36626]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-18T11:34:19.355 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:19 smithi188 sudo[45759]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-18T11:34:19.355 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:19 smithi188 sudo[45759]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-18T11:34:19.355 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:19 smithi188 sudo[45759]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-18T11:34:19.355 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:19 smithi188 sudo[45759]: pam_unix(sudo:session): session closed for user root 2024-02-18T11:34:19.801 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:19 smithi158 sudo[44985]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-18T11:34:19.801 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:19 smithi158 sudo[44985]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-18T11:34:19.802 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:19 smithi158 sudo[44985]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-18T11:34:19.802 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:19 smithi158 sudo[44985]: pam_unix(sudo:session): session closed for user root 2024-02-18T11:34:20.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:19 smithi188 ceph-mon[36925]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-18T11:34:20.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:19 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-18T11:34:20.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:19 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-18T11:34:20.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:19 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T11:34:20.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:19 smithi188 ceph-mon[36925]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-18T11:34:20.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:19 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-18T11:34:20.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:19 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-18T11:34:20.106 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:19 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T11:34:20.106 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:19 smithi188 ceph-mon[36925]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-02-18T11:34:20.106 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:19 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/3753719507' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T11:34:20.106 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:19 smithi188 ceph-mon[36925]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-18T11:34:20.106 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:19 smithi188 ceph-mon[36925]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-18T11:34:20.106 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:19 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-18T11:34:20.106 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:19 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-18T11:34:20.106 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:19 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T11:34:20.106 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:19 smithi188 ceph-mon[36925]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-18T11:34:20.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:19 smithi012 ceph-mon[32460]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-18T11:34:20.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:19 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-18T11:34:20.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:19 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-18T11:34:20.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:19 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T11:34:20.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:19 smithi012 ceph-mon[32460]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-18T11:34:20.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:19 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-18T11:34:20.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:19 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-18T11:34:20.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:19 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T11:34:20.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:19 smithi012 ceph-mon[32460]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-02-18T11:34:20.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:19 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/3753719507' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T11:34:20.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:19 smithi012 ceph-mon[32460]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-18T11:34:20.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:19 smithi012 ceph-mon[32460]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-18T11:34:20.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:19 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-18T11:34:20.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:19 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-18T11:34:20.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:19 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T11:34:20.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:19 smithi012 ceph-mon[32460]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-18T11:34:20.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:19 smithi158 sudo[44988]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-18T11:34:20.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:19 smithi158 sudo[44988]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-18T11:34:20.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:19 smithi158 sudo[44988]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-18T11:34:20.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:19 smithi158 sudo[44988]: pam_unix(sudo:session): session closed for user root 2024-02-18T11:34:20.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:19 smithi158 ceph-mon[36626]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-18T11:34:20.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:19 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-18T11:34:20.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:19 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-18T11:34:20.247 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:19 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T11:34:20.247 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:19 smithi158 ceph-mon[36626]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-18T11:34:20.247 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:19 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-18T11:34:20.247 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:19 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-18T11:34:20.247 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:19 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T11:34:20.247 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:19 smithi158 ceph-mon[36626]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-02-18T11:34:20.248 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:19 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/3753719507' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T11:34:20.248 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:19 smithi158 ceph-mon[36626]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-18T11:34:20.248 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:19 smithi158 ceph-mon[36626]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-18T11:34:20.248 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:19 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-18T11:34:20.248 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:19 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-18T11:34:20.248 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:19 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T11:34:20.249 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:19 smithi158 ceph-mon[36626]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-18T11:34:21.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:20 smithi188 ceph-mon[36925]: pgmap v87: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-18T11:34:21.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:20 smithi012 ceph-mon[32460]: pgmap v87: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-18T11:34:21.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:20 smithi158 ceph-mon[36626]: pgmap v87: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-18T11:34:22.940 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:22 smithi012 ceph-mon[32460]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-02-18T11:34:23.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:22 smithi188 ceph-mon[36925]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-02-18T11:34:23.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:22 smithi158 ceph-mon[36626]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-02-18T11:34:24.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:24 smithi012 ceph-mon[32460]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-02-18T11:34:25.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:24 smithi188 ceph-mon[36925]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-02-18T11:34:25.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:24 smithi158 ceph-mon[36626]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-02-18T11:34:25.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:25 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-18T11:34:25.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:25 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:34:26.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:25 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-18T11:34:26.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:25 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:34:26.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:25 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-18T11:34:26.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:25 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:34:26.983 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:26 smithi012 ceph-mon[32460]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-02-18T11:34:26.983 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:26 smithi012 ceph-mon[32460]: Deploying daemon osd.3 on smithi012 2024-02-18T11:34:27.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:26 smithi188 ceph-mon[36925]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-02-18T11:34:27.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:26 smithi188 ceph-mon[36925]: Deploying daemon osd.3 on smithi012 2024-02-18T11:34:27.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:26 smithi158 ceph-mon[36626]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-02-18T11:34:27.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:26 smithi158 ceph-mon[36626]: Deploying daemon osd.3 on smithi012 2024-02-18T11:34:29.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:28 smithi188 ceph-mon[36925]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-02-18T11:34:29.126 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:28 smithi012 ceph-mon[32460]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-02-18T11:34:29.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:28 smithi158 ceph-mon[36626]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-02-18T11:34:31.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:30 smithi188 ceph-mon[36925]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-02-18T11:34:31.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:30 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:34:31.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:30 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:31.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:30 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:31.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:30 smithi012 ceph-mon[32460]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-02-18T11:34:31.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:30 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:34:31.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:30 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:31.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:30 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:31.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:30 smithi158 ceph-mon[36626]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-02-18T11:34:31.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:30 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:34:31.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:30 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:31.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:30 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:32.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:32 smithi012 ceph-mon[32460]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-02-18T11:34:33.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:32 smithi188 ceph-mon[36925]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-02-18T11:34:33.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:32 smithi158 ceph-mon[36626]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-02-18T11:34:34.344 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:34 smithi012 ceph-mon[32460]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-02-18T11:34:34.344 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:34 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:34.344 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:34 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:34.344 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:34 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:34:34.345 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:34 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:34:34.345 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:34 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:34.355 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:34 smithi188 ceph-mon[36925]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-02-18T11:34:34.355 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:34 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:34.355 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:34 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:34.355 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:34 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:34:34.355 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:34 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:34:34.355 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:34 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:34.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:34:34.407+0000 7f8ff27fc700 1 -- 172.21.15.12:0/765389831 <== mgr.14152 v2:172.21.15.12:6800/1153736685 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f8ffc02d810 con 0x7f8fe80a27d0 2024-02-18T11:34:34.404 INFO:teuthology.orchestra.run.smithi012.stdout:Created osd(s) 3 on host 'smithi012' 2024-02-18T11:34:34.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:34:34.410+0000 7f9003915700 1 -- 172.21.15.12:0/765389831 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f8fe80a27d0 msgr2=0x7f8fe80a4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:34:34.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:34:34.410+0000 7f9003915700 1 --2- 172.21.15.12:0/765389831 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f8fe80a27d0 0x7f8fe80a4c90 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8fec005fd0 tx=0x7f8fec009310 comp rx=0 tx=0).stop 2024-02-18T11:34:34.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:34:34.410+0000 7f9003915700 1 -- 172.21.15.12:0/765389831 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ffc102760 msgr2=0x7f8ffc101fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:34:34.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:34:34.410+0000 7f9003915700 1 --2- 172.21.15.12:0/765389831 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ffc102760 0x7f8ffc101fb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8ff800e9a0 tx=0x7f8ff800ecb0 comp rx=0 tx=0).stop 2024-02-18T11:34:34.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:34:34.410+0000 7f9003915700 1 -- 172.21.15.12:0/765389831 shutdown_connections 2024-02-18T11:34:34.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:34:34.410+0000 7f9003915700 1 --2- 172.21.15.12:0/765389831 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8ffc1079c0 0x7f8ffc100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:34:34.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:34:34.410+0000 7f9003915700 1 --2- 172.21.15.12:0/765389831 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f8fe80a27d0 0x7f8fe80a4c90 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:34:34.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:34:34.410+0000 7f9003915700 1 --2- 172.21.15.12:0/765389831 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ffc102760 0x7f8ffc101fb0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:34:34.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:34:34.410+0000 7f9003915700 1 --2- 172.21.15.12:0/765389831 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ffc105090 0x7f8ffc100710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:34:34.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:34:34.410+0000 7f9003915700 1 -- 172.21.15.12:0/765389831 >> 172.21.15.12:0/765389831 conn(0x7f8ffc0fc170 msgr2=0x7f8ffc103100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:34:34.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:34:34.410+0000 7f9003915700 1 -- 172.21.15.12:0/765389831 shutdown_connections 2024-02-18T11:34:34.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:34:34.410+0000 7f9003915700 1 -- 172.21.15.12:0/765389831 wait complete. 2024-02-18T11:34:34.495 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:34 smithi158 ceph-mon[36626]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-02-18T11:34:34.495 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:34 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:34.495 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:34 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:34.495 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:34 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:34:34.496 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:34 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:34:34.496 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:34 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:34.931 DEBUG:teuthology.orchestra.run.smithi012:osd.3> sudo journalctl -f -n 0 -u ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@osd.3.service 2024-02-18T11:34:34.933 INFO:tasks.cephadm:Deploying osd.4 on smithi158 with /dev/vg_nvme/lv_3... 2024-02-18T11:34:34.934 DEBUG:teuthology.orchestra.run.smithi158:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-02-18T11:34:35.176 INFO:teuthology.orchestra.run.smithi158.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T11:34:35.622 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:35 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:35.622 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:35 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:35.622 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:35 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:34:35.622 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:35 smithi158 ceph-mon[36626]: from='osd.3 [v2:172.21.15.12:6810/3758512503,v1:172.21.15.12:6811/3758512503]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-18T11:34:35.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:35 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:35.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:35 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:35.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:35 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:34:35.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:35 smithi012 ceph-mon[32460]: from='osd.3 [v2:172.21.15.12:6810/3758512503,v1:172.21.15.12:6811/3758512503]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-18T11:34:35.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:35 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:35.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:35 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:35.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:35 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:34:35.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:35 smithi188 ceph-mon[36925]: from='osd.3 [v2:172.21.15.12:6810/3758512503,v1:172.21.15.12:6811/3758512503]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-18T11:34:36.495 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:36 smithi158 ceph-mon[36626]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-02-18T11:34:36.495 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:36 smithi158 ceph-mon[36626]: from='osd.3 [v2:172.21.15.12:6810/3758512503,v1:172.21.15.12:6811/3758512503]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-18T11:34:36.495 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:36 smithi158 ceph-mon[36626]: osdmap e24: 4 total, 3 up, 4 in 2024-02-18T11:34:36.495 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:36 smithi158 ceph-mon[36626]: from='osd.3 [v2:172.21.15.12:6810/3758512503,v1:172.21.15.12:6811/3758512503]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-02-18T11:34:36.496 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:36 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-18T11:34:36.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:36 smithi012 ceph-mon[32460]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-02-18T11:34:36.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:36 smithi012 ceph-mon[32460]: from='osd.3 [v2:172.21.15.12:6810/3758512503,v1:172.21.15.12:6811/3758512503]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-18T11:34:36.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:36 smithi012 ceph-mon[32460]: osdmap e24: 4 total, 3 up, 4 in 2024-02-18T11:34:36.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:36 smithi012 ceph-mon[32460]: from='osd.3 [v2:172.21.15.12:6810/3758512503,v1:172.21.15.12:6811/3758512503]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-02-18T11:34:36.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:36 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-18T11:34:36.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:36 smithi188 ceph-mon[36925]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-02-18T11:34:36.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:36 smithi188 ceph-mon[36925]: from='osd.3 [v2:172.21.15.12:6810/3758512503,v1:172.21.15.12:6811/3758512503]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-18T11:34:36.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:36 smithi188 ceph-mon[36925]: osdmap e24: 4 total, 3 up, 4 in 2024-02-18T11:34:36.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:36 smithi188 ceph-mon[36925]: from='osd.3 [v2:172.21.15.12:6810/3758512503,v1:172.21.15.12:6811/3758512503]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-02-18T11:34:36.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:36 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-18T11:34:37.062 INFO:teuthology.orchestra.run.smithi158.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-02-18T11:34:37.062 INFO:teuthology.orchestra.run.smithi158.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-02-18T11:34:37.062 INFO:teuthology.orchestra.run.smithi158.stderr: stderr: 10+0 records in 2024-02-18T11:34:37.063 INFO:teuthology.orchestra.run.smithi158.stderr:10+0 records out 2024-02-18T11:34:37.063 INFO:teuthology.orchestra.run.smithi158.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00849322 s, 1.2 GB/s 2024-02-18T11:34:37.063 INFO:teuthology.orchestra.run.smithi158.stderr:--> Zapping successful for: 2024-02-18T11:34:37.610 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:37 smithi012 ceph-mon[32460]: from='osd.3 [v2:172.21.15.12:6810/3758512503,v1:172.21.15.12:6811/3758512503]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-02-18T11:34:37.610 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:37 smithi012 ceph-mon[32460]: osdmap e25: 4 total, 3 up, 4 in 2024-02-18T11:34:37.611 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:37 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-18T11:34:37.611 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:37 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-18T11:34:37.611 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:37 smithi012 ceph-mon[32460]: osdmap e26: 4 total, 3 up, 4 in 2024-02-18T11:34:37.611 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:37 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-18T11:34:37.611 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:37 smithi012 ceph-mon[32460]: from='osd.3 [v2:172.21.15.12:6810/3758512503,v1:172.21.15.12:6811/3758512503]' entity='osd.3' 2024-02-18T11:34:37.611 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:37 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-18T11:34:37.611 INFO:journalctl@ceph.osd.3.smithi012.stdout:Feb 18 11:34:37 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-3[51430]: 2024-02-18T11:34:37.240+0000 7f856a21d700 -1 osd.3 0 waiting for initial osdmap 2024-02-18T11:34:37.611 INFO:journalctl@ceph.osd.3.smithi012.stdout:Feb 18 11:34:37 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-3[51430]: 2024-02-18T11:34:37.251+0000 7f8562809700 -1 osd.3 26 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-18T11:34:37.668 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:37 smithi158 ceph-mon[36626]: from='osd.3 [v2:172.21.15.12:6810/3758512503,v1:172.21.15.12:6811/3758512503]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-02-18T11:34:37.669 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:37 smithi158 ceph-mon[36626]: osdmap e25: 4 total, 3 up, 4 in 2024-02-18T11:34:37.669 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:37 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-18T11:34:37.669 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:37 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-18T11:34:37.669 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:37 smithi158 ceph-mon[36626]: osdmap e26: 4 total, 3 up, 4 in 2024-02-18T11:34:37.669 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:37 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-18T11:34:37.669 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:37 smithi158 ceph-mon[36626]: from='osd.3 [v2:172.21.15.12:6810/3758512503,v1:172.21.15.12:6811/3758512503]' entity='osd.3' 2024-02-18T11:34:37.669 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:37 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-18T11:34:37.701 DEBUG:teuthology.orchestra.run.smithi158:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi158:vg_nvme/lv_3 2024-02-18T11:34:37.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:37 smithi188 ceph-mon[36925]: from='osd.3 [v2:172.21.15.12:6810/3758512503,v1:172.21.15.12:6811/3758512503]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-02-18T11:34:37.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:37 smithi188 ceph-mon[36925]: osdmap e25: 4 total, 3 up, 4 in 2024-02-18T11:34:37.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:37 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-18T11:34:37.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:37 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-18T11:34:37.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:37 smithi188 ceph-mon[36925]: osdmap e26: 4 total, 3 up, 4 in 2024-02-18T11:34:37.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:37 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-18T11:34:37.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:37 smithi188 ceph-mon[36925]: from='osd.3 [v2:172.21.15.12:6810/3758512503,v1:172.21.15.12:6811/3758512503]' entity='osd.3' 2024-02-18T11:34:37.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:37 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-18T11:34:37.940 INFO:teuthology.orchestra.run.smithi158.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T11:34:38.641 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:38 smithi158 ceph-mon[36626]: purged_snaps scrub starts 2024-02-18T11:34:38.641 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:38 smithi158 ceph-mon[36626]: purged_snaps scrub ok 2024-02-18T11:34:38.641 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:38 smithi158 ceph-mon[36626]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-02-18T11:34:38.642 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:38 smithi158 ceph-mon[36626]: osd.3 [v2:172.21.15.12:6810/3758512503,v1:172.21.15.12:6811/3758512503] boot 2024-02-18T11:34:38.642 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:38 smithi158 ceph-mon[36626]: osdmap e27: 4 total, 4 up, 4 in 2024-02-18T11:34:38.642 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:38 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-18T11:34:38.642 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:38 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:38.642 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:38 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:38.642 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:38 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:34:38.642 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:38 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:34:38.642 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:38 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:38.642 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:38 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:34:38.642 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:38 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:34:38.642 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:38 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:38.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:38 smithi188 ceph-mon[36925]: purged_snaps scrub starts 2024-02-18T11:34:38.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:38 smithi188 ceph-mon[36925]: purged_snaps scrub ok 2024-02-18T11:34:38.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:38 smithi188 ceph-mon[36925]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-02-18T11:34:38.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:38 smithi188 ceph-mon[36925]: osd.3 [v2:172.21.15.12:6810/3758512503,v1:172.21.15.12:6811/3758512503] boot 2024-02-18T11:34:38.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:38 smithi188 ceph-mon[36925]: osdmap e27: 4 total, 4 up, 4 in 2024-02-18T11:34:38.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:38 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-18T11:34:38.856 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:38 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:38.856 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:38 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:38.856 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:38 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:34:38.856 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:38 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:34:38.856 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:38 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:38.856 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:38 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:34:38.856 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:38 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:34:38.856 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:38 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:38.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:38 smithi012 ceph-mon[32460]: purged_snaps scrub starts 2024-02-18T11:34:38.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:38 smithi012 ceph-mon[32460]: purged_snaps scrub ok 2024-02-18T11:34:38.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:38 smithi012 ceph-mon[32460]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-02-18T11:34:38.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:38 smithi012 ceph-mon[32460]: osd.3 [v2:172.21.15.12:6810/3758512503,v1:172.21.15.12:6811/3758512503] boot 2024-02-18T11:34:38.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:38 smithi012 ceph-mon[32460]: osdmap e27: 4 total, 4 up, 4 in 2024-02-18T11:34:38.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:38 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-18T11:34:38.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:38 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:38.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:38 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:38.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:38 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:34:38.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:38 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:34:38.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:38 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:38.925 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:38 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:34:38.925 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:38 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:34:38.925 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:38 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:39.440 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:34:39.442+0000 7f27616a0700 1 -- 172.21.15.158:0/28673631 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f275c101de0 msgr2=0x7f275c102240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:34:39.440 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:34:39.442+0000 7f27616a0700 1 --2- 172.21.15.158:0/28673631 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f275c101de0 0x7f275c102240 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f27500099c0 tx=0x7f2750009cd0 comp rx=0 tx=0).stop 2024-02-18T11:34:39.440 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:34:39.442+0000 7f27616a0700 1 -- 172.21.15.158:0/28673631 shutdown_connections 2024-02-18T11:34:39.440 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:34:39.442+0000 7f27616a0700 1 --2- 172.21.15.158:0/28673631 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f275c102780 0x7f275c109030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:34:39.440 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:34:39.442+0000 7f27616a0700 1 --2- 172.21.15.158:0/28673631 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f275c100be0 0x7f275c101000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:34:39.440 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:34:39.442+0000 7f27616a0700 1 --2- 172.21.15.158:0/28673631 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f275c101de0 0x7f275c102240 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:34:39.440 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:34:39.442+0000 7f27616a0700 1 -- 172.21.15.158:0/28673631 >> 172.21.15.158:0/28673631 conn(0x7f275c0fc180 msgr2=0x7f275c0fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:34:39.440 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:34:39.443+0000 7f27616a0700 1 -- 172.21.15.158:0/28673631 shutdown_connections 2024-02-18T11:34:39.440 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:34:39.443+0000 7f27616a0700 1 -- 172.21.15.158:0/28673631 wait complete. 2024-02-18T11:34:39.440 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:34:39.443+0000 7f27616a0700 1 Processor -- start 2024-02-18T11:34:39.440 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:34:39.443+0000 7f27616a0700 1 -- start start 2024-02-18T11:34:39.441 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:34:39.443+0000 7f27616a0700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f275c100be0 0x7f275c197cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:34:39.441 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:34:39.443+0000 7f27616a0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f275c101de0 0x7f275c198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:34:39.441 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:34:39.443+0000 7f27616a0700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f275c102780 0x7f275c19f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:34:39.441 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:34:39.443+0000 7f27616a0700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f275c19f840 con 0x7f275c101de0 2024-02-18T11:34:39.441 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:34:39.443+0000 7f27616a0700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f275c19f980 con 0x7f275c100be0 2024-02-18T11:34:39.441 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:34:39.443+0000 7f27616a0700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f275c19fb40 con 0x7f275c102780 2024-02-18T11:34:39.441 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:34:39.444+0000 7f275affd700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f275c100be0 0x7f275c197cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:34:39.441 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:34:39.444+0000 7f275affd700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f275c100be0 0x7f275c197cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.158:3300/0 says I am v2:172.21.15.158:54190/0 (socket says 172.21.15.158:54190) 2024-02-18T11:34:39.441 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:34:39.444+0000 7f275affd700 1 -- 172.21.15.158:0/3684447604 learned_addr learned my addr 172.21.15.158:0/3684447604 (peer_addr_for_me v2:172.21.15.158:0/0) 2024-02-18T11:34:39.441 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:34:39.444+0000 7f275a7fc700 1 --2- 172.21.15.158:0/3684447604 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f275c101de0 0x7f275c198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:34:39.441 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:34:39.444+0000 7f275affd700 1 -- 172.21.15.158:0/3684447604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f275c102780 msgr2=0x7f275c19f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:34:39.441 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:34:39.444+0000 7f275affd700 1 --2- 172.21.15.158:0/3684447604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f275c102780 0x7f275c19f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:34:39.441 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:34:39.444+0000 7f275affd700 1 -- 172.21.15.158:0/3684447604 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f275c101de0 msgr2=0x7f275c198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:34:39.441 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:34:39.444+0000 7f275affd700 1 --2- 172.21.15.158:0/3684447604 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f275c101de0 0x7f275c198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:34:39.442 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:34:39.444+0000 7f275affd700 1 -- 172.21.15.158:0/3684447604 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27500095f0 con 0x7f275c100be0 2024-02-18T11:34:39.442 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:34:39.444+0000 7f275a7fc700 1 --2- 172.21.15.158:0/3684447604 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f275c101de0 0x7f275c198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:34:39.442 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:34:39.444+0000 7f275affd700 1 --2- 172.21.15.158:0/3684447604 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f275c100be0 0x7f275c197cf0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f274c00d980 tx=0x7f274c00dd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:34:39.442 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:34:39.444+0000 7f2743fff700 1 -- 172.21.15.158:0/3684447604 <== mon.2 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f274c00de60 con 0x7f275c100be0 2024-02-18T11:34:39.442 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:34:39.444+0000 7f2743fff700 1 -- 172.21.15.158:0/3684447604 <== mon.2 v2:172.21.15.158:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f274c004500 con 0x7f275c100be0 2024-02-18T11:34:39.442 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:34:39.444+0000 7f27616a0700 1 -- 172.21.15.158:0/3684447604 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f275c19fe20 con 0x7f275c100be0 2024-02-18T11:34:39.442 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:34:39.444+0000 7f2743fff700 1 -- 172.21.15.158:0/3684447604 <== mon.2 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f274c003ce0 con 0x7f275c100be0 2024-02-18T11:34:39.442 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:34:39.444+0000 7f27616a0700 1 -- 172.21.15.158:0/3684447604 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f275c1a0230 con 0x7f275c100be0 2024-02-18T11:34:39.443 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:34:39.445+0000 7f27616a0700 1 -- 172.21.15.158:0/3684447604 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f275c10c520 con 0x7f275c100be0 2024-02-18T11:34:39.443 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:34:39.447+0000 7f2743fff700 1 -- 172.21.15.158:0/3684447604 <== mon.2 v2:172.21.15.158:3300/0 4 ==== mgrmap(e 16) v1 ==== 137245+0+0 (secure 0 0 0) 0x7f274c009cb0 con 0x7f275c100be0 2024-02-18T11:34:39.443 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:34:39.448+0000 7f2743fff700 1 --2- 172.21.15.158:0/3684447604 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f27440a27e0 0x7f27440a4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:34:39.443 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:34:39.448+0000 7f2743fff700 1 -- 172.21.15.158:0/3684447604 <== mon.2 v2:172.21.15.158:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 3698+0+0 (secure 0 0 0) 0x7f274c0cd3c0 con 0x7f275c100be0 2024-02-18T11:34:39.443 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:34:39.448+0000 7f275a7fc700 1 --2- 172.21.15.158:0/3684447604 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f27440a27e0 0x7f27440a4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:34:39.444 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:34:39.449+0000 7f275a7fc700 1 --2- 172.21.15.158:0/3684447604 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f27440a27e0 0x7f27440a4ca0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f27500038c0 tx=0x7f2750005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:34:39.446 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:34:39.452+0000 7f2743fff700 1 -- 172.21.15.158:0/3684447604 <== mon.2 v2:172.21.15.158:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f274c097350 con 0x7f275c100be0 2024-02-18T11:34:39.648 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:34:39.653+0000 7f27616a0700 1 -- 172.21.15.158:0/3684447604 --> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi158:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f275c068a10 con 0x7f27440a27e0 2024-02-18T11:34:39.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:39 smithi188 ceph-mon[36925]: Detected new or changed devices on smithi012 2024-02-18T11:34:39.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:39 smithi188 ceph-mon[36925]: Adjusting osd_memory_target on smithi012 to 8546M 2024-02-18T11:34:39.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:39 smithi188 ceph-mon[36925]: osdmap e28: 4 total, 4 up, 4 in 2024-02-18T11:34:39.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:39 smithi012 ceph-mon[32460]: Detected new or changed devices on smithi012 2024-02-18T11:34:39.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:39 smithi012 ceph-mon[32460]: Adjusting osd_memory_target on smithi012 to 8546M 2024-02-18T11:34:39.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:39 smithi012 ceph-mon[32460]: osdmap e28: 4 total, 4 up, 4 in 2024-02-18T11:34:39.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:39 smithi158 ceph-mon[36626]: Detected new or changed devices on smithi012 2024-02-18T11:34:39.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:39 smithi158 ceph-mon[36626]: Adjusting osd_memory_target on smithi012 to 8546M 2024-02-18T11:34:39.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:39 smithi158 ceph-mon[36626]: osdmap e28: 4 total, 4 up, 4 in 2024-02-18T11:34:40.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:40 smithi188 ceph-mon[36925]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-18T11:34:40.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:40 smithi188 ceph-mon[36925]: osdmap e29: 4 total, 4 up, 4 in 2024-02-18T11:34:40.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:40 smithi188 ceph-mon[36925]: from='client.24227 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi158:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:34:40.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:40 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T11:34:40.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:40 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T11:34:40.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:40 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:34:40.910 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:40 smithi158 ceph-mon[36626]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-18T11:34:40.910 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:40 smithi158 ceph-mon[36626]: osdmap e29: 4 total, 4 up, 4 in 2024-02-18T11:34:40.910 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:40 smithi158 ceph-mon[36626]: from='client.24227 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi158:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:34:40.910 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:40 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T11:34:40.910 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:40 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T11:34:40.910 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:40 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:34:40.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:40 smithi012 ceph-mon[32460]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-18T11:34:40.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:40 smithi012 ceph-mon[32460]: osdmap e29: 4 total, 4 up, 4 in 2024-02-18T11:34:40.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:40 smithi012 ceph-mon[32460]: from='client.24227 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi158:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:34:40.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:40 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T11:34:40.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:40 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T11:34:40.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:40 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:34:41.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:41 smithi188 ceph-mon[36925]: osdmap e30: 4 total, 4 up, 4 in 2024-02-18T11:34:41.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:41 smithi012 ceph-mon[32460]: osdmap e30: 4 total, 4 up, 4 in 2024-02-18T11:34:41.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:41 smithi158 ceph-mon[36626]: osdmap e30: 4 total, 4 up, 4 in 2024-02-18T11:34:42.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:42 smithi188 ceph-mon[36925]: pgmap v105: 1 pgs: 1 peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-02-18T11:34:42.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:42 smithi188 ceph-mon[36925]: osdmap e31: 4 total, 4 up, 4 in 2024-02-18T11:34:42.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:42 smithi188 ceph-mon[36925]: from='client.? 172.21.15.158:0/567862217' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a92e20ec-af1c-4ef4-a0cb-caba5fcc71aa"}]: dispatch 2024-02-18T11:34:42.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:42 smithi188 ceph-mon[36925]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a92e20ec-af1c-4ef4-a0cb-caba5fcc71aa"}]: dispatch 2024-02-18T11:34:42.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:42 smithi188 ceph-mon[36925]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a92e20ec-af1c-4ef4-a0cb-caba5fcc71aa"}]': finished 2024-02-18T11:34:42.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:42 smithi188 ceph-mon[36925]: osdmap e32: 5 total, 4 up, 5 in 2024-02-18T11:34:42.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:42 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-18T11:34:42.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:42 smithi012 ceph-mon[32460]: pgmap v105: 1 pgs: 1 peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-02-18T11:34:42.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:42 smithi012 ceph-mon[32460]: osdmap e31: 4 total, 4 up, 4 in 2024-02-18T11:34:42.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:42 smithi012 ceph-mon[32460]: from='client.? 172.21.15.158:0/567862217' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a92e20ec-af1c-4ef4-a0cb-caba5fcc71aa"}]: dispatch 2024-02-18T11:34:42.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:42 smithi012 ceph-mon[32460]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a92e20ec-af1c-4ef4-a0cb-caba5fcc71aa"}]: dispatch 2024-02-18T11:34:42.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:42 smithi012 ceph-mon[32460]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a92e20ec-af1c-4ef4-a0cb-caba5fcc71aa"}]': finished 2024-02-18T11:34:42.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:42 smithi012 ceph-mon[32460]: osdmap e32: 5 total, 4 up, 5 in 2024-02-18T11:34:42.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:42 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-18T11:34:42.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:42 smithi158 ceph-mon[36626]: pgmap v105: 1 pgs: 1 peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-02-18T11:34:42.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:42 smithi158 ceph-mon[36626]: osdmap e31: 4 total, 4 up, 4 in 2024-02-18T11:34:42.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:42 smithi158 ceph-mon[36626]: from='client.? 172.21.15.158:0/567862217' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a92e20ec-af1c-4ef4-a0cb-caba5fcc71aa"}]: dispatch 2024-02-18T11:34:42.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:42 smithi158 ceph-mon[36626]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a92e20ec-af1c-4ef4-a0cb-caba5fcc71aa"}]: dispatch 2024-02-18T11:34:42.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:42 smithi158 ceph-mon[36626]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a92e20ec-af1c-4ef4-a0cb-caba5fcc71aa"}]': finished 2024-02-18T11:34:42.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:42 smithi158 ceph-mon[36626]: osdmap e32: 5 total, 4 up, 5 in 2024-02-18T11:34:42.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:42 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-18T11:34:43.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:43 smithi188 ceph-mon[36925]: from='client.? 172.21.15.158:0/233425996' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T11:34:43.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:43 smithi012 ceph-mon[32460]: from='client.? 172.21.15.158:0/233425996' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T11:34:43.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:43 smithi158 ceph-mon[36626]: from='client.? 172.21.15.158:0/233425996' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T11:34:44.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:44 smithi188 ceph-mon[36925]: pgmap v108: 1 pgs: 1 peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-02-18T11:34:44.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:44 smithi012 ceph-mon[32460]: pgmap v108: 1 pgs: 1 peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-02-18T11:34:44.996 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:44 smithi158 ceph-mon[36626]: pgmap v108: 1 pgs: 1 peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-02-18T11:34:46.719 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:46 smithi158 ceph-mon[36626]: pgmap v109: 1 pgs: 1 peering; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-02-18T11:34:46.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:46 smithi188 ceph-mon[36925]: pgmap v109: 1 pgs: 1 peering; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-02-18T11:34:46.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:46 smithi012 ceph-mon[32460]: pgmap v109: 1 pgs: 1 peering; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-02-18T11:34:48.638 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:48 smithi158 ceph-mon[36626]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-02-18T11:34:48.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:48 smithi188 ceph-mon[36925]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-02-18T11:34:48.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:48 smithi012 ceph-mon[32460]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-02-18T11:34:50.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:50 smithi188 ceph-mon[36925]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-18T11:34:50.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:50 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-18T11:34:50.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:50 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:34:50.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:50 smithi188 ceph-mon[36925]: Deploying daemon osd.4 on smithi158 2024-02-18T11:34:50.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:50 smithi012 ceph-mon[32460]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-18T11:34:50.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:50 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-18T11:34:50.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:50 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:34:50.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:50 smithi012 ceph-mon[32460]: Deploying daemon osd.4 on smithi158 2024-02-18T11:34:50.972 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:50 smithi158 ceph-mon[36626]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-18T11:34:50.972 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:50 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-18T11:34:50.972 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:50 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:34:50.972 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:50 smithi158 ceph-mon[36626]: Deploying daemon osd.4 on smithi158 2024-02-18T11:34:52.745 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:52 smithi158 ceph-mon[36626]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-18T11:34:52.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:52 smithi188 ceph-mon[36925]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-18T11:34:52.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:52 smithi012 ceph-mon[32460]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-18T11:34:54.596 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:54 smithi158 ceph-mon[36626]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-18T11:34:54.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:54 smithi188 ceph-mon[36925]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-18T11:34:54.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:54 smithi012 ceph-mon[32460]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-18T11:34:56.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:56 smithi012 ceph-mon[32460]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-18T11:34:56.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:56 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:56.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:56 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:56.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:56 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:34:56.495 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:56 smithi158 ceph-mon[36626]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-18T11:34:56.495 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:56 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:56.495 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:56 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:56.495 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:56 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:34:56.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:56 smithi188 ceph-mon[36925]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-18T11:34:56.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:56 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:56.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:56 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:56.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:56 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:34:58.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:58 smithi158 ceph-mon[36626]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-18T11:34:58.268 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:34:58.272+0000 7f2743fff700 1 -- 172.21.15.158:0/3684447604 <== mgr.14152 v2:172.21.15.12:6800/1153736685 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f275c068a10 con 0x7f27440a27e0 2024-02-18T11:34:58.268 INFO:teuthology.orchestra.run.smithi158.stdout:Created osd(s) 4 on host 'smithi158' 2024-02-18T11:34:58.270 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:34:58.275+0000 7f27616a0700 1 -- 172.21.15.158:0/3684447604 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f27440a27e0 msgr2=0x7f27440a4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:34:58.270 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:34:58.275+0000 7f27616a0700 1 --2- 172.21.15.158:0/3684447604 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f27440a27e0 0x7f27440a4ca0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f27500038c0 tx=0x7f2750005fd0 comp rx=0 tx=0).stop 2024-02-18T11:34:58.270 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:34:58.275+0000 7f27616a0700 1 -- 172.21.15.158:0/3684447604 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f275c100be0 msgr2=0x7f275c197cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:34:58.270 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:34:58.275+0000 7f27616a0700 1 --2- 172.21.15.158:0/3684447604 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f275c100be0 0x7f275c197cf0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f274c00d980 tx=0x7f274c00dd40 comp rx=0 tx=0).stop 2024-02-18T11:34:58.270 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:34:58.275+0000 7f27616a0700 1 -- 172.21.15.158:0/3684447604 shutdown_connections 2024-02-18T11:34:58.270 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:34:58.275+0000 7f27616a0700 1 --2- 172.21.15.158:0/3684447604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f275c102780 0x7f275c19f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:34:58.270 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:34:58.275+0000 7f27616a0700 1 --2- 172.21.15.158:0/3684447604 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f275c100be0 0x7f275c197cf0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:34:58.270 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:34:58.275+0000 7f27616a0700 1 --2- 172.21.15.158:0/3684447604 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f27440a27e0 0x7f27440a4ca0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:34:58.270 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:34:58.275+0000 7f27616a0700 1 --2- 172.21.15.158:0/3684447604 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f275c101de0 0x7f275c198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:34:58.270 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:34:58.275+0000 7f27616a0700 1 -- 172.21.15.158:0/3684447604 >> 172.21.15.158:0/3684447604 conn(0x7f275c0fc180 msgr2=0x7f275c069960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:34:58.270 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:34:58.275+0000 7f27616a0700 1 -- 172.21.15.158:0/3684447604 shutdown_connections 2024-02-18T11:34:58.270 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:34:58.275+0000 7f27616a0700 1 -- 172.21.15.158:0/3684447604 wait complete. 2024-02-18T11:34:58.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:58 smithi188 ceph-mon[36925]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-18T11:34:58.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:58 smithi012 ceph-mon[32460]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-18T11:34:59.239 DEBUG:teuthology.orchestra.run.smithi158:osd.4> sudo journalctl -f -n 0 -u ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@osd.4.service 2024-02-18T11:34:59.241 INFO:tasks.cephadm:Deploying osd.5 on smithi188 with /dev/vg_nvme/lv_3... 2024-02-18T11:34:59.241 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-02-18T11:34:59.484 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.c/config 2024-02-18T11:34:59.495 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:59 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:59.495 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:59 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:59.495 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:59 smithi158 ceph-mon[36626]: from='osd.4 [v2:172.21.15.158:6808/3892820997,v1:172.21.15.158:6809/3892820997]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-18T11:34:59.495 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:34:59 smithi158 ceph-mon[36626]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-18T11:34:59.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:59 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:59.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:59 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:59.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:59 smithi188 ceph-mon[36925]: from='osd.4 [v2:172.21.15.158:6808/3892820997,v1:172.21.15.158:6809/3892820997]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-18T11:34:59.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:34:59 smithi188 ceph-mon[36925]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-18T11:34:59.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:59 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:59.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:59 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:34:59.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:59 smithi012 ceph-mon[32460]: from='osd.4 [v2:172.21.15.158:6808/3892820997,v1:172.21.15.158:6809/3892820997]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-18T11:34:59.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:34:59 smithi012 ceph-mon[32460]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-18T11:35:00.341 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:00 smithi158 ceph-mon[36626]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-18T11:35:00.341 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:00 smithi158 ceph-mon[36626]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-18T11:35:00.341 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:00 smithi158 ceph-mon[36626]: osdmap e33: 5 total, 4 up, 5 in 2024-02-18T11:35:00.341 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:00 smithi158 ceph-mon[36626]: from='osd.4 [v2:172.21.15.158:6808/3892820997,v1:172.21.15.158:6809/3892820997]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi158", "root=default"]}]: dispatch 2024-02-18T11:35:00.341 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:00 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-18T11:35:00.341 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:00 smithi158 ceph-mon[36626]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi158", "root=default"]}]: dispatch 2024-02-18T11:35:00.341 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:00 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:00.342 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:00 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:00.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:00 smithi188 ceph-mon[36925]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-18T11:35:00.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:00 smithi188 ceph-mon[36925]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-18T11:35:00.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:00 smithi188 ceph-mon[36925]: osdmap e33: 5 total, 4 up, 5 in 2024-02-18T11:35:00.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:00 smithi188 ceph-mon[36925]: from='osd.4 [v2:172.21.15.158:6808/3892820997,v1:172.21.15.158:6809/3892820997]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi158", "root=default"]}]: dispatch 2024-02-18T11:35:00.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:00 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-18T11:35:00.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:00 smithi188 ceph-mon[36925]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi158", "root=default"]}]: dispatch 2024-02-18T11:35:00.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:00 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:00.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:00 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:00.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:00 smithi012 ceph-mon[32460]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-18T11:35:00.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:00 smithi012 ceph-mon[32460]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-18T11:35:00.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:00 smithi012 ceph-mon[32460]: osdmap e33: 5 total, 4 up, 5 in 2024-02-18T11:35:00.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:00 smithi012 ceph-mon[32460]: from='osd.4 [v2:172.21.15.158:6808/3892820997,v1:172.21.15.158:6809/3892820997]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi158", "root=default"]}]: dispatch 2024-02-18T11:35:00.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:00 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-18T11:35:00.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:00 smithi012 ceph-mon[32460]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi158", "root=default"]}]: dispatch 2024-02-18T11:35:00.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:00 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:00.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:00 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:01.196 INFO:journalctl@ceph.osd.4.smithi158.stdout:Feb 18 11:35:00 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-4[47822]: 2024-02-18T11:35:00.967+0000 7fac7709b700 -1 osd.4 0 waiting for initial osdmap 2024-02-18T11:35:01.196 INFO:journalctl@ceph.osd.4.smithi158.stdout:Feb 18 11:35:00 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-4[47822]: 2024-02-18T11:35:00.975+0000 7fac6f687700 -1 osd.4 34 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-18T11:35:01.323 INFO:teuthology.orchestra.run.smithi188.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-02-18T11:35:01.324 INFO:teuthology.orchestra.run.smithi188.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-02-18T11:35:01.324 INFO:teuthology.orchestra.run.smithi188.stderr: stderr: 10+0 records in 2024-02-18T11:35:01.324 INFO:teuthology.orchestra.run.smithi188.stderr:10+0 records out 2024-02-18T11:35:01.324 INFO:teuthology.orchestra.run.smithi188.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0137439 s, 763 MB/s 2024-02-18T11:35:01.324 INFO:teuthology.orchestra.run.smithi188.stderr:--> Zapping successful for: 2024-02-18T11:35:01.495 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:01 smithi158 ceph-mon[36626]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi158", "root=default"]}]': finished 2024-02-18T11:35:01.495 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:01 smithi158 ceph-mon[36626]: osdmap e34: 5 total, 4 up, 5 in 2024-02-18T11:35:01.495 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:01 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-18T11:35:01.495 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:01 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-18T11:35:01.495 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:01 smithi158 ceph-mon[36626]: from='osd.4 ' entity='osd.4' 2024-02-18T11:35:01.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:01 smithi188 ceph-mon[36925]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi158", "root=default"]}]': finished 2024-02-18T11:35:01.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:01 smithi188 ceph-mon[36925]: osdmap e34: 5 total, 4 up, 5 in 2024-02-18T11:35:01.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:01 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-18T11:35:01.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:01 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-18T11:35:01.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:01 smithi188 ceph-mon[36925]: from='osd.4 ' entity='osd.4' 2024-02-18T11:35:01.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:01 smithi012 ceph-mon[32460]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi158", "root=default"]}]': finished 2024-02-18T11:35:01.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:01 smithi012 ceph-mon[32460]: osdmap e34: 5 total, 4 up, 5 in 2024-02-18T11:35:01.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:01 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-18T11:35:01.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:01 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-18T11:35:01.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:01 smithi012 ceph-mon[32460]: from='osd.4 ' entity='osd.4' 2024-02-18T11:35:01.892 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi188:vg_nvme/lv_3 2024-02-18T11:35:02.135 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.c/config 2024-02-18T11:35:02.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:02 smithi012 ceph-mon[32460]: purged_snaps scrub starts 2024-02-18T11:35:02.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:02 smithi012 ceph-mon[32460]: purged_snaps scrub ok 2024-02-18T11:35:02.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:02 smithi012 ceph-mon[32460]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-18T11:35:02.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:02 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-18T11:35:02.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:02 smithi012 ceph-mon[32460]: osd.4 [v2:172.21.15.158:6808/3892820997,v1:172.21.15.158:6809/3892820997] boot 2024-02-18T11:35:02.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:02 smithi012 ceph-mon[32460]: osdmap e35: 5 total, 5 up, 5 in 2024-02-18T11:35:02.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:02 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-18T11:35:02.560 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:02 smithi158 ceph-mon[36626]: purged_snaps scrub starts 2024-02-18T11:35:02.560 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:02 smithi158 ceph-mon[36626]: purged_snaps scrub ok 2024-02-18T11:35:02.560 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:02 smithi158 ceph-mon[36626]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-18T11:35:02.560 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:02 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-18T11:35:02.560 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:02 smithi158 ceph-mon[36626]: osd.4 [v2:172.21.15.158:6808/3892820997,v1:172.21.15.158:6809/3892820997] boot 2024-02-18T11:35:02.560 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:02 smithi158 ceph-mon[36626]: osdmap e35: 5 total, 5 up, 5 in 2024-02-18T11:35:02.560 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:02 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-18T11:35:02.576 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:02 smithi188 ceph-mon[36925]: purged_snaps scrub starts 2024-02-18T11:35:02.576 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:02 smithi188 ceph-mon[36925]: purged_snaps scrub ok 2024-02-18T11:35:02.576 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:02 smithi188 ceph-mon[36925]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-18T11:35:02.576 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:02 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-18T11:35:02.576 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:02 smithi188 ceph-mon[36925]: osd.4 [v2:172.21.15.158:6808/3892820997,v1:172.21.15.158:6809/3892820997] boot 2024-02-18T11:35:02.576 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:02 smithi188 ceph-mon[36925]: osdmap e35: 5 total, 5 up, 5 in 2024-02-18T11:35:02.576 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:02 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-18T11:35:03.728 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:35:03.730+0000 7fd79e691700 1 -- 172.21.15.188:0/307125446 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd798103940 msgr2=0x7fd79810a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:35:03.729 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:35:03.730+0000 7fd79e691700 1 --2- 172.21.15.188:0/307125446 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd798103940 0x7fd79810a1f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd79000b0a0 tx=0x7fd79000b3b0 comp rx=0 tx=0).stop 2024-02-18T11:35:03.729 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:35:03.731+0000 7fd79e691700 1 -- 172.21.15.188:0/307125446 shutdown_connections 2024-02-18T11:35:03.729 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:35:03.731+0000 7fd79e691700 1 --2- 172.21.15.188:0/307125446 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd798103940 0x7fd79810a1f0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:35:03.729 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:35:03.731+0000 7fd79e691700 1 --2- 172.21.15.188:0/307125446 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd7980fffc0 0x7fd7981003e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:35:03.729 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:35:03.731+0000 7fd79e691700 1 --2- 172.21.15.188:0/307125446 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd798102fa0 0x7fd798103400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:35:03.729 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:35:03.731+0000 7fd79e691700 1 -- 172.21.15.188:0/307125446 >> 172.21.15.188:0/307125446 conn(0x7fd7980fb940 msgr2=0x7fd7980fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:35:03.729 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:35:03.731+0000 7fd79e691700 1 -- 172.21.15.188:0/307125446 shutdown_connections 2024-02-18T11:35:03.729 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:35:03.731+0000 7fd79e691700 1 -- 172.21.15.188:0/307125446 wait complete. 2024-02-18T11:35:03.729 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:35:03.731+0000 7fd79e691700 1 Processor -- start 2024-02-18T11:35:03.729 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:35:03.731+0000 7fd79e691700 1 -- start start 2024-02-18T11:35:03.729 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:35:03.731+0000 7fd79e691700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7980fffc0 0x7fd798102900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:35:03.729 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:35:03.731+0000 7fd79e691700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd798102fa0 0x7fd798100f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:35:03.729 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:35:03.731+0000 7fd79e691700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd798103940 0x7fd798101490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:35:03.730 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:35:03.732+0000 7fd79e691700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd798101c10 con 0x7fd798102fa0 2024-02-18T11:35:03.730 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:35:03.732+0000 7fd79e691700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd798101d80 con 0x7fd798103940 2024-02-18T11:35:03.730 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:35:03.732+0000 7fd79e691700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7981a1d20 con 0x7fd7980fffc0 2024-02-18T11:35:03.730 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:35:03.732+0000 7fd797fff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7980fffc0 0x7fd798102900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:35:03.730 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:35:03.732+0000 7fd797fff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7980fffc0 0x7fd798102900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:36452/0 (socket says 172.21.15.188:36452) 2024-02-18T11:35:03.730 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:35:03.732+0000 7fd797fff700 1 -- 172.21.15.188:0/1672192646 learned_addr learned my addr 172.21.15.188:0/1672192646 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-02-18T11:35:03.730 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:35:03.732+0000 7fd79cc2e700 1 --2- 172.21.15.188:0/1672192646 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd798103940 0x7fd798101490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:35:03.730 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:35:03.732+0000 7fd797fff700 1 -- 172.21.15.188:0/1672192646 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd798103940 msgr2=0x7fd798101490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:35:03.730 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:35:03.732+0000 7fd797fff700 1 --2- 172.21.15.188:0/1672192646 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd798103940 0x7fd798101490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:35:03.730 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:35:03.732+0000 7fd7977fe700 1 --2- 172.21.15.188:0/1672192646 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd798102fa0 0x7fd798100f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:35:03.730 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:35:03.732+0000 7fd797fff700 1 -- 172.21.15.188:0/1672192646 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd798102fa0 msgr2=0x7fd798100f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:35:03.730 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:35:03.732+0000 7fd797fff700 1 --2- 172.21.15.188:0/1672192646 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd798102fa0 0x7fd798100f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:35:03.730 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:35:03.732+0000 7fd797fff700 1 -- 172.21.15.188:0/1672192646 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd790009d00 con 0x7fd7980fffc0 2024-02-18T11:35:03.730 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:35:03.732+0000 7fd79cc2e700 1 --2- 172.21.15.188:0/1672192646 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd798103940 0x7fd798101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:35:03.730 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:35:03.732+0000 7fd797fff700 1 --2- 172.21.15.188:0/1672192646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7980fffc0 0x7fd798102900 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd78800c990 tx=0x7fd78800cd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:35:03.731 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:35:03.732+0000 7fd7957fa700 1 -- 172.21.15.188:0/1672192646 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd788007b00 con 0x7fd7980fffc0 2024-02-18T11:35:03.731 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:35:03.732+0000 7fd7957fa700 1 -- 172.21.15.188:0/1672192646 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd788007c60 con 0x7fd7980fffc0 2024-02-18T11:35:03.731 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:35:03.732+0000 7fd79e691700 1 -- 172.21.15.188:0/1672192646 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7981a1fa0 con 0x7fd7980fffc0 2024-02-18T11:35:03.731 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:35:03.732+0000 7fd7957fa700 1 -- 172.21.15.188:0/1672192646 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd788018660 con 0x7fd7980fffc0 2024-02-18T11:35:03.731 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:35:03.732+0000 7fd79e691700 1 -- 172.21.15.188:0/1672192646 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7981a2410 con 0x7fd7980fffc0 2024-02-18T11:35:03.731 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:35:03.733+0000 7fd79e691700 1 -- 172.21.15.188:0/1672192646 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd798109090 con 0x7fd7980fffc0 2024-02-18T11:35:03.732 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:35:03.735+0000 7fd7957fa700 1 -- 172.21.15.188:0/1672192646 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 16) v1 ==== 137245+0+0 (secure 0 0 0) 0x7fd788004020 con 0x7fd7980fffc0 2024-02-18T11:35:03.732 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:35:03.736+0000 7fd7957fa700 1 --2- 172.21.15.188:0/1672192646 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fd7800a27e0 0x7fd7800a4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:35:03.732 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:35:03.736+0000 7fd7957fa700 1 -- 172.21.15.188:0/1672192646 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4101+0+0 (secure 0 0 0) 0x7fd7880cd690 con 0x7fd7980fffc0 2024-02-18T11:35:03.732 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:35:03.736+0000 7fd7977fe700 1 --2- 172.21.15.188:0/1672192646 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fd7800a27e0 0x7fd7800a4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:35:03.732 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:35:03.736+0000 7fd7977fe700 1 --2- 172.21.15.188:0/1672192646 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fd7800a27e0 0x7fd7800a4ca0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd78c005950 tx=0x7fd78c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:35:03.734 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:35:03.739+0000 7fd7957fa700 1 -- 172.21.15.188:0/1672192646 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd788097490 con 0x7fd7980fffc0 2024-02-18T11:35:03.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:03 smithi188 ceph-mon[36925]: Detected new or changed devices on smithi158 2024-02-18T11:35:03.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:03 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:03.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:03 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:03.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:03 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:35:03.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:03 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:35:03.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:03 smithi188 ceph-mon[36925]: Adjusting osd_memory_target on smithi158 to 8540M 2024-02-18T11:35:03.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:03 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:03.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:03 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:35:03.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:03 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:35:03.856 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:03 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:03.856 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:03 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:35:03.856 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:03 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:35:03.856 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:03 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:35:03.856 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:03 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:03.856 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:03 smithi188 ceph-mon[36925]: osdmap e36: 5 total, 5 up, 5 in 2024-02-18T11:35:03.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:03 smithi012 ceph-mon[32460]: Detected new or changed devices on smithi158 2024-02-18T11:35:03.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:03 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:03.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:03 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:03.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:03 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:35:03.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:03 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:35:03.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:03 smithi012 ceph-mon[32460]: Adjusting osd_memory_target on smithi158 to 8540M 2024-02-18T11:35:03.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:03 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:03.925 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:03 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:35:03.925 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:03 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:35:03.925 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:03 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:03.925 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:03 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:35:03.925 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:03 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:35:03.925 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:03 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:35:03.925 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:03 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:03.925 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:03 smithi012 ceph-mon[32460]: osdmap e36: 5 total, 5 up, 5 in 2024-02-18T11:35:03.934 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:35:03.939+0000 7fd79e691700 1 -- 172.21.15.188:0/1672192646 --> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fd79802d9d0 con 0x7fd7800a27e0 2024-02-18T11:35:03.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:03 smithi158 ceph-mon[36626]: Detected new or changed devices on smithi158 2024-02-18T11:35:03.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:03 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:03.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:03 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:03.996 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:03 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:35:03.996 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:03 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:35:03.996 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:03 smithi158 ceph-mon[36626]: Adjusting osd_memory_target on smithi158 to 8540M 2024-02-18T11:35:03.996 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:03 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:03.996 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:03 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:35:03.996 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:03 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:35:03.996 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:03 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:03.996 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:03 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:35:03.996 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:03 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:35:03.996 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:03 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:35:03.996 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:03 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:03.997 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:03 smithi158 ceph-mon[36626]: osdmap e36: 5 total, 5 up, 5 in 2024-02-18T11:35:05.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:04 smithi188 ceph-mon[36925]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-18T11:35:05.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:04 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T11:35:05.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:04 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T11:35:05.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:04 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:35:05.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:04 smithi012 ceph-mon[32460]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-18T11:35:05.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:04 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T11:35:05.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:04 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T11:35:05.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:04 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:35:05.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:04 smithi158 ceph-mon[36626]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-18T11:35:05.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:04 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T11:35:05.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:04 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T11:35:05.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:04 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:35:06.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:05 smithi188 ceph-mon[36925]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:35:06.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:05 smithi012 ceph-mon[32460]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:35:06.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:05 smithi158 ceph-mon[36626]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:35:07.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:06 smithi188 ceph-mon[36925]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-18T11:35:07.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:06 smithi188 ceph-mon[36925]: from='client.? 172.21.15.188:0/1714570860' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d9c2e0c8-85f5-4ae2-bf3e-125c1be3a2db"}]: dispatch 2024-02-18T11:35:07.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:06 smithi188 ceph-mon[36925]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d9c2e0c8-85f5-4ae2-bf3e-125c1be3a2db"}]: dispatch 2024-02-18T11:35:07.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:06 smithi188 ceph-mon[36925]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d9c2e0c8-85f5-4ae2-bf3e-125c1be3a2db"}]': finished 2024-02-18T11:35:07.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:06 smithi188 ceph-mon[36925]: osdmap e37: 6 total, 5 up, 6 in 2024-02-18T11:35:07.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:06 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-18T11:35:07.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:06 smithi012 ceph-mon[32460]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-18T11:35:07.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:06 smithi012 ceph-mon[32460]: from='client.? 172.21.15.188:0/1714570860' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d9c2e0c8-85f5-4ae2-bf3e-125c1be3a2db"}]: dispatch 2024-02-18T11:35:07.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:06 smithi012 ceph-mon[32460]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d9c2e0c8-85f5-4ae2-bf3e-125c1be3a2db"}]: dispatch 2024-02-18T11:35:07.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:06 smithi012 ceph-mon[32460]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d9c2e0c8-85f5-4ae2-bf3e-125c1be3a2db"}]': finished 2024-02-18T11:35:07.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:06 smithi012 ceph-mon[32460]: osdmap e37: 6 total, 5 up, 6 in 2024-02-18T11:35:07.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:06 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-18T11:35:07.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:06 smithi158 ceph-mon[36626]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-18T11:35:07.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:06 smithi158 ceph-mon[36626]: from='client.? 172.21.15.188:0/1714570860' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d9c2e0c8-85f5-4ae2-bf3e-125c1be3a2db"}]: dispatch 2024-02-18T11:35:07.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:06 smithi158 ceph-mon[36626]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d9c2e0c8-85f5-4ae2-bf3e-125c1be3a2db"}]: dispatch 2024-02-18T11:35:07.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:06 smithi158 ceph-mon[36626]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d9c2e0c8-85f5-4ae2-bf3e-125c1be3a2db"}]': finished 2024-02-18T11:35:07.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:06 smithi158 ceph-mon[36626]: osdmap e37: 6 total, 5 up, 6 in 2024-02-18T11:35:07.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:06 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-18T11:35:08.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:07 smithi012 ceph-mon[32460]: from='client.? 172.21.15.188:0/403127018' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T11:35:08.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:07 smithi158 ceph-mon[36626]: from='client.? 172.21.15.188:0/403127018' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T11:35:08.355 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:07 smithi188 ceph-mon[36925]: from='client.? 172.21.15.188:0/403127018' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T11:35:09.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:08 smithi012 ceph-mon[32460]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-18T11:35:09.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:08 smithi158 ceph-mon[36626]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-18T11:35:09.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:08 smithi188 ceph-mon[36925]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-18T11:35:11.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:10 smithi188 ceph-mon[36925]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-18T11:35:11.176 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:10 smithi012 ceph-mon[32460]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-18T11:35:11.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:10 smithi158 ceph-mon[36626]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-18T11:35:12.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:12 smithi158 ceph-mon[36626]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-18T11:35:13.031 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:12 smithi188 ceph-mon[36925]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-18T11:35:13.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:12 smithi012 ceph-mon[32460]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-18T11:35:15.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:14 smithi188 ceph-mon[36925]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-18T11:35:15.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:14 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-18T11:35:15.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:14 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:35:15.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:14 smithi012 ceph-mon[32460]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-18T11:35:15.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:14 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-18T11:35:15.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:14 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:35:15.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:14 smithi158 ceph-mon[36626]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-18T11:35:15.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:14 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-18T11:35:15.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:14 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:35:16.009 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:15 smithi188 ceph-mon[36925]: Deploying daemon osd.5 on smithi188 2024-02-18T11:35:16.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:15 smithi012 ceph-mon[32460]: Deploying daemon osd.5 on smithi188 2024-02-18T11:35:16.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:15 smithi158 ceph-mon[36626]: Deploying daemon osd.5 on smithi188 2024-02-18T11:35:17.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:16 smithi188 ceph-mon[36925]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-18T11:35:17.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:16 smithi012 ceph-mon[32460]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-18T11:35:17.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:16 smithi158 ceph-mon[36626]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-18T11:35:19.056 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:18 smithi188 ceph-mon[36925]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-18T11:35:19.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:18 smithi012 ceph-mon[32460]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-18T11:35:19.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:18 smithi158 ceph-mon[36626]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-18T11:35:20.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:20 smithi188 ceph-mon[36925]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-18T11:35:20.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:20 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:20.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:20 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:35:20.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:20 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:20.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:20 smithi012 ceph-mon[32460]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-18T11:35:20.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:20 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:20.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:20 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:35:20.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:20 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:20.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:20 smithi158 ceph-mon[36626]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-18T11:35:20.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:20 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:20.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:20 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:35:20.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:20 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:22.735 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:22 smithi188 ceph-mon[36925]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-18T11:35:22.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:22 smithi012 ceph-mon[32460]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-18T11:35:22.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:22 smithi158 ceph-mon[36626]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-18T11:35:23.054 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:35:23.058+0000 7fd7957fa700 1 -- 172.21.15.188:0/1672192646 <== mgr.14152 v2:172.21.15.12:6800/1153736685 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fd79802d9d0 con 0x7fd7800a27e0 2024-02-18T11:35:23.054 INFO:teuthology.orchestra.run.smithi188.stdout:Created osd(s) 5 on host 'smithi188' 2024-02-18T11:35:23.056 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:35:23.060+0000 7fd79e691700 1 -- 172.21.15.188:0/1672192646 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fd7800a27e0 msgr2=0x7fd7800a4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:35:23.056 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:35:23.060+0000 7fd79e691700 1 --2- 172.21.15.188:0/1672192646 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fd7800a27e0 0x7fd7800a4ca0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd78c005950 tx=0x7fd78c0058e0 comp rx=0 tx=0).stop 2024-02-18T11:35:23.056 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:35:23.060+0000 7fd79e691700 1 -- 172.21.15.188:0/1672192646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7980fffc0 msgr2=0x7fd798102900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:35:23.056 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:35:23.060+0000 7fd79e691700 1 --2- 172.21.15.188:0/1672192646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7980fffc0 0x7fd798102900 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd78800c990 tx=0x7fd78800cd50 comp rx=0 tx=0).stop 2024-02-18T11:35:23.056 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:35:23.060+0000 7fd79e691700 1 -- 172.21.15.188:0/1672192646 shutdown_connections 2024-02-18T11:35:23.056 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:35:23.060+0000 7fd79e691700 1 --2- 172.21.15.188:0/1672192646 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd798103940 0x7fd798101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:35:23.056 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:35:23.060+0000 7fd79e691700 1 --2- 172.21.15.188:0/1672192646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7980fffc0 0x7fd798102900 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:35:23.056 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:35:23.060+0000 7fd79e691700 1 --2- 172.21.15.188:0/1672192646 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fd7800a27e0 0x7fd7800a4ca0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:35:23.057 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:35:23.060+0000 7fd79e691700 1 --2- 172.21.15.188:0/1672192646 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd798102fa0 0x7fd798100f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:35:23.057 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:35:23.060+0000 7fd79e691700 1 -- 172.21.15.188:0/1672192646 >> 172.21.15.188:0/1672192646 conn(0x7fd7980fb940 msgr2=0x7fd7981061d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:35:23.057 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:35:23.060+0000 7fd79e691700 1 -- 172.21.15.188:0/1672192646 shutdown_connections 2024-02-18T11:35:23.057 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:35:23.060+0000 7fd79e691700 1 -- 172.21.15.188:0/1672192646 wait complete. 2024-02-18T11:35:23.693 DEBUG:teuthology.orchestra.run.smithi188:osd.5> sudo journalctl -f -n 0 -u ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@osd.5.service 2024-02-18T11:35:23.695 INFO:tasks.cephadm:Deploying osd.6 on smithi012 with /dev/vg_nvme/lv_2... 2024-02-18T11:35:23.695 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-02-18T11:35:23.947 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T11:35:24.355 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:24 smithi188 ceph-mon[36925]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-18T11:35:24.355 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:24 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:24.355 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:24 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:24.355 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:24 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:24.355 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:24 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:24.355 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:24 smithi188 ceph-mon[36925]: from='osd.5 [v2:172.21.15.188:6808/3939495896,v1:172.21.15.188:6809/3939495896]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-18T11:35:24.355 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:24 smithi188 ceph-mon[36925]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-18T11:35:24.365 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:24 smithi012 ceph-mon[32460]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-18T11:35:24.365 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:24 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:24.365 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:24 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:24.365 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:24 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:24.365 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:24 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:24.365 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:24 smithi012 ceph-mon[32460]: from='osd.5 [v2:172.21.15.188:6808/3939495896,v1:172.21.15.188:6809/3939495896]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-18T11:35:24.366 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:24 smithi012 ceph-mon[32460]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-18T11:35:24.495 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:24 smithi158 ceph-mon[36626]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-18T11:35:24.495 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:24 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:24.495 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:24 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:24.495 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:24 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:24.495 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:24 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:24.495 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:24 smithi158 ceph-mon[36626]: from='osd.5 [v2:172.21.15.188:6808/3939495896,v1:172.21.15.188:6809/3939495896]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-18T11:35:24.495 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:24 smithi158 ceph-mon[36626]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-18T11:35:25.423 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:25 smithi012 ceph-mon[32460]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-18T11:35:25.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:25 smithi012 ceph-mon[32460]: osdmap e38: 6 total, 5 up, 6 in 2024-02-18T11:35:25.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:25 smithi012 ceph-mon[32460]: from='osd.5 [v2:172.21.15.188:6808/3939495896,v1:172.21.15.188:6809/3939495896]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-02-18T11:35:25.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:25 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-18T11:35:25.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:25 smithi012 ceph-mon[32460]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-02-18T11:35:25.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:25 smithi188 ceph-mon[36925]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-18T11:35:25.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:25 smithi188 ceph-mon[36925]: osdmap e38: 6 total, 5 up, 6 in 2024-02-18T11:35:25.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:25 smithi188 ceph-mon[36925]: from='osd.5 [v2:172.21.15.188:6808/3939495896,v1:172.21.15.188:6809/3939495896]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-02-18T11:35:25.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:25 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-18T11:35:25.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:25 smithi188 ceph-mon[36925]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-02-18T11:35:25.727 INFO:teuthology.orchestra.run.smithi012.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-02-18T11:35:25.727 INFO:teuthology.orchestra.run.smithi012.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-02-18T11:35:25.727 INFO:teuthology.orchestra.run.smithi012.stderr: stderr: 10+0 records in 2024-02-18T11:35:25.727 INFO:teuthology.orchestra.run.smithi012.stderr:10+0 records out 2024-02-18T11:35:25.727 INFO:teuthology.orchestra.run.smithi012.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114768 s, 914 MB/s 2024-02-18T11:35:25.728 INFO:teuthology.orchestra.run.smithi012.stderr:--> Zapping successful for: 2024-02-18T11:35:25.745 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:25 smithi158 ceph-mon[36626]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-18T11:35:25.745 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:25 smithi158 ceph-mon[36626]: osdmap e38: 6 total, 5 up, 6 in 2024-02-18T11:35:25.745 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:25 smithi158 ceph-mon[36626]: from='osd.5 [v2:172.21.15.188:6808/3939495896,v1:172.21.15.188:6809/3939495896]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-02-18T11:35:25.745 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:25 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-18T11:35:25.745 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:25 smithi158 ceph-mon[36626]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-02-18T11:35:26.314 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi012:vg_nvme/lv_2 2024-02-18T11:35:26.562 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T11:35:26.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:26 smithi188 ceph-mon[36925]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-18T11:35:26.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:26 smithi188 ceph-mon[36925]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-02-18T11:35:26.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:26 smithi188 ceph-mon[36925]: osdmap e39: 6 total, 5 up, 6 in 2024-02-18T11:35:26.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:26 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-18T11:35:26.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:26 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-18T11:35:26.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:26 smithi188 ceph-mon[36925]: from='osd.5 ' entity='osd.5' 2024-02-18T11:35:26.605 INFO:journalctl@ceph.osd.5.smithi188.stdout:Feb 18 11:35:26 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-5[48668]: 2024-02-18T11:35:26.189+0000 7f7799573700 -1 osd.5 0 waiting for initial osdmap 2024-02-18T11:35:26.605 INFO:journalctl@ceph.osd.5.smithi188.stdout:Feb 18 11:35:26 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-5[48668]: 2024-02-18T11:35:26.199+0000 7f779035c700 -1 osd.5 39 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-18T11:35:26.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:26 smithi012 ceph-mon[32460]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-18T11:35:26.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:26 smithi012 ceph-mon[32460]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-02-18T11:35:26.668 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:26 smithi012 ceph-mon[32460]: osdmap e39: 6 total, 5 up, 6 in 2024-02-18T11:35:26.668 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:26 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-18T11:35:26.668 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:26 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-18T11:35:26.668 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:26 smithi012 ceph-mon[32460]: from='osd.5 ' entity='osd.5' 2024-02-18T11:35:26.745 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:26 smithi158 ceph-mon[36626]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-18T11:35:26.745 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:26 smithi158 ceph-mon[36626]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-02-18T11:35:26.745 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:26 smithi158 ceph-mon[36626]: osdmap e39: 6 total, 5 up, 6 in 2024-02-18T11:35:26.746 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:26 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-18T11:35:26.746 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:26 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-18T11:35:26.746 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:26 smithi158 ceph-mon[36626]: from='osd.5 ' entity='osd.5' 2024-02-18T11:35:27.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:27 smithi012 ceph-mon[32460]: purged_snaps scrub starts 2024-02-18T11:35:27.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:27 smithi012 ceph-mon[32460]: purged_snaps scrub ok 2024-02-18T11:35:27.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:27 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-18T11:35:27.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:27 smithi012 ceph-mon[32460]: osd.5 [v2:172.21.15.188:6808/3939495896,v1:172.21.15.188:6809/3939495896] boot 2024-02-18T11:35:27.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:27 smithi012 ceph-mon[32460]: osdmap e40: 6 total, 6 up, 6 in 2024-02-18T11:35:27.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:27 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-18T11:35:27.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:27 smithi012 ceph-mon[32460]: osdmap e41: 6 total, 6 up, 6 in 2024-02-18T11:35:27.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:27 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:27.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:27 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:27.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:27 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:35:27.675 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:27 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:35:27.675 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:27 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:27.675 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:27 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:35:27.675 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:27 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:35:27.675 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:27 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:27.675 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:27 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:35:27.675 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:27 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:35:27.675 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:27 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:35:27.675 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:27 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:27.745 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:27 smithi158 ceph-mon[36626]: purged_snaps scrub starts 2024-02-18T11:35:27.745 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:27 smithi158 ceph-mon[36626]: purged_snaps scrub ok 2024-02-18T11:35:27.745 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:27 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-18T11:35:27.745 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:27 smithi158 ceph-mon[36626]: osd.5 [v2:172.21.15.188:6808/3939495896,v1:172.21.15.188:6809/3939495896] boot 2024-02-18T11:35:27.745 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:27 smithi158 ceph-mon[36626]: osdmap e40: 6 total, 6 up, 6 in 2024-02-18T11:35:27.745 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:27 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-18T11:35:27.746 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:27 smithi158 ceph-mon[36626]: osdmap e41: 6 total, 6 up, 6 in 2024-02-18T11:35:27.746 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:27 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:27.746 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:27 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:27.746 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:27 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:35:27.746 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:27 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:35:27.746 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:27 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:27.746 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:27 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:35:27.746 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:27 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:35:27.746 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:27 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:27.746 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:27 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:35:27.746 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:27 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:35:27.746 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:27 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:35:27.746 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:27 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:27.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:27 smithi188 ceph-mon[36925]: purged_snaps scrub starts 2024-02-18T11:35:27.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:27 smithi188 ceph-mon[36925]: purged_snaps scrub ok 2024-02-18T11:35:27.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:27 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-18T11:35:27.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:27 smithi188 ceph-mon[36925]: osd.5 [v2:172.21.15.188:6808/3939495896,v1:172.21.15.188:6809/3939495896] boot 2024-02-18T11:35:27.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:27 smithi188 ceph-mon[36925]: osdmap e40: 6 total, 6 up, 6 in 2024-02-18T11:35:27.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:27 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-18T11:35:27.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:27 smithi188 ceph-mon[36925]: osdmap e41: 6 total, 6 up, 6 in 2024-02-18T11:35:27.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:27 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:27.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:27 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:27.856 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:27 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:35:27.856 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:27 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:35:27.856 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:27 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:27.856 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:27 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:35:27.856 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:27 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:35:27.856 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:27 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:27.856 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:27 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:35:27.856 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:27 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:35:27.856 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:27 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:35:27.856 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:27 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:28.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:35:28.245+0000 7fb168781700 1 -- 172.21.15.12:0/2492299268 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb160102f80 msgr2=0x7fb1601033e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:35:28.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:35:28.245+0000 7fb168781700 1 --2- 172.21.15.12:0/2492299268 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb160102f80 0x7fb1601033e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb154009990 tx=0x7fb154009ca0 comp rx=0 tx=0).stop 2024-02-18T11:35:28.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:35:28.245+0000 7fb168781700 1 -- 172.21.15.12:0/2492299268 shutdown_connections 2024-02-18T11:35:28.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:35:28.245+0000 7fb168781700 1 --2- 172.21.15.12:0/2492299268 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb160103920 0x7fb16010a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:35:28.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:35:28.245+0000 7fb168781700 1 --2- 172.21.15.12:0/2492299268 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb1600fffa0 0x7fb1601003c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:35:28.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:35:28.245+0000 7fb168781700 1 --2- 172.21.15.12:0/2492299268 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb160102f80 0x7fb1601033e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:35:28.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:35:28.245+0000 7fb168781700 1 -- 172.21.15.12:0/2492299268 >> 172.21.15.12:0/2492299268 conn(0x7fb1600fb920 msgr2=0x7fb1600fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:35:28.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:35:28.245+0000 7fb168781700 1 -- 172.21.15.12:0/2492299268 shutdown_connections 2024-02-18T11:35:28.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:35:28.245+0000 7fb168781700 1 -- 172.21.15.12:0/2492299268 wait complete. 2024-02-18T11:35:28.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:35:28.246+0000 7fb168781700 1 Processor -- start 2024-02-18T11:35:28.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:35:28.246+0000 7fb168781700 1 -- start start 2024-02-18T11:35:28.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:35:28.246+0000 7fb168781700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb1600fffa0 0x7fb160197d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:35:28.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:35:28.246+0000 7fb168781700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb160102f80 0x7fb160198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:35:28.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:35:28.246+0000 7fb168781700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb160103920 0x7fb16019f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:35:28.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:35:28.246+0000 7fb168781700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb16019f8c0 con 0x7fb160102f80 2024-02-18T11:35:28.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:35:28.246+0000 7fb168781700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb16019fa00 con 0x7fb1600fffa0 2024-02-18T11:35:28.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:35:28.246+0000 7fb168781700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb16019fb90 con 0x7fb160103920 2024-02-18T11:35:28.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:35:28.246+0000 7fb165d1c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb160102f80 0x7fb160198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:35:28.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:35:28.246+0000 7fb165d1c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb160102f80 0x7fb160198270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35772/0 (socket says 172.21.15.12:35772) 2024-02-18T11:35:28.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:35:28.246+0000 7fb165d1c700 1 -- 172.21.15.12:0/2344895221 learned_addr learned my addr 172.21.15.12:0/2344895221 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:35:28.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:35:28.246+0000 7fb16651d700 1 --2- 172.21.15.12:0/2344895221 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb1600fffa0 0x7fb160197d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:35:28.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:35:28.246+0000 7fb166d1e700 1 --2- 172.21.15.12:0/2344895221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb160103920 0x7fb16019f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:35:28.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:35:28.246+0000 7fb165d1c700 1 -- 172.21.15.12:0/2344895221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb160103920 msgr2=0x7fb16019f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:35:28.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:35:28.246+0000 7fb165d1c700 1 --2- 172.21.15.12:0/2344895221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb160103920 0x7fb16019f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:35:28.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:35:28.246+0000 7fb165d1c700 1 -- 172.21.15.12:0/2344895221 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb1600fffa0 msgr2=0x7fb160197d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:35:28.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:35:28.246+0000 7fb165d1c700 1 --2- 172.21.15.12:0/2344895221 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb1600fffa0 0x7fb160197d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:35:28.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:35:28.246+0000 7fb165d1c700 1 -- 172.21.15.12:0/2344895221 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1540095f0 con 0x7fb160102f80 2024-02-18T11:35:28.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:35:28.246+0000 7fb165d1c700 1 --2- 172.21.15.12:0/2344895221 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb160102f80 0x7fb160198270 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb154000c00 tx=0x7fb15400f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:35:28.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:35:28.247+0000 7fb15f7fe700 1 -- 172.21.15.12:0/2344895221 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb15401d070 con 0x7fb160102f80 2024-02-18T11:35:28.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:35:28.247+0000 7fb15f7fe700 1 -- 172.21.15.12:0/2344895221 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb15400fcb0 con 0x7fb160102f80 2024-02-18T11:35:28.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:35:28.247+0000 7fb15f7fe700 1 -- 172.21.15.12:0/2344895221 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb154004d40 con 0x7fb160102f80 2024-02-18T11:35:28.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:35:28.247+0000 7fb168781700 1 -- 172.21.15.12:0/2344895221 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb16019fe10 con 0x7fb160102f80 2024-02-18T11:35:28.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:35:28.247+0000 7fb168781700 1 -- 172.21.15.12:0/2344895221 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1601a02a0 con 0x7fb160102f80 2024-02-18T11:35:28.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:35:28.248+0000 7fb168781700 1 -- 172.21.15.12:0/2344895221 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb160068970 con 0x7fb160102f80 2024-02-18T11:35:28.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:35:28.249+0000 7fb15f7fe700 1 -- 172.21.15.12:0/2344895221 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) v1 ==== 137245+0+0 (secure 0 0 0) 0x7fb154005050 con 0x7fb160102f80 2024-02-18T11:35:28.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:35:28.249+0000 7fb15f7fe700 1 --2- 172.21.15.12:0/2344895221 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fb1480a2860 0x7fb1480a4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:35:28.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:35:28.249+0000 7fb15f7fe700 1 -- 172.21.15.12:0/2344895221 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4562+0+0 (secure 0 0 0) 0x7fb1540ce7e0 con 0x7fb160102f80 2024-02-18T11:35:28.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:35:28.249+0000 7fb16651d700 1 --2- 172.21.15.12:0/2344895221 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fb1480a2860 0x7fb1480a4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:35:28.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:35:28.250+0000 7fb16651d700 1 --2- 172.21.15.12:0/2344895221 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fb1480a2860 0x7fb1480a4d20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb160069760 tx=0x7fb1580094f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:35:28.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:35:28.254+0000 7fb15f7fe700 1 -- 172.21.15.12:0/2344895221 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb1540984a0 con 0x7fb160102f80 2024-02-18T11:35:28.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:35:28.468+0000 7fb168781700 1 -- 172.21.15.12:0/2344895221 --> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fb16002d950 con 0x7fb1480a2860 2024-02-18T11:35:28.856 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:28 smithi188 ceph-mon[36925]: Detected new or changed devices on smithi188 2024-02-18T11:35:28.856 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:28 smithi188 ceph-mon[36925]: Adjusting osd_memory_target on smithi188 to 8540M 2024-02-18T11:35:28.856 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:28 smithi188 ceph-mon[36925]: pgmap v139: 1 pgs: 1 peering; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-18T11:35:28.856 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:28 smithi188 ceph-mon[36925]: osdmap e42: 6 total, 6 up, 6 in 2024-02-18T11:35:28.857 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:28 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T11:35:28.857 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:28 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T11:35:28.857 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:28 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:35:28.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:28 smithi012 ceph-mon[32460]: Detected new or changed devices on smithi188 2024-02-18T11:35:28.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:28 smithi012 ceph-mon[32460]: Adjusting osd_memory_target on smithi188 to 8540M 2024-02-18T11:35:28.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:28 smithi012 ceph-mon[32460]: pgmap v139: 1 pgs: 1 peering; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-18T11:35:28.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:28 smithi012 ceph-mon[32460]: osdmap e42: 6 total, 6 up, 6 in 2024-02-18T11:35:28.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:28 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T11:35:28.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:28 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T11:35:28.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:28 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:35:28.997 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:28 smithi158 ceph-mon[36626]: Detected new or changed devices on smithi188 2024-02-18T11:35:28.997 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:28 smithi158 ceph-mon[36626]: Adjusting osd_memory_target on smithi188 to 8540M 2024-02-18T11:35:28.997 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:28 smithi158 ceph-mon[36626]: pgmap v139: 1 pgs: 1 peering; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-18T11:35:28.997 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:28 smithi158 ceph-mon[36626]: osdmap e42: 6 total, 6 up, 6 in 2024-02-18T11:35:28.997 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:28 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T11:35:28.998 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:28 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T11:35:28.998 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:28 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:35:29.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:29 smithi012 ceph-mon[32460]: from='client.14352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:35:29.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:29 smithi012 ceph-mon[32460]: osdmap e43: 6 total, 6 up, 6 in 2024-02-18T11:35:29.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:29 smithi012 ceph-mon[32460]: osdmap e44: 6 total, 6 up, 6 in 2024-02-18T11:35:29.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:29 smithi188 ceph-mon[36925]: from='client.14352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:35:29.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:29 smithi188 ceph-mon[36925]: osdmap e43: 6 total, 6 up, 6 in 2024-02-18T11:35:29.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:29 smithi188 ceph-mon[36925]: osdmap e44: 6 total, 6 up, 6 in 2024-02-18T11:35:29.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:29 smithi158 ceph-mon[36626]: from='client.14352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:35:29.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:29 smithi158 ceph-mon[36626]: osdmap e43: 6 total, 6 up, 6 in 2024-02-18T11:35:29.996 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:29 smithi158 ceph-mon[36626]: osdmap e44: 6 total, 6 up, 6 in 2024-02-18T11:35:30.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:30 smithi188 ceph-mon[36925]: pgmap v142: 1 pgs: 1 peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-02-18T11:35:30.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:30 smithi012 ceph-mon[32460]: pgmap v142: 1 pgs: 1 peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-02-18T11:35:30.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:30 smithi158 ceph-mon[36626]: pgmap v142: 1 pgs: 1 peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-02-18T11:35:31.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:31 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/1264800617' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4a523512-de6b-49f0-be81-00f2f00c1c10"}]: dispatch 2024-02-18T11:35:31.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:31 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/1264800617' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4a523512-de6b-49f0-be81-00f2f00c1c10"}]': finished 2024-02-18T11:35:31.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:31 smithi188 ceph-mon[36925]: osdmap e45: 7 total, 6 up, 7 in 2024-02-18T11:35:31.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:31 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-18T11:35:31.896 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:31 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/1264800617' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4a523512-de6b-49f0-be81-00f2f00c1c10"}]: dispatch 2024-02-18T11:35:31.896 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:31 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/1264800617' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4a523512-de6b-49f0-be81-00f2f00c1c10"}]': finished 2024-02-18T11:35:31.896 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:31 smithi158 ceph-mon[36626]: osdmap e45: 7 total, 6 up, 7 in 2024-02-18T11:35:31.897 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:31 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-18T11:35:31.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:31 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/1264800617' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4a523512-de6b-49f0-be81-00f2f00c1c10"}]: dispatch 2024-02-18T11:35:31.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:31 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/1264800617' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4a523512-de6b-49f0-be81-00f2f00c1c10"}]': finished 2024-02-18T11:35:31.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:31 smithi012 ceph-mon[32460]: osdmap e45: 7 total, 6 up, 7 in 2024-02-18T11:35:31.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:31 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-18T11:35:32.771 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:32 smithi188 ceph-mon[36925]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 129 KiB/s, 0 objects/s recovering 2024-02-18T11:35:32.771 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:32 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/400638523' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T11:35:32.909 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:32 smithi158 ceph-mon[36626]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 129 KiB/s, 0 objects/s recovering 2024-02-18T11:35:32.910 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:32 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/400638523' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T11:35:32.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:32 smithi012 ceph-mon[32460]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 129 KiB/s, 0 objects/s recovering 2024-02-18T11:35:32.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:32 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/400638523' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T11:35:34.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:34 smithi188 ceph-mon[36925]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-02-18T11:35:34.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:34 smithi012 ceph-mon[32460]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-02-18T11:35:34.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:34 smithi158 ceph-mon[36626]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-02-18T11:35:36.620 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:36 smithi012 ceph-mon[32460]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-02-18T11:35:36.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:36 smithi188 ceph-mon[36925]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-02-18T11:35:36.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:36 smithi158 ceph-mon[36626]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-02-18T11:35:38.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:38 smithi188 ceph-mon[36925]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-18T11:35:38.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:38 smithi012 ceph-mon[32460]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-18T11:35:38.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:38 smithi158 ceph-mon[36626]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-18T11:35:39.770 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:39 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-02-18T11:35:39.770 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:39 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:35:39.770 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:39 smithi012 ceph-mon[32460]: Deploying daemon osd.6 on smithi012 2024-02-18T11:35:39.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:39 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-02-18T11:35:39.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:39 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:35:39.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:39 smithi188 ceph-mon[36925]: Deploying daemon osd.6 on smithi012 2024-02-18T11:35:39.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:39 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-02-18T11:35:39.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:39 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:35:39.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:39 smithi158 ceph-mon[36626]: Deploying daemon osd.6 on smithi012 2024-02-18T11:35:40.800 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:40 smithi012 ceph-mon[32460]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-02-18T11:35:40.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:40 smithi188 ceph-mon[36925]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-02-18T11:35:40.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:40 smithi158 ceph-mon[36626]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-02-18T11:35:42.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:42 smithi188 ceph-mon[36925]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-18T11:35:42.900 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:42 smithi012 ceph-mon[32460]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-18T11:35:42.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:42 smithi158 ceph-mon[36626]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-18T11:35:44.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:44 smithi012 ceph-mon[32460]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-18T11:35:44.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:44 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:44.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:44 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:44.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:44 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:35:44.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:44 smithi188 ceph-mon[36925]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-18T11:35:44.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:44 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:44.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:44 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:44.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:44 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:35:44.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:44 smithi158 ceph-mon[36626]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-18T11:35:44.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:44 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:44.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:44 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:44.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:44 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:35:46.850 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:46 smithi012 ceph-mon[32460]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-18T11:35:46.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:46 smithi188 ceph-mon[36925]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-18T11:35:46.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:46 smithi158 ceph-mon[36626]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-18T11:35:47.481 INFO:teuthology.orchestra.run.smithi012.stdout:Created osd(s) 6 on host 'smithi012' 2024-02-18T11:35:47.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:35:47.486+0000 7fb15f7fe700 1 -- 172.21.15.12:0/2344895221 <== mgr.14152 v2:172.21.15.12:6800/1153736685 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fb16002d950 con 0x7fb1480a2860 2024-02-18T11:35:47.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:35:47.488+0000 7fb168781700 1 -- 172.21.15.12:0/2344895221 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fb1480a2860 msgr2=0x7fb1480a4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:35:47.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:35:47.488+0000 7fb168781700 1 --2- 172.21.15.12:0/2344895221 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fb1480a2860 0x7fb1480a4d20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb160069760 tx=0x7fb1580094f0 comp rx=0 tx=0).stop 2024-02-18T11:35:47.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:35:47.488+0000 7fb168781700 1 -- 172.21.15.12:0/2344895221 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb160102f80 msgr2=0x7fb160198270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:35:47.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:35:47.488+0000 7fb168781700 1 --2- 172.21.15.12:0/2344895221 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb160102f80 0x7fb160198270 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb154000c00 tx=0x7fb15400f740 comp rx=0 tx=0).stop 2024-02-18T11:35:47.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:35:47.488+0000 7fb168781700 1 -- 172.21.15.12:0/2344895221 shutdown_connections 2024-02-18T11:35:47.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:35:47.488+0000 7fb168781700 1 --2- 172.21.15.12:0/2344895221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb160103920 0x7fb16019f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:35:47.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:35:47.488+0000 7fb168781700 1 --2- 172.21.15.12:0/2344895221 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb1600fffa0 0x7fb160197d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:35:47.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:35:47.488+0000 7fb168781700 1 --2- 172.21.15.12:0/2344895221 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fb1480a2860 0x7fb1480a4d20 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:35:47.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:35:47.488+0000 7fb168781700 1 --2- 172.21.15.12:0/2344895221 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb160102f80 0x7fb160198270 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:35:47.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:35:47.488+0000 7fb168781700 1 -- 172.21.15.12:0/2344895221 >> 172.21.15.12:0/2344895221 conn(0x7fb1600fb920 msgr2=0x7fb160106280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:35:47.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:35:47.488+0000 7fb168781700 1 -- 172.21.15.12:0/2344895221 shutdown_connections 2024-02-18T11:35:47.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:35:47.488+0000 7fb168781700 1 -- 172.21.15.12:0/2344895221 wait complete. 2024-02-18T11:35:47.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:47 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:47.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:47 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:47.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:47 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:35:47.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:47 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:35:47.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:47 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:47.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:47 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:47.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:47 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:47.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:47 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:35:48.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:47 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:48.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:47 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:48.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:47 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:35:48.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:47 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:35:48.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:47 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:48.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:47 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:48.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:47 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:48.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:47 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:35:48.134 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:47 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:48.134 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:47 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:48.134 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:47 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:35:48.134 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:47 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:35:48.135 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:47 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:48.135 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:47 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:48.135 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:47 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:48.135 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:47 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:35:48.164 DEBUG:teuthology.orchestra.run.smithi012:osd.6> sudo journalctl -f -n 0 -u ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@osd.6.service 2024-02-18T11:35:48.166 INFO:tasks.cephadm:Deploying osd.7 on smithi158 with /dev/vg_nvme/lv_2... 2024-02-18T11:35:48.167 DEBUG:teuthology.orchestra.run.smithi158:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-02-18T11:35:48.414 INFO:teuthology.orchestra.run.smithi158.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T11:35:48.801 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:48 smithi158 ceph-mon[36626]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-18T11:35:48.801 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:48 smithi158 ceph-mon[36626]: from='osd.6 [v2:172.21.15.12:6818/1206337767,v1:172.21.15.12:6819/1206337767]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-18T11:35:49.099 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:48 smithi012 ceph-mon[32460]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-18T11:35:49.099 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:48 smithi012 ceph-mon[32460]: from='osd.6 [v2:172.21.15.12:6818/1206337767,v1:172.21.15.12:6819/1206337767]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-18T11:35:49.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:48 smithi188 ceph-mon[36925]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-18T11:35:49.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:48 smithi188 ceph-mon[36925]: from='osd.6 [v2:172.21.15.12:6818/1206337767,v1:172.21.15.12:6819/1206337767]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-18T11:35:49.833 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:49 smithi158 ceph-mon[36626]: from='osd.6 [v2:172.21.15.12:6818/1206337767,v1:172.21.15.12:6819/1206337767]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-02-18T11:35:49.833 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:49 smithi158 ceph-mon[36626]: osdmap e46: 7 total, 6 up, 7 in 2024-02-18T11:35:49.833 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:49 smithi158 ceph-mon[36626]: from='osd.6 [v2:172.21.15.12:6818/1206337767,v1:172.21.15.12:6819/1206337767]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-02-18T11:35:49.834 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:49 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-18T11:35:49.834 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:49 smithi158 ceph-mon[36626]: from='osd.6 [v2:172.21.15.12:6818/1206337767,v1:172.21.15.12:6819/1206337767]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-02-18T11:35:49.834 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:49 smithi158 ceph-mon[36626]: osdmap e47: 7 total, 6 up, 7 in 2024-02-18T11:35:49.834 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:49 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-18T11:35:50.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:49 smithi188 ceph-mon[36925]: from='osd.6 [v2:172.21.15.12:6818/1206337767,v1:172.21.15.12:6819/1206337767]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-02-18T11:35:50.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:49 smithi188 ceph-mon[36925]: osdmap e46: 7 total, 6 up, 7 in 2024-02-18T11:35:50.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:49 smithi188 ceph-mon[36925]: from='osd.6 [v2:172.21.15.12:6818/1206337767,v1:172.21.15.12:6819/1206337767]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-02-18T11:35:50.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:49 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-18T11:35:50.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:49 smithi188 ceph-mon[36925]: from='osd.6 [v2:172.21.15.12:6818/1206337767,v1:172.21.15.12:6819/1206337767]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-02-18T11:35:50.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:49 smithi188 ceph-mon[36925]: osdmap e47: 7 total, 6 up, 7 in 2024-02-18T11:35:50.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:49 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-18T11:35:50.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:49 smithi012 ceph-mon[32460]: from='osd.6 [v2:172.21.15.12:6818/1206337767,v1:172.21.15.12:6819/1206337767]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-02-18T11:35:50.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:49 smithi012 ceph-mon[32460]: osdmap e46: 7 total, 6 up, 7 in 2024-02-18T11:35:50.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:49 smithi012 ceph-mon[32460]: from='osd.6 [v2:172.21.15.12:6818/1206337767,v1:172.21.15.12:6819/1206337767]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-02-18T11:35:50.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:49 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-18T11:35:50.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:49 smithi012 ceph-mon[32460]: from='osd.6 [v2:172.21.15.12:6818/1206337767,v1:172.21.15.12:6819/1206337767]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-02-18T11:35:50.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:49 smithi012 ceph-mon[32460]: osdmap e47: 7 total, 6 up, 7 in 2024-02-18T11:35:50.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:49 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-18T11:35:50.174 INFO:journalctl@ceph.osd.6.smithi012.stdout:Feb 18 11:35:49 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-6[58474]: 2024-02-18T11:35:49.723+0000 7fcbe2b99700 -1 osd.6 0 waiting for initial osdmap 2024-02-18T11:35:50.175 INFO:journalctl@ceph.osd.6.smithi012.stdout:Feb 18 11:35:49 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-6[58474]: 2024-02-18T11:35:49.737+0000 7fcbdf18d700 -1 osd.6 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-18T11:35:50.438 INFO:teuthology.orchestra.run.smithi158.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-02-18T11:35:50.438 INFO:teuthology.orchestra.run.smithi158.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-02-18T11:35:50.438 INFO:teuthology.orchestra.run.smithi158.stderr: stderr: 10+0 records in 2024-02-18T11:35:50.439 INFO:teuthology.orchestra.run.smithi158.stderr:10+0 records out 2024-02-18T11:35:50.439 INFO:teuthology.orchestra.run.smithi158.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00856464 s, 1.2 GB/s 2024-02-18T11:35:50.439 INFO:teuthology.orchestra.run.smithi158.stderr:--> Zapping successful for: 2024-02-18T11:35:50.820 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:50 smithi158 ceph-mon[36626]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-18T11:35:50.820 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:50 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-18T11:35:50.820 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:50 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-18T11:35:50.831 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:50 smithi012 ceph-mon[32460]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-18T11:35:50.831 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:50 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-18T11:35:50.831 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:50 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-18T11:35:51.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:50 smithi188 ceph-mon[36925]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-18T11:35:51.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:50 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-18T11:35:51.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:50 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-18T11:35:51.148 DEBUG:teuthology.orchestra.run.smithi158:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi158:vg_nvme/lv_2 2024-02-18T11:35:51.394 INFO:teuthology.orchestra.run.smithi158.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T11:35:51.745 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:51 smithi158 ceph-mon[36626]: purged_snaps scrub starts 2024-02-18T11:35:51.745 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:51 smithi158 ceph-mon[36626]: purged_snaps scrub ok 2024-02-18T11:35:51.745 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:51 smithi158 ceph-mon[36626]: osd.6 [v2:172.21.15.12:6818/1206337767,v1:172.21.15.12:6819/1206337767] boot 2024-02-18T11:35:51.745 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:51 smithi158 ceph-mon[36626]: osdmap e48: 7 total, 7 up, 7 in 2024-02-18T11:35:51.745 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:51 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-18T11:35:51.745 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:51 smithi158 ceph-mon[36626]: Detected new or changed devices on smithi012 2024-02-18T11:35:51.745 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:51 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:51.745 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:51 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:51.745 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:51 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:35:51.745 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:51 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:35:51.745 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:51 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:35:51.746 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:51 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:51.746 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:51 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:35:51.746 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:51 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:35:51.746 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:51 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:51.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:51 smithi012 ceph-mon[32460]: purged_snaps scrub starts 2024-02-18T11:35:51.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:51 smithi012 ceph-mon[32460]: purged_snaps scrub ok 2024-02-18T11:35:51.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:51 smithi012 ceph-mon[32460]: osd.6 [v2:172.21.15.12:6818/1206337767,v1:172.21.15.12:6819/1206337767] boot 2024-02-18T11:35:51.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:51 smithi012 ceph-mon[32460]: osdmap e48: 7 total, 7 up, 7 in 2024-02-18T11:35:51.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:51 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-18T11:35:51.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:51 smithi012 ceph-mon[32460]: Detected new or changed devices on smithi012 2024-02-18T11:35:51.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:51 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:51.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:51 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:51.925 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:51 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:35:51.925 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:51 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:35:51.925 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:51 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:35:51.925 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:51 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:51.925 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:51 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:35:51.925 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:51 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:35:51.925 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:51 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:52.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:51 smithi188 ceph-mon[36925]: purged_snaps scrub starts 2024-02-18T11:35:52.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:51 smithi188 ceph-mon[36925]: purged_snaps scrub ok 2024-02-18T11:35:52.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:51 smithi188 ceph-mon[36925]: osd.6 [v2:172.21.15.12:6818/1206337767,v1:172.21.15.12:6819/1206337767] boot 2024-02-18T11:35:52.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:51 smithi188 ceph-mon[36925]: osdmap e48: 7 total, 7 up, 7 in 2024-02-18T11:35:52.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:51 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-18T11:35:52.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:51 smithi188 ceph-mon[36925]: Detected new or changed devices on smithi012 2024-02-18T11:35:52.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:51 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:52.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:51 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:52.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:51 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:35:52.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:51 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:35:52.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:51 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:35:52.106 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:51 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:52.106 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:51 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:35:52.106 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:51 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:35:52.106 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:51 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:35:52.767 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:52 smithi158 ceph-mon[36626]: Adjusting osd_memory_target on smithi012 to 5697M 2024-02-18T11:35:52.767 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:52 smithi158 ceph-mon[36626]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-18T11:35:52.768 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:52 smithi158 ceph-mon[36626]: osdmap e49: 7 total, 7 up, 7 in 2024-02-18T11:35:52.977 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:35:52.980+0000 7fcf0ff58700 1 -- 172.21.15.158:0/3239146058 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcf08103880 msgr2=0x7fcf0810a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:35:52.977 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:35:52.980+0000 7fcf0ff58700 1 --2- 172.21.15.158:0/3239146058 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcf08103880 0x7fcf0810a130 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fcf0000b0d0 tx=0x7fcf0000b3e0 comp rx=0 tx=0).stop 2024-02-18T11:35:52.977 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:35:52.980+0000 7fcf0ff58700 1 -- 172.21.15.158:0/3239146058 shutdown_connections 2024-02-18T11:35:52.977 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:35:52.980+0000 7fcf0ff58700 1 --2- 172.21.15.158:0/3239146058 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcf08103880 0x7fcf0810a130 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:35:52.977 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:35:52.980+0000 7fcf0ff58700 1 --2- 172.21.15.158:0/3239146058 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf08101ce0 0x7fcf08102100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:35:52.977 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:35:52.980+0000 7fcf0ff58700 1 --2- 172.21.15.158:0/3239146058 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf08102ee0 0x7fcf08103340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:35:52.978 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:35:52.980+0000 7fcf0ff58700 1 -- 172.21.15.158:0/3239146058 >> 172.21.15.158:0/3239146058 conn(0x7fcf080fd2c0 msgr2=0x7fcf080ff6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:35:52.978 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:35:52.980+0000 7fcf0ff58700 1 -- 172.21.15.158:0/3239146058 shutdown_connections 2024-02-18T11:35:52.978 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:35:52.980+0000 7fcf0ff58700 1 -- 172.21.15.158:0/3239146058 wait complete. 2024-02-18T11:35:52.978 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:35:52.980+0000 7fcf0ff58700 1 Processor -- start 2024-02-18T11:35:52.978 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:35:52.980+0000 7fcf0ff58700 1 -- start start 2024-02-18T11:35:52.978 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:35:52.981+0000 7fcf0ff58700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcf08101ce0 0x7fcf08199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:35:52.978 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:35:52.981+0000 7fcf0ff58700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf08102ee0 0x7fcf0819a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:35:52.978 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:35:52.981+0000 7fcf0ff58700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf08103880 0x7fcf081a1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:35:52.978 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:35:52.981+0000 7fcf0ff58700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf081a1b20 con 0x7fcf08102ee0 2024-02-18T11:35:52.978 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:35:52.981+0000 7fcf0ff58700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf081a1c60 con 0x7fcf08101ce0 2024-02-18T11:35:52.978 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:35:52.981+0000 7fcf0ff58700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf081a1df0 con 0x7fcf08103880 2024-02-18T11:35:52.978 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:35:52.981+0000 7fcf0dcf4700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcf08101ce0 0x7fcf08199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:35:52.978 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:35:52.981+0000 7fcf0dcf4700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcf08101ce0 0x7fcf08199f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.158:3300/0 says I am v2:172.21.15.158:34340/0 (socket says 172.21.15.158:34340) 2024-02-18T11:35:52.978 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:35:52.981+0000 7fcf0dcf4700 1 -- 172.21.15.158:0/3275010970 learned_addr learned my addr 172.21.15.158:0/3275010970 (peer_addr_for_me v2:172.21.15.158:0/0) 2024-02-18T11:35:52.979 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:35:52.981+0000 7fcf0e4f5700 1 --2- 172.21.15.158:0/3275010970 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf08103880 0x7fcf081a1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:35:52.979 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:35:52.981+0000 7fcf0d4f3700 1 --2- 172.21.15.158:0/3275010970 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf08102ee0 0x7fcf0819a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:35:52.979 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:35:52.981+0000 7fcf0dcf4700 1 -- 172.21.15.158:0/3275010970 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf08103880 msgr2=0x7fcf081a1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:35:52.979 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:35:52.981+0000 7fcf0dcf4700 1 --2- 172.21.15.158:0/3275010970 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf08103880 0x7fcf081a1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:35:52.979 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:35:52.981+0000 7fcf0dcf4700 1 -- 172.21.15.158:0/3275010970 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf08102ee0 msgr2=0x7fcf0819a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:35:52.979 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:35:52.981+0000 7fcf0dcf4700 1 --2- 172.21.15.158:0/3275010970 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf08102ee0 0x7fcf0819a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:35:52.979 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:35:52.981+0000 7fcf0dcf4700 1 -- 172.21.15.158:0/3275010970 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf00009d00 con 0x7fcf08101ce0 2024-02-18T11:35:52.979 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:35:52.981+0000 7fcf0e4f5700 1 --2- 172.21.15.158:0/3275010970 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf08103880 0x7fcf081a1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:35:52.979 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:35:52.981+0000 7fcf0dcf4700 1 --2- 172.21.15.158:0/3275010970 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcf08101ce0 0x7fcf08199f90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fcf0400d980 tx=0x7fcf0400dd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:35:52.979 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:35:52.981+0000 7fcefeffd700 1 -- 172.21.15.158:0/3275010970 <== mon.2 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf04009b40 con 0x7fcf08101ce0 2024-02-18T11:35:52.979 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:35:52.981+0000 7fcefeffd700 1 -- 172.21.15.158:0/3275010970 <== mon.2 v2:172.21.15.158:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcf0400de60 con 0x7fcf08101ce0 2024-02-18T11:35:52.979 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:35:52.981+0000 7fcf0ff58700 1 -- 172.21.15.158:0/3275010970 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf081a20d0 con 0x7fcf08101ce0 2024-02-18T11:35:52.979 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:35:52.981+0000 7fcefeffd700 1 -- 172.21.15.158:0/3275010970 <== mon.2 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0400f620 con 0x7fcf08101ce0 2024-02-18T11:35:52.979 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:35:52.981+0000 7fcf0ff58700 1 -- 172.21.15.158:0/3275010970 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf081a2580 con 0x7fcf08101ce0 2024-02-18T11:35:52.979 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:35:52.982+0000 7fcf0ff58700 1 -- 172.21.15.158:0/3275010970 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf08068970 con 0x7fcf08101ce0 2024-02-18T11:35:52.980 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:35:52.983+0000 7fcefeffd700 1 -- 172.21.15.158:0/3275010970 <== mon.2 v2:172.21.15.158:3300/0 4 ==== mgrmap(e 16) v1 ==== 137245+0+0 (secure 0 0 0) 0x7fcf04010460 con 0x7fcf08101ce0 2024-02-18T11:35:52.980 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:35:52.984+0000 7fcefeffd700 1 --2- 172.21.15.158:0/3275010970 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fcef40a27e0 0x7fcef40a4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:35:52.980 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:35:52.984+0000 7fcefeffd700 1 -- 172.21.15.158:0/3275010970 <== mon.2 v2:172.21.15.158:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 4965+0+0 (secure 0 0 0) 0x7fcf040cd420 con 0x7fcf08101ce0 2024-02-18T11:35:52.981 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:35:52.984+0000 7fcf0d4f3700 1 --2- 172.21.15.158:0/3275010970 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fcef40a27e0 0x7fcef40a4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:35:52.981 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:35:52.985+0000 7fcf0d4f3700 1 --2- 172.21.15.158:0/3275010970 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fcef40a27e0 0x7fcef40a4ca0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fcef8005fd0 tx=0x7fcef8005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:35:52.982 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:35:52.988+0000 7fcefeffd700 1 -- 172.21.15.158:0/3275010970 <== mon.2 v2:172.21.15.158:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcf04096ec0 con 0x7fcf08101ce0 2024-02-18T11:35:53.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:52 smithi188 ceph-mon[36925]: Adjusting osd_memory_target on smithi012 to 5697M 2024-02-18T11:35:53.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:52 smithi188 ceph-mon[36925]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-18T11:35:53.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:52 smithi188 ceph-mon[36925]: osdmap e49: 7 total, 7 up, 7 in 2024-02-18T11:35:53.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:52 smithi012 ceph-mon[32460]: Adjusting osd_memory_target on smithi012 to 5697M 2024-02-18T11:35:53.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:52 smithi012 ceph-mon[32460]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-18T11:35:53.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:52 smithi012 ceph-mon[32460]: osdmap e49: 7 total, 7 up, 7 in 2024-02-18T11:35:53.181 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:35:53.186+0000 7fcf0ff58700 1 -- 172.21.15.158:0/3275010970 --> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi158:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fcf08075600 con 0x7fcef40a27e0 2024-02-18T11:35:53.992 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:53 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T11:35:53.993 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:53 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T11:35:53.993 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:53 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:35:54.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:53 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T11:35:54.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:53 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T11:35:54.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:53 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:35:54.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:53 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T11:35:54.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:53 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T11:35:54.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:53 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:35:54.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:54 smithi158 ceph-mon[36626]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-18T11:35:54.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:54 smithi158 ceph-mon[36626]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi158:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:35:55.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:54 smithi188 ceph-mon[36925]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-18T11:35:55.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:54 smithi188 ceph-mon[36925]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi158:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:35:55.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:54 smithi012 ceph-mon[32460]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-18T11:35:55.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:54 smithi012 ceph-mon[32460]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi158:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:35:57.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:56 smithi188 ceph-mon[36925]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-18T11:35:57.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:56 smithi188 ceph-mon[36925]: from='client.? 172.21.15.158:0/1145401465' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "363271a6-df93-44c9-b0e5-3d29768aa96c"}]: dispatch 2024-02-18T11:35:57.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:56 smithi188 ceph-mon[36925]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "363271a6-df93-44c9-b0e5-3d29768aa96c"}]: dispatch 2024-02-18T11:35:57.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:56 smithi188 ceph-mon[36925]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "363271a6-df93-44c9-b0e5-3d29768aa96c"}]': finished 2024-02-18T11:35:57.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:56 smithi188 ceph-mon[36925]: osdmap e50: 8 total, 7 up, 8 in 2024-02-18T11:35:57.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:56 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-18T11:35:57.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:56 smithi188 ceph-mon[36925]: from='client.? 172.21.15.158:0/1760735095' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T11:35:57.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:56 smithi012 ceph-mon[32460]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-18T11:35:57.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:56 smithi012 ceph-mon[32460]: from='client.? 172.21.15.158:0/1145401465' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "363271a6-df93-44c9-b0e5-3d29768aa96c"}]: dispatch 2024-02-18T11:35:57.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:56 smithi012 ceph-mon[32460]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "363271a6-df93-44c9-b0e5-3d29768aa96c"}]: dispatch 2024-02-18T11:35:57.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:56 smithi012 ceph-mon[32460]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "363271a6-df93-44c9-b0e5-3d29768aa96c"}]': finished 2024-02-18T11:35:57.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:56 smithi012 ceph-mon[32460]: osdmap e50: 8 total, 7 up, 8 in 2024-02-18T11:35:57.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:56 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-18T11:35:57.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:56 smithi012 ceph-mon[32460]: from='client.? 172.21.15.158:0/1760735095' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T11:35:57.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:56 smithi158 ceph-mon[36626]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-18T11:35:57.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:56 smithi158 ceph-mon[36626]: from='client.? 172.21.15.158:0/1145401465' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "363271a6-df93-44c9-b0e5-3d29768aa96c"}]: dispatch 2024-02-18T11:35:57.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:56 smithi158 ceph-mon[36626]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "363271a6-df93-44c9-b0e5-3d29768aa96c"}]: dispatch 2024-02-18T11:35:57.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:56 smithi158 ceph-mon[36626]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "363271a6-df93-44c9-b0e5-3d29768aa96c"}]': finished 2024-02-18T11:35:57.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:56 smithi158 ceph-mon[36626]: osdmap e50: 8 total, 7 up, 8 in 2024-02-18T11:35:57.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:56 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-18T11:35:57.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:56 smithi158 ceph-mon[36626]: from='client.? 172.21.15.158:0/1760735095' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T11:35:59.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:35:58 smithi188 ceph-mon[36925]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-18T11:35:59.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:35:58 smithi012 ceph-mon[32460]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-18T11:35:59.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:35:58 smithi158 ceph-mon[36626]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-18T11:36:01.008 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:00 smithi158 ceph-mon[36626]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-18T11:36:01.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:00 smithi188 ceph-mon[36925]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-18T11:36:01.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:00 smithi012 ceph-mon[32460]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-18T11:36:03.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:02 smithi188 ceph-mon[36925]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-18T11:36:03.111 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:02 smithi158 ceph-mon[36626]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-18T11:36:03.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:02 smithi012 ceph-mon[32460]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-18T11:36:04.022 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:03 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-02-18T11:36:04.022 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:03 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:36:04.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:03 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-02-18T11:36:04.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:03 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:36:04.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:03 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-02-18T11:36:04.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:03 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:36:04.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:04 smithi158 ceph-mon[36626]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-18T11:36:04.996 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:04 smithi158 ceph-mon[36626]: Deploying daemon osd.7 on smithi158 2024-02-18T11:36:05.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:04 smithi188 ceph-mon[36925]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-18T11:36:05.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:04 smithi188 ceph-mon[36925]: Deploying daemon osd.7 on smithi158 2024-02-18T11:36:05.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:04 smithi012 ceph-mon[32460]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-18T11:36:05.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:04 smithi012 ceph-mon[32460]: Deploying daemon osd.7 on smithi158 2024-02-18T11:36:06.818 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:06 smithi158 ceph-mon[36626]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-18T11:36:07.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:06 smithi188 ceph-mon[36925]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-18T11:36:07.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:06 smithi012 ceph-mon[32460]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-18T11:36:09.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:08 smithi188 ceph-mon[36925]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-18T11:36:09.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:08 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:36:09.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:08 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:36:09.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:08 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:36:09.111 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:08 smithi158 ceph-mon[36626]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-18T11:36:09.111 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:08 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:36:09.111 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:08 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:36:09.111 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:08 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:36:09.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:08 smithi012 ceph-mon[32460]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-18T11:36:09.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:08 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:36:09.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:08 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:36:09.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:08 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:36:10.897 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:10 smithi158 ceph-mon[36626]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-18T11:36:11.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:10 smithi188 ceph-mon[36925]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-18T11:36:11.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:10 smithi012 ceph-mon[32460]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-18T11:36:12.548 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:36:12.553+0000 7fcefeffd700 1 -- 172.21.15.158:0/3275010970 <== mgr.14152 v2:172.21.15.12:6800/1153736685 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fcf08075600 con 0x7fcef40a27e0 2024-02-18T11:36:12.548 INFO:teuthology.orchestra.run.smithi158.stdout:Created osd(s) 7 on host 'smithi158' 2024-02-18T11:36:12.550 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:36:12.555+0000 7fcf0ff58700 1 -- 172.21.15.158:0/3275010970 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fcef40a27e0 msgr2=0x7fcef40a4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:36:12.550 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:36:12.555+0000 7fcf0ff58700 1 --2- 172.21.15.158:0/3275010970 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fcef40a27e0 0x7fcef40a4ca0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fcef8005fd0 tx=0x7fcef8005f60 comp rx=0 tx=0).stop 2024-02-18T11:36:12.550 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:36:12.555+0000 7fcf0ff58700 1 -- 172.21.15.158:0/3275010970 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcf08101ce0 msgr2=0x7fcf08199f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:36:12.551 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:36:12.555+0000 7fcf0ff58700 1 --2- 172.21.15.158:0/3275010970 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcf08101ce0 0x7fcf08199f90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fcf0400d980 tx=0x7fcf0400dd40 comp rx=0 tx=0).stop 2024-02-18T11:36:12.551 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:36:12.555+0000 7fcf0ff58700 1 -- 172.21.15.158:0/3275010970 shutdown_connections 2024-02-18T11:36:12.551 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:36:12.555+0000 7fcf0ff58700 1 --2- 172.21.15.158:0/3275010970 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf08103880 0x7fcf081a1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:36:12.551 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:36:12.555+0000 7fcf0ff58700 1 --2- 172.21.15.158:0/3275010970 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcf08101ce0 0x7fcf08199f90 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:36:12.551 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:36:12.555+0000 7fcf0ff58700 1 --2- 172.21.15.158:0/3275010970 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fcef40a27e0 0x7fcef40a4ca0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:36:12.551 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:36:12.555+0000 7fcf0ff58700 1 --2- 172.21.15.158:0/3275010970 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf08102ee0 0x7fcf0819a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:36:12.551 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:36:12.555+0000 7fcf0ff58700 1 -- 172.21.15.158:0/3275010970 >> 172.21.15.158:0/3275010970 conn(0x7fcf080fd2c0 msgr2=0x7fcf08073ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:36:12.551 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:36:12.556+0000 7fcf0ff58700 1 -- 172.21.15.158:0/3275010970 shutdown_connections 2024-02-18T11:36:12.551 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:36:12.556+0000 7fcf0ff58700 1 -- 172.21.15.158:0/3275010970 wait complete. 2024-02-18T11:36:12.745 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:12 smithi158 ceph-mon[36626]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-18T11:36:12.745 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:12 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:36:12.745 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:12 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:36:12.745 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:12 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:36:12.746 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:12 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:36:12.746 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:12 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:36:12.746 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:12 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:36:12.746 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:12 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:36:12.746 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:12 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:36:12.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:12 smithi188 ceph-mon[36925]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-18T11:36:12.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:12 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:36:12.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:12 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:36:12.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:12 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:36:12.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:12 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:36:12.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:12 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:36:12.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:12 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:36:12.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:12 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:36:12.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:12 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:36:12.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:12 smithi012 ceph-mon[32460]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-18T11:36:12.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:12 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:36:12.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:12 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:36:12.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:12 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:36:12.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:12 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:36:12.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:12 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:36:12.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:12 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:36:12.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:12 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:36:12.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:12 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:36:13.176 DEBUG:teuthology.orchestra.run.smithi158:osd.7> sudo journalctl -f -n 0 -u ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@osd.7.service 2024-02-18T11:36:13.179 INFO:tasks.cephadm:Deploying osd.8 on smithi188 with /dev/vg_nvme/lv_2... 2024-02-18T11:36:13.179 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-02-18T11:36:13.422 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.c/config 2024-02-18T11:36:13.745 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:13 smithi158 ceph-mon[36626]: from='osd.7 [v2:172.21.15.158:6816/3283400032,v1:172.21.15.158:6817/3283400032]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-18T11:36:13.745 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:13 smithi158 ceph-mon[36626]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-18T11:36:13.842 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:13 smithi188 ceph-mon[36925]: from='osd.7 [v2:172.21.15.158:6816/3283400032,v1:172.21.15.158:6817/3283400032]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-18T11:36:13.842 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:13 smithi188 ceph-mon[36925]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-18T11:36:13.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:13 smithi012 ceph-mon[32460]: from='osd.7 [v2:172.21.15.158:6816/3283400032,v1:172.21.15.158:6817/3283400032]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-18T11:36:13.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:13 smithi012 ceph-mon[32460]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-18T11:36:14.836 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:14 smithi188 ceph-mon[36925]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-18T11:36:14.836 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:14 smithi188 ceph-mon[36925]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-02-18T11:36:14.836 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:14 smithi188 ceph-mon[36925]: from='osd.7 [v2:172.21.15.158:6816/3283400032,v1:172.21.15.158:6817/3283400032]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi158", "root=default"]}]: dispatch 2024-02-18T11:36:14.836 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:14 smithi188 ceph-mon[36925]: osdmap e51: 8 total, 7 up, 8 in 2024-02-18T11:36:14.836 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:14 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-18T11:36:14.836 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:14 smithi188 ceph-mon[36925]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi158", "root=default"]}]: dispatch 2024-02-18T11:36:14.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:14 smithi012 ceph-mon[32460]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-18T11:36:14.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:14 smithi012 ceph-mon[32460]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-02-18T11:36:14.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:14 smithi012 ceph-mon[32460]: from='osd.7 [v2:172.21.15.158:6816/3283400032,v1:172.21.15.158:6817/3283400032]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi158", "root=default"]}]: dispatch 2024-02-18T11:36:14.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:14 smithi012 ceph-mon[32460]: osdmap e51: 8 total, 7 up, 8 in 2024-02-18T11:36:14.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:14 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-18T11:36:14.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:14 smithi012 ceph-mon[32460]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi158", "root=default"]}]: dispatch 2024-02-18T11:36:14.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:14 smithi158 ceph-mon[36626]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-18T11:36:14.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:14 smithi158 ceph-mon[36626]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-02-18T11:36:14.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:14 smithi158 ceph-mon[36626]: from='osd.7 [v2:172.21.15.158:6816/3283400032,v1:172.21.15.158:6817/3283400032]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi158", "root=default"]}]: dispatch 2024-02-18T11:36:14.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:14 smithi158 ceph-mon[36626]: osdmap e51: 8 total, 7 up, 8 in 2024-02-18T11:36:14.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:14 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-18T11:36:14.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:14 smithi158 ceph-mon[36626]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi158", "root=default"]}]: dispatch 2024-02-18T11:36:15.538 INFO:teuthology.orchestra.run.smithi188.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-02-18T11:36:15.538 INFO:teuthology.orchestra.run.smithi188.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-02-18T11:36:15.538 INFO:teuthology.orchestra.run.smithi188.stderr: stderr: 10+0 records in 2024-02-18T11:36:15.538 INFO:teuthology.orchestra.run.smithi188.stderr:10+0 records out 2024-02-18T11:36:15.538 INFO:teuthology.orchestra.run.smithi188.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113949 s, 920 MB/s 2024-02-18T11:36:15.538 INFO:teuthology.orchestra.run.smithi188.stderr:--> Zapping successful for: 2024-02-18T11:36:15.611 INFO:journalctl@ceph.osd.7.smithi158.stdout:Feb 18 11:36:15 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-7[54444]: 2024-02-18T11:36:15.297+0000 7f24d7038700 -1 osd.7 0 waiting for initial osdmap 2024-02-18T11:36:15.612 INFO:journalctl@ceph.osd.7.smithi158.stdout:Feb 18 11:36:15 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-7[54444]: 2024-02-18T11:36:15.308+0000 7f24cde21700 -1 osd.7 52 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-18T11:36:15.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:15 smithi188 ceph-mon[36925]: purged_snaps scrub starts 2024-02-18T11:36:15.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:15 smithi188 ceph-mon[36925]: purged_snaps scrub ok 2024-02-18T11:36:15.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:15 smithi188 ceph-mon[36925]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi158", "root=default"]}]': finished 2024-02-18T11:36:15.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:15 smithi188 ceph-mon[36925]: osdmap e52: 8 total, 7 up, 8 in 2024-02-18T11:36:15.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:15 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-18T11:36:15.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:15 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-18T11:36:15.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:15 smithi188 ceph-mon[36925]: from='osd.7 ' entity='osd.7' 2024-02-18T11:36:15.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:15 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-18T11:36:15.872 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:15 smithi158 ceph-mon[36626]: purged_snaps scrub starts 2024-02-18T11:36:15.872 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:15 smithi158 ceph-mon[36626]: purged_snaps scrub ok 2024-02-18T11:36:15.872 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:15 smithi158 ceph-mon[36626]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi158", "root=default"]}]': finished 2024-02-18T11:36:15.872 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:15 smithi158 ceph-mon[36626]: osdmap e52: 8 total, 7 up, 8 in 2024-02-18T11:36:15.872 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:15 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-18T11:36:15.872 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:15 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-18T11:36:15.872 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:15 smithi158 ceph-mon[36626]: from='osd.7 ' entity='osd.7' 2024-02-18T11:36:15.872 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:15 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-18T11:36:15.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:15 smithi012 ceph-mon[32460]: purged_snaps scrub starts 2024-02-18T11:36:15.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:15 smithi012 ceph-mon[32460]: purged_snaps scrub ok 2024-02-18T11:36:15.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:15 smithi012 ceph-mon[32460]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi158", "root=default"]}]': finished 2024-02-18T11:36:15.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:15 smithi012 ceph-mon[32460]: osdmap e52: 8 total, 7 up, 8 in 2024-02-18T11:36:15.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:15 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-18T11:36:15.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:15 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-18T11:36:15.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:15 smithi012 ceph-mon[32460]: from='osd.7 ' entity='osd.7' 2024-02-18T11:36:15.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:15 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-18T11:36:16.180 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi188:vg_nvme/lv_2 2024-02-18T11:36:16.424 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.c/config 2024-02-18T11:36:16.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:16 smithi188 ceph-mon[36925]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-18T11:36:16.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:16 smithi188 ceph-mon[36925]: osd.7 [v2:172.21.15.158:6816/3283400032,v1:172.21.15.158:6817/3283400032] boot 2024-02-18T11:36:16.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:16 smithi188 ceph-mon[36925]: osdmap e53: 8 total, 8 up, 8 in 2024-02-18T11:36:16.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:16 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-18T11:36:16.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:16 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:36:16.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:16 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:36:16.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:16 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:36:16.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:16 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:36:16.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:16 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:36:16.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:16 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:36:16.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:16 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:36:16.856 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:16 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:36:16.856 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:16 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:36:16.856 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:16 smithi188 ceph-mon[36925]: osdmap e54: 8 total, 8 up, 8 in 2024-02-18T11:36:16.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:16 smithi012 ceph-mon[32460]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-18T11:36:16.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:16 smithi012 ceph-mon[32460]: osd.7 [v2:172.21.15.158:6816/3283400032,v1:172.21.15.158:6817/3283400032] boot 2024-02-18T11:36:16.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:16 smithi012 ceph-mon[32460]: osdmap e53: 8 total, 8 up, 8 in 2024-02-18T11:36:16.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:16 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-18T11:36:16.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:16 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:36:16.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:16 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:36:16.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:16 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:36:16.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:16 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:36:16.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:16 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:36:16.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:16 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:36:16.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:16 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:36:16.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:16 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:36:16.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:16 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:36:16.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:16 smithi012 ceph-mon[32460]: osdmap e54: 8 total, 8 up, 8 in 2024-02-18T11:36:16.996 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:16 smithi158 ceph-mon[36626]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-18T11:36:16.996 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:16 smithi158 ceph-mon[36626]: osd.7 [v2:172.21.15.158:6816/3283400032,v1:172.21.15.158:6817/3283400032] boot 2024-02-18T11:36:16.996 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:16 smithi158 ceph-mon[36626]: osdmap e53: 8 total, 8 up, 8 in 2024-02-18T11:36:16.996 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:16 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-18T11:36:16.996 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:16 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:36:16.996 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:16 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:36:16.996 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:16 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:36:16.997 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:16 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:36:16.997 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:16 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:36:16.997 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:16 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:36:16.997 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:16 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:36:16.997 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:16 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:36:16.997 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:16 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:36:16.997 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:16 smithi158 ceph-mon[36626]: osdmap e54: 8 total, 8 up, 8 in 2024-02-18T11:36:17.844 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:17 smithi188 ceph-mon[36925]: Detected new or changed devices on smithi158 2024-02-18T11:36:17.844 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:17 smithi188 ceph-mon[36925]: Adjusting osd_memory_target on smithi158 to 5693M 2024-02-18T11:36:17.844 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:17 smithi188 ceph-mon[36925]: osdmap e55: 8 total, 8 up, 8 in 2024-02-18T11:36:17.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:17 smithi012 ceph-mon[32460]: Detected new or changed devices on smithi158 2024-02-18T11:36:17.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:17 smithi012 ceph-mon[32460]: Adjusting osd_memory_target on smithi158 to 5693M 2024-02-18T11:36:17.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:17 smithi012 ceph-mon[32460]: osdmap e55: 8 total, 8 up, 8 in 2024-02-18T11:36:17.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:17 smithi158 ceph-mon[36626]: Detected new or changed devices on smithi158 2024-02-18T11:36:17.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:17 smithi158 ceph-mon[36626]: Adjusting osd_memory_target on smithi158 to 5693M 2024-02-18T11:36:17.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:17 smithi158 ceph-mon[36626]: osdmap e55: 8 total, 8 up, 8 in 2024-02-18T11:36:18.063 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:36:18.045+0000 7f1f8990f700 1 -- 172.21.15.188:0/681957864 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1f84101ce0 msgr2=0x7f1f84102100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:36:18.063 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:36:18.045+0000 7f1f8990f700 1 --2- 172.21.15.188:0/681957864 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1f84101ce0 0x7f1f84102100 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1f78009990 tx=0x7f1f78009ca0 comp rx=0 tx=0).stop 2024-02-18T11:36:18.064 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:36:18.045+0000 7f1f8990f700 1 -- 172.21.15.188:0/681957864 shutdown_connections 2024-02-18T11:36:18.064 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:36:18.045+0000 7f1f8990f700 1 --2- 172.21.15.188:0/681957864 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f84103880 0x7f1f8410a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:36:18.064 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:36:18.045+0000 7f1f8990f700 1 --2- 172.21.15.188:0/681957864 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1f84101ce0 0x7f1f84102100 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:36:18.064 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:36:18.045+0000 7f1f8990f700 1 --2- 172.21.15.188:0/681957864 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1f84102ee0 0x7f1f84103340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:36:18.064 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:36:18.045+0000 7f1f8990f700 1 -- 172.21.15.188:0/681957864 >> 172.21.15.188:0/681957864 conn(0x7f1f840fd2c0 msgr2=0x7f1f840ff6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:36:18.064 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:36:18.045+0000 7f1f8990f700 1 -- 172.21.15.188:0/681957864 shutdown_connections 2024-02-18T11:36:18.064 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:36:18.045+0000 7f1f8990f700 1 -- 172.21.15.188:0/681957864 wait complete. 2024-02-18T11:36:18.064 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:36:18.046+0000 7f1f8990f700 1 Processor -- start 2024-02-18T11:36:18.064 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:36:18.046+0000 7f1f8990f700 1 -- start start 2024-02-18T11:36:18.064 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:36:18.046+0000 7f1f8990f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1f84101ce0 0x7f1f84197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:36:18.064 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:36:18.046+0000 7f1f8990f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f84102ee0 0x7f1f84198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:36:18.064 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:36:18.046+0000 7f1f8990f700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1f84103880 0x7f1f8419f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:36:18.065 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:36:18.046+0000 7f1f8990f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8419f790 con 0x7f1f84102ee0 2024-02-18T11:36:18.065 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:36:18.046+0000 7f1f8990f700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8419f8d0 con 0x7f1f84103880 2024-02-18T11:36:18.065 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:36:18.046+0000 7f1f8990f700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8419fa90 con 0x7f1f84101ce0 2024-02-18T11:36:18.065 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:36:18.046+0000 7f1f82ffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1f84101ce0 0x7f1f84197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:36:18.065 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:36:18.046+0000 7f1f82ffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1f84101ce0 0x7f1f84197c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:53958/0 (socket says 172.21.15.188:53958) 2024-02-18T11:36:18.065 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:36:18.046+0000 7f1f82ffd700 1 -- 172.21.15.188:0/2136968344 learned_addr learned my addr 172.21.15.188:0/2136968344 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-02-18T11:36:18.065 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:36:18.046+0000 7f1f827fc700 1 --2- 172.21.15.188:0/2136968344 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f84102ee0 0x7f1f84198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:36:18.065 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:36:18.046+0000 7f1f837fe700 1 --2- 172.21.15.188:0/2136968344 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1f84103880 0x7f1f8419f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:36:18.065 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:36:18.046+0000 7f1f82ffd700 1 -- 172.21.15.188:0/2136968344 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1f84103880 msgr2=0x7f1f8419f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:36:18.065 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:36:18.046+0000 7f1f82ffd700 1 --2- 172.21.15.188:0/2136968344 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1f84103880 0x7f1f8419f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:36:18.065 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:36:18.046+0000 7f1f82ffd700 1 -- 172.21.15.188:0/2136968344 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f84102ee0 msgr2=0x7f1f84198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:36:18.065 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:36:18.046+0000 7f1f82ffd700 1 --2- 172.21.15.188:0/2136968344 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f84102ee0 0x7f1f84198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:36:18.066 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:36:18.046+0000 7f1f82ffd700 1 -- 172.21.15.188:0/2136968344 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f780095f0 con 0x7f1f84101ce0 2024-02-18T11:36:18.066 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:36:18.046+0000 7f1f827fc700 1 --2- 172.21.15.188:0/2136968344 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f84102ee0 0x7f1f84198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:36:18.066 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:36:18.046+0000 7f1f82ffd700 1 --2- 172.21.15.188:0/2136968344 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1f84101ce0 0x7f1f84197c40 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f1f780095c0 tx=0x7f1f7800f740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:36:18.066 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:36:18.047+0000 7f1f8890d700 1 -- 172.21.15.188:0/2136968344 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7801d070 con 0x7f1f84101ce0 2024-02-18T11:36:18.066 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:36:18.047+0000 7f1f8890d700 1 -- 172.21.15.188:0/2136968344 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1f7800fc30 con 0x7f1f84101ce0 2024-02-18T11:36:18.066 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:36:18.047+0000 7f1f8990f700 1 -- 172.21.15.188:0/2136968344 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f8419fd10 con 0x7f1f84101ce0 2024-02-18T11:36:18.066 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:36:18.047+0000 7f1f8890d700 1 -- 172.21.15.188:0/2136968344 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f780173f0 con 0x7f1f84101ce0 2024-02-18T11:36:18.066 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:36:18.047+0000 7f1f8990f700 1 -- 172.21.15.188:0/2136968344 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f841a0120 con 0x7f1f84101ce0 2024-02-18T11:36:18.066 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:36:18.048+0000 7f1f8990f700 1 -- 172.21.15.188:0/2136968344 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f84068970 con 0x7f1f84101ce0 2024-02-18T11:36:18.066 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:36:18.049+0000 7f1f8890d700 1 -- 172.21.15.188:0/2136968344 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 16) v1 ==== 137245+0+0 (secure 0 0 0) 0x7f1f78017550 con 0x7f1f84101ce0 2024-02-18T11:36:18.066 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:36:18.049+0000 7f1f8890d700 1 --2- 172.21.15.188:0/2136968344 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f1f6c0a27e0 0x7f1f6c0a4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:36:18.066 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:36:18.050+0000 7f1f8890d700 1 -- 172.21.15.188:0/2136968344 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5426+0+0 (secure 0 0 0) 0x7f1f780cedd0 con 0x7f1f84101ce0 2024-02-18T11:36:18.066 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:36:18.050+0000 7f1f827fc700 1 --2- 172.21.15.188:0/2136968344 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f1f6c0a27e0 0x7f1f6c0a4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:36:18.066 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:36:18.050+0000 7f1f827fc700 1 --2- 172.21.15.188:0/2136968344 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f1f6c0a27e0 0x7f1f6c0a4ca0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1f74005950 tx=0x7f1f740058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:36:18.067 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:36:18.053+0000 7f1f8890d700 1 -- 172.21.15.188:0/2136968344 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1f78098730 con 0x7f1f84101ce0 2024-02-18T11:36:18.251 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:36:18.256+0000 7f1f8990f700 1 -- 172.21.15.188:0/2136968344 --> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f1f8402d990 con 0x7f1f6c0a27e0 2024-02-18T11:36:18.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:18 smithi012 ceph-mon[32460]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-02-18T11:36:18.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:18 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T11:36:18.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:18 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T11:36:18.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:18 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:36:18.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:18 smithi012 ceph-mon[32460]: osdmap e56: 8 total, 8 up, 8 in 2024-02-18T11:36:18.946 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:18 smithi188 ceph-mon[36925]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-02-18T11:36:18.947 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:18 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T11:36:18.947 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:18 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T11:36:18.947 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:18 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:36:18.947 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:18 smithi188 ceph-mon[36925]: osdmap e56: 8 total, 8 up, 8 in 2024-02-18T11:36:18.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:18 smithi158 ceph-mon[36626]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-02-18T11:36:18.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:18 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T11:36:18.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:18 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T11:36:18.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:18 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:36:18.996 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:18 smithi158 ceph-mon[36626]: osdmap e56: 8 total, 8 up, 8 in 2024-02-18T11:36:19.711 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:19 smithi188 ceph-mon[36925]: from='client.24289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:36:19.712 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:19 smithi188 ceph-mon[36925]: osdmap e57: 8 total, 8 up, 8 in 2024-02-18T11:36:19.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:19 smithi012 ceph-mon[32460]: from='client.24289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:36:19.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:19 smithi012 ceph-mon[32460]: osdmap e57: 8 total, 8 up, 8 in 2024-02-18T11:36:19.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:19 smithi158 ceph-mon[36626]: from='client.24289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:36:19.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:19 smithi158 ceph-mon[36626]: osdmap e57: 8 total, 8 up, 8 in 2024-02-18T11:36:20.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:20 smithi012 ceph-mon[32460]: pgmap v180: 1 pgs: 1 remapped+peering; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-02-18T11:36:20.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:20 smithi158 ceph-mon[36626]: pgmap v180: 1 pgs: 1 remapped+peering; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-02-18T11:36:21.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:20 smithi188 ceph-mon[36925]: pgmap v180: 1 pgs: 1 remapped+peering; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-02-18T11:36:21.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:21 smithi012 ceph-mon[32460]: from='client.? 172.21.15.188:0/3176492031' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9a089528-3e8e-45d3-9279-ddb54088b96a"}]: dispatch 2024-02-18T11:36:21.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:21 smithi012 ceph-mon[32460]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9a089528-3e8e-45d3-9279-ddb54088b96a"}]: dispatch 2024-02-18T11:36:21.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:21 smithi012 ceph-mon[32460]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9a089528-3e8e-45d3-9279-ddb54088b96a"}]': finished 2024-02-18T11:36:21.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:21 smithi012 ceph-mon[32460]: osdmap e58: 9 total, 8 up, 9 in 2024-02-18T11:36:21.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:21 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-18T11:36:21.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:21 smithi012 ceph-mon[32460]: from='client.? 172.21.15.188:0/3078342092' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T11:36:21.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:21 smithi158 ceph-mon[36626]: from='client.? 172.21.15.188:0/3176492031' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9a089528-3e8e-45d3-9279-ddb54088b96a"}]: dispatch 2024-02-18T11:36:21.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:21 smithi158 ceph-mon[36626]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9a089528-3e8e-45d3-9279-ddb54088b96a"}]: dispatch 2024-02-18T11:36:21.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:21 smithi158 ceph-mon[36626]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9a089528-3e8e-45d3-9279-ddb54088b96a"}]': finished 2024-02-18T11:36:21.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:21 smithi158 ceph-mon[36626]: osdmap e58: 9 total, 8 up, 9 in 2024-02-18T11:36:21.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:21 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-18T11:36:21.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:21 smithi158 ceph-mon[36626]: from='client.? 172.21.15.188:0/3078342092' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T11:36:22.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:21 smithi188 ceph-mon[36925]: from='client.? 172.21.15.188:0/3176492031' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9a089528-3e8e-45d3-9279-ddb54088b96a"}]: dispatch 2024-02-18T11:36:22.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:21 smithi188 ceph-mon[36925]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9a089528-3e8e-45d3-9279-ddb54088b96a"}]: dispatch 2024-02-18T11:36:22.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:21 smithi188 ceph-mon[36925]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9a089528-3e8e-45d3-9279-ddb54088b96a"}]': finished 2024-02-18T11:36:22.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:21 smithi188 ceph-mon[36925]: osdmap e58: 9 total, 8 up, 9 in 2024-02-18T11:36:22.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:21 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-18T11:36:22.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:21 smithi188 ceph-mon[36925]: from='client.? 172.21.15.188:0/3078342092' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T11:36:23.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:22 smithi188 ceph-mon[36925]: pgmap v183: 1 pgs: 1 remapped+peering; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-02-18T11:36:23.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:22 smithi012 ceph-mon[32460]: pgmap v183: 1 pgs: 1 remapped+peering; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-02-18T11:36:23.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:22 smithi158 ceph-mon[36626]: pgmap v183: 1 pgs: 1 remapped+peering; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-02-18T11:36:25.061 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:24 smithi188 ceph-mon[36925]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-02-18T11:36:25.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:24 smithi012 ceph-mon[32460]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-02-18T11:36:25.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:24 smithi158 ceph-mon[36626]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-02-18T11:36:27.077 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:26 smithi188 ceph-mon[36925]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-02-18T11:36:27.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:26 smithi012 ceph-mon[32460]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-02-18T11:36:27.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:26 smithi158 ceph-mon[36626]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-02-18T11:36:29.101 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:28 smithi188 ceph-mon[36925]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-02-18T11:36:29.101 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:28 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-02-18T11:36:29.101 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:28 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:36:29.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:28 smithi012 ceph-mon[32460]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-02-18T11:36:29.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:28 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-02-18T11:36:29.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:28 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:36:29.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:28 smithi158 ceph-mon[36626]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-02-18T11:36:29.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:28 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-02-18T11:36:29.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:28 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:36:30.074 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:29 smithi188 ceph-mon[36925]: Deploying daemon osd.8 on smithi188 2024-02-18T11:36:30.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:29 smithi012 ceph-mon[32460]: Deploying daemon osd.8 on smithi188 2024-02-18T11:36:30.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:29 smithi158 ceph-mon[36626]: Deploying daemon osd.8 on smithi188 2024-02-18T11:36:31.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:30 smithi188 ceph-mon[36925]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-02-18T11:36:31.146 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:30 smithi158 ceph-mon[36626]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-02-18T11:36:31.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:30 smithi012 ceph-mon[32460]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-02-18T11:36:33.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:32 smithi188 ceph-mon[36925]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-02-18T11:36:33.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:32 smithi012 ceph-mon[32460]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-02-18T11:36:33.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:32 smithi158 ceph-mon[36626]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-02-18T11:36:35.030 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:34 smithi188 ceph-mon[36925]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-02-18T11:36:35.031 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:34 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:36:35.031 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:34 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:36:35.031 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:34 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:36:35.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:34 smithi012 ceph-mon[32460]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-02-18T11:36:35.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:34 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:36:35.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:34 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:36:35.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:34 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:36:35.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:34 smithi158 ceph-mon[36626]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-02-18T11:36:35.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:34 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:36:35.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:34 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:36:35.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:34 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:36:37.003 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:36 smithi188 ceph-mon[36925]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-02-18T11:36:37.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:36 smithi012 ceph-mon[32460]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-02-18T11:36:37.248 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:36 smithi158 ceph-mon[36626]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-02-18T11:36:37.373 INFO:teuthology.orchestra.run.smithi188.stdout:Created osd(s) 8 on host 'smithi188' 2024-02-18T11:36:37.373 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:36:37.377+0000 7f1f8890d700 1 -- 172.21.15.188:0/2136968344 <== mgr.14152 v2:172.21.15.12:6800/1153736685 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f1f8402d990 con 0x7f1f6c0a27e0 2024-02-18T11:36:37.376 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:36:37.380+0000 7f1f8990f700 1 -- 172.21.15.188:0/2136968344 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f1f6c0a27e0 msgr2=0x7f1f6c0a4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:36:37.376 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:36:37.380+0000 7f1f8990f700 1 --2- 172.21.15.188:0/2136968344 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f1f6c0a27e0 0x7f1f6c0a4ca0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1f74005950 tx=0x7f1f740058e0 comp rx=0 tx=0).stop 2024-02-18T11:36:37.376 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:36:37.380+0000 7f1f8990f700 1 -- 172.21.15.188:0/2136968344 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1f84101ce0 msgr2=0x7f1f84197c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:36:37.376 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:36:37.380+0000 7f1f8990f700 1 --2- 172.21.15.188:0/2136968344 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1f84101ce0 0x7f1f84197c40 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f1f780095c0 tx=0x7f1f7800f740 comp rx=0 tx=0).stop 2024-02-18T11:36:37.376 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:36:37.380+0000 7f1f8990f700 1 -- 172.21.15.188:0/2136968344 shutdown_connections 2024-02-18T11:36:37.376 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:36:37.380+0000 7f1f8990f700 1 --2- 172.21.15.188:0/2136968344 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1f84103880 0x7f1f8419f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:36:37.376 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:36:37.380+0000 7f1f8990f700 1 --2- 172.21.15.188:0/2136968344 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1f84101ce0 0x7f1f84197c40 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:36:37.376 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:36:37.380+0000 7f1f8990f700 1 --2- 172.21.15.188:0/2136968344 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f1f6c0a27e0 0x7f1f6c0a4ca0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:36:37.376 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:36:37.380+0000 7f1f8990f700 1 --2- 172.21.15.188:0/2136968344 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f84102ee0 0x7f1f84198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:36:37.376 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:36:37.380+0000 7f1f8990f700 1 -- 172.21.15.188:0/2136968344 >> 172.21.15.188:0/2136968344 conn(0x7f1f840fd2c0 msgr2=0x7f1f84106260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:36:37.376 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:36:37.380+0000 7f1f8990f700 1 -- 172.21.15.188:0/2136968344 shutdown_connections 2024-02-18T11:36:37.376 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:36:37.380+0000 7f1f8990f700 1 -- 172.21.15.188:0/2136968344 wait complete. 2024-02-18T11:36:38.042 DEBUG:teuthology.orchestra.run.smithi188:osd.8> sudo journalctl -f -n 0 -u ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@osd.8.service 2024-02-18T11:36:38.044 INFO:tasks.cephadm:Deploying osd.9 on smithi012 with /dev/vg_nvme/lv_1... 2024-02-18T11:36:38.044 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-02-18T11:36:38.300 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T11:36:38.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:38 smithi188 ceph-mon[36925]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-02-18T11:36:38.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:38 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:36:38.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:38 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:36:38.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:38 smithi188 ceph-mon[36925]: from='osd.8 [v2:172.21.15.188:6816/2110170682,v1:172.21.15.188:6817/2110170682]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-02-18T11:36:38.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:38 smithi188 ceph-mon[36925]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-02-18T11:36:38.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:38 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:36:38.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:38 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:36:38.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:38 smithi012 ceph-mon[32460]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-02-18T11:36:38.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:38 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:36:38.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:38 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:36:38.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:38 smithi012 ceph-mon[32460]: from='osd.8 [v2:172.21.15.188:6816/2110170682,v1:172.21.15.188:6817/2110170682]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-02-18T11:36:38.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:38 smithi012 ceph-mon[32460]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-02-18T11:36:38.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:38 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:36:38.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:38 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:36:38.745 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:38 smithi158 ceph-mon[36626]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-02-18T11:36:38.745 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:38 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:36:38.745 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:38 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:36:38.745 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:38 smithi158 ceph-mon[36626]: from='osd.8 [v2:172.21.15.188:6816/2110170682,v1:172.21.15.188:6817/2110170682]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-02-18T11:36:38.745 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:38 smithi158 ceph-mon[36626]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-02-18T11:36:38.745 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:38 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:36:38.745 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:38 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:36:40.036 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:39 smithi188 ceph-mon[36925]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-02-18T11:36:40.036 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:39 smithi188 ceph-mon[36925]: osdmap e59: 9 total, 8 up, 9 in 2024-02-18T11:36:40.037 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:39 smithi188 ceph-mon[36925]: from='osd.8 [v2:172.21.15.188:6816/2110170682,v1:172.21.15.188:6817/2110170682]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-02-18T11:36:40.037 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:39 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-18T11:36:40.037 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:39 smithi188 ceph-mon[36925]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-02-18T11:36:40.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:39 smithi012 ceph-mon[32460]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-02-18T11:36:40.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:39 smithi012 ceph-mon[32460]: osdmap e59: 9 total, 8 up, 9 in 2024-02-18T11:36:40.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:39 smithi012 ceph-mon[32460]: from='osd.8 [v2:172.21.15.188:6816/2110170682,v1:172.21.15.188:6817/2110170682]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-02-18T11:36:40.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:39 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-18T11:36:40.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:39 smithi012 ceph-mon[32460]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-02-18T11:36:40.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:39 smithi158 ceph-mon[36626]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-02-18T11:36:40.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:39 smithi158 ceph-mon[36626]: osdmap e59: 9 total, 8 up, 9 in 2024-02-18T11:36:40.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:39 smithi158 ceph-mon[36626]: from='osd.8 [v2:172.21.15.188:6816/2110170682,v1:172.21.15.188:6817/2110170682]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-02-18T11:36:40.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:39 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-18T11:36:40.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:39 smithi158 ceph-mon[36626]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-02-18T11:36:40.397 INFO:teuthology.orchestra.run.smithi012.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-02-18T11:36:40.398 INFO:teuthology.orchestra.run.smithi012.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-02-18T11:36:40.398 INFO:teuthology.orchestra.run.smithi012.stderr: stderr: 10+0 records in 2024-02-18T11:36:40.398 INFO:teuthology.orchestra.run.smithi012.stderr:10+0 records out 2024-02-18T11:36:40.398 INFO:teuthology.orchestra.run.smithi012.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0128247 s, 818 MB/s 2024-02-18T11:36:40.398 INFO:teuthology.orchestra.run.smithi012.stderr:--> Zapping successful for: 2024-02-18T11:36:40.969 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:40 smithi188 ceph-mon[36925]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-02-18T11:36:40.969 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:40 smithi188 ceph-mon[36925]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-02-18T11:36:40.969 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:40 smithi188 ceph-mon[36925]: osdmap e60: 9 total, 8 up, 9 in 2024-02-18T11:36:40.969 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:40 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-18T11:36:40.969 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:40 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-18T11:36:40.969 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:40 smithi188 ceph-mon[36925]: from='osd.8 ' entity='osd.8' 2024-02-18T11:36:40.970 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:40 smithi188 ceph-mon[36925]: osd.8 [v2:172.21.15.188:6816/2110170682,v1:172.21.15.188:6817/2110170682] boot 2024-02-18T11:36:40.970 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:40 smithi188 ceph-mon[36925]: osdmap e61: 9 total, 9 up, 9 in 2024-02-18T11:36:40.970 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:40 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-18T11:36:40.970 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:36:40 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:36:40.635+0000 7f9717bf3700 -1 osd.8 0 waiting for initial osdmap 2024-02-18T11:36:40.970 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:36:40 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:36:40.651+0000 7f97101df700 -1 osd.8 60 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-18T11:36:41.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:40 smithi012 ceph-mon[32460]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-02-18T11:36:41.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:40 smithi012 ceph-mon[32460]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-02-18T11:36:41.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:40 smithi012 ceph-mon[32460]: osdmap e60: 9 total, 8 up, 9 in 2024-02-18T11:36:41.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:40 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-18T11:36:41.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:40 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-18T11:36:41.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:40 smithi012 ceph-mon[32460]: from='osd.8 ' entity='osd.8' 2024-02-18T11:36:41.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:40 smithi012 ceph-mon[32460]: osd.8 [v2:172.21.15.188:6816/2110170682,v1:172.21.15.188:6817/2110170682] boot 2024-02-18T11:36:41.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:40 smithi012 ceph-mon[32460]: osdmap e61: 9 total, 9 up, 9 in 2024-02-18T11:36:41.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:40 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-18T11:36:41.226 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi012:vg_nvme/lv_1 2024-02-18T11:36:41.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:40 smithi158 ceph-mon[36626]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-02-18T11:36:41.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:40 smithi158 ceph-mon[36626]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-02-18T11:36:41.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:40 smithi158 ceph-mon[36626]: osdmap e60: 9 total, 8 up, 9 in 2024-02-18T11:36:41.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:40 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-18T11:36:41.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:40 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-18T11:36:41.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:40 smithi158 ceph-mon[36626]: from='osd.8 ' entity='osd.8' 2024-02-18T11:36:41.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:40 smithi158 ceph-mon[36626]: osd.8 [v2:172.21.15.188:6816/2110170682,v1:172.21.15.188:6817/2110170682] boot 2024-02-18T11:36:41.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:40 smithi158 ceph-mon[36626]: osdmap e61: 9 total, 9 up, 9 in 2024-02-18T11:36:41.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:40 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-18T11:36:41.470 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T11:36:42.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:41 smithi188 ceph-mon[36925]: purged_snaps scrub starts 2024-02-18T11:36:42.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:41 smithi188 ceph-mon[36925]: purged_snaps scrub ok 2024-02-18T11:36:42.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:41 smithi188 ceph-mon[36925]: osdmap e62: 9 total, 9 up, 9 in 2024-02-18T11:36:42.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:41 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:36:42.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:41 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:36:42.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:41 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:36:42.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:41 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:36:42.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:41 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:36:42.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:41 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:36:42.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:41 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:36:42.106 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:41 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:36:42.106 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:41 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:36:42.157 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:41 smithi012 ceph-mon[32460]: purged_snaps scrub starts 2024-02-18T11:36:42.157 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:41 smithi012 ceph-mon[32460]: purged_snaps scrub ok 2024-02-18T11:36:42.157 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:41 smithi012 ceph-mon[32460]: osdmap e62: 9 total, 9 up, 9 in 2024-02-18T11:36:42.157 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:41 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:36:42.157 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:41 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:36:42.157 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:41 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:36:42.157 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:41 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:36:42.157 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:41 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:36:42.157 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:41 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:36:42.157 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:41 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:36:42.157 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:41 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:36:42.158 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:41 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:36:42.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:41 smithi158 ceph-mon[36626]: purged_snaps scrub starts 2024-02-18T11:36:42.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:41 smithi158 ceph-mon[36626]: purged_snaps scrub ok 2024-02-18T11:36:42.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:41 smithi158 ceph-mon[36626]: osdmap e62: 9 total, 9 up, 9 in 2024-02-18T11:36:42.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:41 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:36:42.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:41 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:36:42.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:41 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:36:42.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:41 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:36:42.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:41 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:36:42.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:41 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:36:42.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:41 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:36:42.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:41 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:36:42.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:41 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:36:43.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:36:43.048+0000 7f99850cb700 1 -- 172.21.15.12:0/1590858718 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9980102780 msgr2=0x7f9980109030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:36:43.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:36:43.048+0000 7f99850cb700 1 --2- 172.21.15.12:0/1590858718 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9980102780 0x7f9980109030 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f997800b0d0 tx=0x7f997800b3e0 comp rx=0 tx=0).stop 2024-02-18T11:36:43.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:36:43.048+0000 7f99850cb700 1 -- 172.21.15.12:0/1590858718 shutdown_connections 2024-02-18T11:36:43.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:36:43.048+0000 7f99850cb700 1 --2- 172.21.15.12:0/1590858718 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9980102780 0x7f9980109030 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:36:43.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:36:43.048+0000 7f99850cb700 1 --2- 172.21.15.12:0/1590858718 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9980100be0 0x7f9980101000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:36:43.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:36:43.048+0000 7f99850cb700 1 --2- 172.21.15.12:0/1590858718 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9980101de0 0x7f9980102240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:36:43.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:36:43.048+0000 7f99850cb700 1 -- 172.21.15.12:0/1590858718 >> 172.21.15.12:0/1590858718 conn(0x7f99800fc1a0 msgr2=0x7f99800fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:36:43.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:36:43.048+0000 7f99850cb700 1 -- 172.21.15.12:0/1590858718 shutdown_connections 2024-02-18T11:36:43.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:36:43.048+0000 7f99850cb700 1 -- 172.21.15.12:0/1590858718 wait complete. 2024-02-18T11:36:43.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:36:43.049+0000 7f99850cb700 1 Processor -- start 2024-02-18T11:36:43.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:36:43.049+0000 7f99850cb700 1 -- start start 2024-02-18T11:36:43.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:36:43.049+0000 7f99850cb700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9980100be0 0x7f998019a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:36:43.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:36:43.049+0000 7f99850cb700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9980101de0 0x7f998019a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:36:43.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:36:43.049+0000 7f99850cb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9980102780 0x7f99801a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:36:43.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:36:43.049+0000 7f99850cb700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99801a1cb0 con 0x7f9980102780 2024-02-18T11:36:43.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:36:43.049+0000 7f99850cb700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99801a1df0 con 0x7f9980100be0 2024-02-18T11:36:43.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:36:43.049+0000 7f99850cb700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99801a1f80 con 0x7f9980101de0 2024-02-18T11:36:43.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:36:43.049+0000 7f997f59e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9980102780 0x7f99801a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:36:43.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:36:43.049+0000 7f997f59e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9980102780 0x7f99801a16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54412/0 (socket says 172.21.15.12:54412) 2024-02-18T11:36:43.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:36:43.049+0000 7f997f59e700 1 -- 172.21.15.12:0/4014739934 learned_addr learned my addr 172.21.15.12:0/4014739934 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:36:43.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:36:43.049+0000 7f997e59c700 1 --2- 172.21.15.12:0/4014739934 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9980101de0 0x7f998019a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:36:43.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:36:43.049+0000 7f997ed9d700 1 --2- 172.21.15.12:0/4014739934 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9980100be0 0x7f998019a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:36:43.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:36:43.050+0000 7f997f59e700 1 -- 172.21.15.12:0/4014739934 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9980101de0 msgr2=0x7f998019a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:36:43.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:36:43.050+0000 7f997f59e700 1 --2- 172.21.15.12:0/4014739934 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9980101de0 0x7f998019a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:36:43.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:36:43.050+0000 7f997f59e700 1 -- 172.21.15.12:0/4014739934 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9980100be0 msgr2=0x7f998019a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:36:43.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:36:43.050+0000 7f997f59e700 1 --2- 172.21.15.12:0/4014739934 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9980100be0 0x7f998019a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:36:43.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:36:43.050+0000 7f997f59e700 1 -- 172.21.15.12:0/4014739934 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9978009d00 con 0x7f9980102780 2024-02-18T11:36:43.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:36:43.050+0000 7f997ed9d700 1 --2- 172.21.15.12:0/4014739934 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9980100be0 0x7f998019a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:36:43.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:36:43.050+0000 7f997f59e700 1 --2- 172.21.15.12:0/4014739934 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9980102780 0x7f99801a16e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9978009370 tx=0x7f99780093a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:36:43.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:36:43.050+0000 7f9967fff700 1 -- 172.21.15.12:0/4014739934 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f997800e040 con 0x7f9980102780 2024-02-18T11:36:43.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:36:43.050+0000 7f9967fff700 1 -- 172.21.15.12:0/4014739934 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9978005530 con 0x7f9980102780 2024-02-18T11:36:43.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:36:43.050+0000 7f99850cb700 1 -- 172.21.15.12:0/4014739934 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99801a2200 con 0x7f9980102780 2024-02-18T11:36:43.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:36:43.050+0000 7f9967fff700 1 -- 172.21.15.12:0/4014739934 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9978012b50 con 0x7f9980102780 2024-02-18T11:36:43.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:36:43.050+0000 7f99850cb700 1 -- 172.21.15.12:0/4014739934 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99801a26f0 con 0x7f9980102780 2024-02-18T11:36:43.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:36:43.051+0000 7f99850cb700 1 -- 172.21.15.12:0/4014739934 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9980068970 con 0x7f9980102780 2024-02-18T11:36:43.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:36:43.052+0000 7f9967fff700 1 -- 172.21.15.12:0/4014739934 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) v1 ==== 137245+0+0 (secure 0 0 0) 0x7f9978012cb0 con 0x7f9980102780 2024-02-18T11:36:43.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:36:43.053+0000 7f9967fff700 1 --2- 172.21.15.12:0/4014739934 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f99680a27e0 0x7f99680a4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:36:43.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:36:43.053+0000 7f997ed9d700 1 --2- 172.21.15.12:0/4014739934 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f99680a27e0 0x7f99680a4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:36:43.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:36:43.053+0000 7f9967fff700 1 -- 172.21.15.12:0/4014739934 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5858+0+0 (secure 0 0 0) 0x7f99780cf4f0 con 0x7f9980102780 2024-02-18T11:36:43.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:36:43.053+0000 7f997ed9d700 1 --2- 172.21.15.12:0/4014739934 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f99680a27e0 0x7f99680a4ca0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9980101c40 tx=0x7f9970008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:36:43.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:36:43.057+0000 7f9967fff700 1 -- 172.21.15.12:0/4014739934 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9978098ca0 con 0x7f9980102780 2024-02-18T11:36:43.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:42 smithi188 ceph-mon[36925]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 436 MiB used, 804 GiB / 805 GiB avail 2024-02-18T11:36:43.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:42 smithi188 ceph-mon[36925]: Detected new or changed devices on smithi188 2024-02-18T11:36:43.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:42 smithi188 ceph-mon[36925]: Adjusting osd_memory_target on smithi188 to 5693M 2024-02-18T11:36:43.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:42 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:36:43.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:42 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:36:43.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:42 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:36:43.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:42 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:36:43.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:42 smithi188 ceph-mon[36925]: osdmap e63: 9 total, 9 up, 9 in 2024-02-18T11:36:43.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:42 smithi012 ceph-mon[32460]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 436 MiB used, 804 GiB / 805 GiB avail 2024-02-18T11:36:43.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:42 smithi012 ceph-mon[32460]: Detected new or changed devices on smithi188 2024-02-18T11:36:43.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:42 smithi012 ceph-mon[32460]: Adjusting osd_memory_target on smithi188 to 5693M 2024-02-18T11:36:43.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:42 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:36:43.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:42 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:36:43.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:42 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:36:43.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:42 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:36:43.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:42 smithi012 ceph-mon[32460]: osdmap e63: 9 total, 9 up, 9 in 2024-02-18T11:36:43.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:42 smithi158 ceph-mon[36626]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 436 MiB used, 804 GiB / 805 GiB avail 2024-02-18T11:36:43.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:42 smithi158 ceph-mon[36626]: Detected new or changed devices on smithi188 2024-02-18T11:36:43.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:42 smithi158 ceph-mon[36626]: Adjusting osd_memory_target on smithi188 to 5693M 2024-02-18T11:36:43.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:42 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:36:43.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:42 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:36:43.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:42 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:36:43.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:42 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:36:43.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:42 smithi158 ceph-mon[36626]: osdmap e63: 9 total, 9 up, 9 in 2024-02-18T11:36:43.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:36:43.272+0000 7f99850cb700 1 -- 172.21.15.12:0/4014739934 --> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f998010b0a0 con 0x7f99680a27e0 2024-02-18T11:36:44.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:43 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T11:36:44.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:43 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T11:36:44.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:43 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:36:44.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:43 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T11:36:44.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:43 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T11:36:44.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:43 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:36:44.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:43 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T11:36:44.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:43 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T11:36:44.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:43 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:36:45.119 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:44 smithi188 ceph-mon[36925]: pgmap v199: 1 pgs: 1 remapped+peering; 577 KiB data, 436 MiB used, 804 GiB / 805 GiB avail 2024-02-18T11:36:45.119 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:44 smithi188 ceph-mon[36925]: from='client.14409 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:36:45.119 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:44 smithi188 ceph-mon[36925]: osdmap e64: 9 total, 9 up, 9 in 2024-02-18T11:36:45.124 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:44 smithi012 ceph-mon[32460]: pgmap v199: 1 pgs: 1 remapped+peering; 577 KiB data, 436 MiB used, 804 GiB / 805 GiB avail 2024-02-18T11:36:45.124 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:44 smithi012 ceph-mon[32460]: from='client.14409 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:36:45.124 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:44 smithi012 ceph-mon[32460]: osdmap e64: 9 total, 9 up, 9 in 2024-02-18T11:36:45.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:44 smithi158 ceph-mon[36626]: pgmap v199: 1 pgs: 1 remapped+peering; 577 KiB data, 436 MiB used, 804 GiB / 805 GiB avail 2024-02-18T11:36:45.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:44 smithi158 ceph-mon[36626]: from='client.14409 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:36:45.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:44 smithi158 ceph-mon[36626]: osdmap e64: 9 total, 9 up, 9 in 2024-02-18T11:36:46.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:45 smithi012 ceph-mon[32460]: osdmap e65: 9 total, 9 up, 9 in 2024-02-18T11:36:46.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:45 smithi158 ceph-mon[36626]: osdmap e65: 9 total, 9 up, 9 in 2024-02-18T11:36:46.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:45 smithi188 ceph-mon[36925]: osdmap e65: 9 total, 9 up, 9 in 2024-02-18T11:36:47.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:46 smithi158 ceph-mon[36626]: pgmap v202: 1 pgs: 1 remapped+peering; 577 KiB data, 436 MiB used, 804 GiB / 805 GiB avail 2024-02-18T11:36:47.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:46 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/268798633' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b12e2727-d649-4464-90e1-fafbc1754832"}]: dispatch 2024-02-18T11:36:47.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:46 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/268798633' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b12e2727-d649-4464-90e1-fafbc1754832"}]': finished 2024-02-18T11:36:47.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:46 smithi158 ceph-mon[36626]: osdmap e66: 10 total, 9 up, 10 in 2024-02-18T11:36:47.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:46 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-18T11:36:47.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:46 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/1811844803' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T11:36:47.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:46 smithi188 ceph-mon[36925]: pgmap v202: 1 pgs: 1 remapped+peering; 577 KiB data, 436 MiB used, 804 GiB / 805 GiB avail 2024-02-18T11:36:47.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:46 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/268798633' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b12e2727-d649-4464-90e1-fafbc1754832"}]: dispatch 2024-02-18T11:36:47.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:46 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/268798633' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b12e2727-d649-4464-90e1-fafbc1754832"}]': finished 2024-02-18T11:36:47.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:46 smithi188 ceph-mon[36925]: osdmap e66: 10 total, 9 up, 10 in 2024-02-18T11:36:47.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:46 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-18T11:36:47.355 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:46 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/1811844803' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T11:36:47.423 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:46 smithi012 ceph-mon[32460]: pgmap v202: 1 pgs: 1 remapped+peering; 577 KiB data, 436 MiB used, 804 GiB / 805 GiB avail 2024-02-18T11:36:47.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:46 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/268798633' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b12e2727-d649-4464-90e1-fafbc1754832"}]: dispatch 2024-02-18T11:36:47.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:46 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/268798633' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b12e2727-d649-4464-90e1-fafbc1754832"}]': finished 2024-02-18T11:36:47.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:46 smithi012 ceph-mon[32460]: osdmap e66: 10 total, 9 up, 10 in 2024-02-18T11:36:47.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:46 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-18T11:36:47.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:46 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/1811844803' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T11:36:48.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:47 smithi158 ceph-mon[36626]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail; 129 KiB/s, 0 objects/s recovering 2024-02-18T11:36:48.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:47 smithi188 ceph-mon[36925]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail; 129 KiB/s, 0 objects/s recovering 2024-02-18T11:36:48.423 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:47 smithi012 ceph-mon[32460]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail; 129 KiB/s, 0 objects/s recovering 2024-02-18T11:36:50.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:50 smithi012 ceph-mon[32460]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-18T11:36:50.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:50 smithi188 ceph-mon[36925]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-18T11:36:50.495 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:50 smithi158 ceph-mon[36626]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-18T11:36:52.294 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:52 smithi012 ceph-mon[32460]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-02-18T11:36:52.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:52 smithi188 ceph-mon[36925]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-02-18T11:36:52.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:52 smithi158 ceph-mon[36626]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-02-18T11:36:54.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:54 smithi188 ceph-mon[36925]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-02-18T11:36:54.355 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:54 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-02-18T11:36:54.355 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:54 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:36:54.355 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:54 smithi188 ceph-mon[36925]: Deploying daemon osd.9 on smithi012 2024-02-18T11:36:54.370 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:54 smithi012 ceph-mon[32460]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-02-18T11:36:54.370 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:54 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-02-18T11:36:54.370 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:54 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:36:54.370 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:54 smithi012 ceph-mon[32460]: Deploying daemon osd.9 on smithi012 2024-02-18T11:36:54.495 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:54 smithi158 ceph-mon[36626]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-02-18T11:36:54.495 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:54 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-02-18T11:36:54.495 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:54 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:36:54.495 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:54 smithi158 ceph-mon[36626]: Deploying daemon osd.9 on smithi012 2024-02-18T11:36:56.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:56 smithi188 ceph-mon[36925]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-18T11:36:56.372 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:56 smithi012 ceph-mon[32460]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-18T11:36:56.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:56 smithi158 ceph-mon[36626]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-18T11:36:58.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:36:58 smithi012 ceph-mon[32460]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-02-18T11:36:58.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:36:58 smithi188 ceph-mon[36925]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-02-18T11:36:58.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:36:58 smithi158 ceph-mon[36626]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-02-18T11:37:00.423 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:00 smithi012 ceph-mon[32460]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-02-18T11:37:00.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:00 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:00.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:00 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:00.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:00 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:37:00.495 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:00 smithi158 ceph-mon[36626]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-02-18T11:37:00.495 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:00 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:00.495 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:00 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:00.495 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:00 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:37:00.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:00 smithi188 ceph-mon[36925]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-02-18T11:37:00.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:00 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:00.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:00 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:00.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:00 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:37:02.406 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:02 smithi188 ceph-mon[36925]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-02-18T11:37:02.423 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:02 smithi012 ceph-mon[32460]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-02-18T11:37:02.494 INFO:teuthology.orchestra.run.smithi012.stdout:Created osd(s) 9 on host 'smithi012' 2024-02-18T11:37:02.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:02.498+0000 7f9967fff700 1 -- 172.21.15.12:0/4014739934 <== mgr.14152 v2:172.21.15.12:6800/1153736685 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f998010b0a0 con 0x7f99680a27e0 2024-02-18T11:37:02.495 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:02 smithi158 ceph-mon[36626]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-02-18T11:37:02.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:02.500+0000 7f99850cb700 1 -- 172.21.15.12:0/4014739934 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f99680a27e0 msgr2=0x7f99680a4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:37:02.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:02.500+0000 7f99850cb700 1 --2- 172.21.15.12:0/4014739934 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f99680a27e0 0x7f99680a4ca0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9980101c40 tx=0x7f9970008040 comp rx=0 tx=0).stop 2024-02-18T11:37:02.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:02.500+0000 7f99850cb700 1 -- 172.21.15.12:0/4014739934 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9980102780 msgr2=0x7f99801a16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:37:02.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:02.500+0000 7f99850cb700 1 --2- 172.21.15.12:0/4014739934 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9980102780 0x7f99801a16e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9978009370 tx=0x7f99780093a0 comp rx=0 tx=0).stop 2024-02-18T11:37:02.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:02.500+0000 7f99850cb700 1 -- 172.21.15.12:0/4014739934 shutdown_connections 2024-02-18T11:37:02.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:02.500+0000 7f99850cb700 1 --2- 172.21.15.12:0/4014739934 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f99680a27e0 0x7f99680a4ca0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:37:02.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:02.500+0000 7f99850cb700 1 --2- 172.21.15.12:0/4014739934 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9980102780 0x7f99801a16e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:37:02.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:02.500+0000 7f99850cb700 1 --2- 172.21.15.12:0/4014739934 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9980100be0 0x7f998019a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:37:02.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:02.500+0000 7f99850cb700 1 --2- 172.21.15.12:0/4014739934 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9980101de0 0x7f998019a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:37:02.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:02.500+0000 7f99850cb700 1 -- 172.21.15.12:0/4014739934 >> 172.21.15.12:0/4014739934 conn(0x7f99800fc1a0 msgr2=0x7f9980109980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:37:02.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:02.500+0000 7f99850cb700 1 -- 172.21.15.12:0/4014739934 shutdown_connections 2024-02-18T11:37:02.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:02.500+0000 7f99850cb700 1 -- 172.21.15.12:0/4014739934 wait complete. 2024-02-18T11:37:03.086 DEBUG:teuthology.orchestra.run.smithi012:osd.9> sudo journalctl -f -n 0 -u ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@osd.9.service 2024-02-18T11:37:03.089 INFO:tasks.cephadm:Deploying osd.10 on smithi158 with /dev/vg_nvme/lv_1... 2024-02-18T11:37:03.089 DEBUG:teuthology.orchestra.run.smithi158:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-02-18T11:37:03.337 INFO:teuthology.orchestra.run.smithi158.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T11:37:03.744 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:03 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:03.745 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:03 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:03.745 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:03 smithi158 ceph-mon[36626]: from='osd.9 [v2:172.21.15.12:6826/1825928986,v1:172.21.15.12:6827/1825928986]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-02-18T11:37:03.745 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:03 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:03.745 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:03 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:03.827 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:03 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:03.828 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:03 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:03.828 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:03 smithi012 ceph-mon[32460]: from='osd.9 [v2:172.21.15.12:6826/1825928986,v1:172.21.15.12:6827/1825928986]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-02-18T11:37:03.828 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:03 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:03.828 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:03 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:03.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:03 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:03.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:03 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:03.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:03 smithi188 ceph-mon[36925]: from='osd.9 [v2:172.21.15.12:6826/1825928986,v1:172.21.15.12:6827/1825928986]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-02-18T11:37:03.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:03 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:03.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:03 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:04.709 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:04 smithi012 ceph-mon[32460]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-02-18T11:37:04.709 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:04 smithi012 ceph-mon[32460]: from='osd.9 [v2:172.21.15.12:6826/1825928986,v1:172.21.15.12:6827/1825928986]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-02-18T11:37:04.709 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:04 smithi012 ceph-mon[32460]: osdmap e67: 10 total, 9 up, 10 in 2024-02-18T11:37:04.709 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:04 smithi012 ceph-mon[32460]: from='osd.9 [v2:172.21.15.12:6826/1825928986,v1:172.21.15.12:6827/1825928986]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-02-18T11:37:04.709 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:04 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-18T11:37:04.710 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:37:04 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:37:04.475+0000 7f7a73bea700 -1 osd.9 0 waiting for initial osdmap 2024-02-18T11:37:04.710 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:37:04 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:37:04.490+0000 7f7a701de700 -1 osd.9 68 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-18T11:37:04.744 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:04 smithi158 ceph-mon[36626]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-02-18T11:37:04.745 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:04 smithi158 ceph-mon[36626]: from='osd.9 [v2:172.21.15.12:6826/1825928986,v1:172.21.15.12:6827/1825928986]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-02-18T11:37:04.745 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:04 smithi158 ceph-mon[36626]: osdmap e67: 10 total, 9 up, 10 in 2024-02-18T11:37:04.745 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:04 smithi158 ceph-mon[36626]: from='osd.9 [v2:172.21.15.12:6826/1825928986,v1:172.21.15.12:6827/1825928986]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-02-18T11:37:04.745 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:04 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-18T11:37:04.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:04 smithi188 ceph-mon[36925]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-02-18T11:37:04.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:04 smithi188 ceph-mon[36925]: from='osd.9 [v2:172.21.15.12:6826/1825928986,v1:172.21.15.12:6827/1825928986]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-02-18T11:37:04.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:04 smithi188 ceph-mon[36925]: osdmap e67: 10 total, 9 up, 10 in 2024-02-18T11:37:04.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:04 smithi188 ceph-mon[36925]: from='osd.9 [v2:172.21.15.12:6826/1825928986,v1:172.21.15.12:6827/1825928986]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-02-18T11:37:04.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:04 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-18T11:37:05.292 INFO:teuthology.orchestra.run.smithi158.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-02-18T11:37:05.292 INFO:teuthology.orchestra.run.smithi158.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-02-18T11:37:05.292 INFO:teuthology.orchestra.run.smithi158.stderr: stderr: 10+0 records in 2024-02-18T11:37:05.292 INFO:teuthology.orchestra.run.smithi158.stderr:10+0 records out 2024-02-18T11:37:05.293 INFO:teuthology.orchestra.run.smithi158.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00837506 s, 1.3 GB/s 2024-02-18T11:37:05.293 INFO:teuthology.orchestra.run.smithi158.stderr:--> Zapping successful for: 2024-02-18T11:37:05.520 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:05 smithi158 ceph-mon[36626]: from='osd.9 [v2:172.21.15.12:6826/1825928986,v1:172.21.15.12:6827/1825928986]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-02-18T11:37:05.520 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:05 smithi158 ceph-mon[36626]: osdmap e68: 10 total, 9 up, 10 in 2024-02-18T11:37:05.520 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:05 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-18T11:37:05.520 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:05 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-18T11:37:05.816 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:05 smithi012 ceph-mon[32460]: from='osd.9 [v2:172.21.15.12:6826/1825928986,v1:172.21.15.12:6827/1825928986]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-02-18T11:37:05.816 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:05 smithi012 ceph-mon[32460]: osdmap e68: 10 total, 9 up, 10 in 2024-02-18T11:37:05.816 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:05 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-18T11:37:05.816 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:05 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-18T11:37:05.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:05 smithi188 ceph-mon[36925]: from='osd.9 [v2:172.21.15.12:6826/1825928986,v1:172.21.15.12:6827/1825928986]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-02-18T11:37:05.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:05 smithi188 ceph-mon[36925]: osdmap e68: 10 total, 9 up, 10 in 2024-02-18T11:37:05.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:05 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-18T11:37:05.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:05 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-18T11:37:06.019 DEBUG:teuthology.orchestra.run.smithi158:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi158:vg_nvme/lv_1 2024-02-18T11:37:06.261 INFO:teuthology.orchestra.run.smithi158.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T11:37:06.711 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:06 smithi158 ceph-mon[36626]: purged_snaps scrub starts 2024-02-18T11:37:06.711 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:06 smithi158 ceph-mon[36626]: purged_snaps scrub ok 2024-02-18T11:37:06.711 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:06 smithi158 ceph-mon[36626]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-02-18T11:37:06.711 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:06 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-18T11:37:06.711 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:06 smithi158 ceph-mon[36626]: osd.9 [v2:172.21.15.12:6826/1825928986,v1:172.21.15.12:6827/1825928986] boot 2024-02-18T11:37:06.711 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:06 smithi158 ceph-mon[36626]: osdmap e69: 10 total, 10 up, 10 in 2024-02-18T11:37:06.711 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:06 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-18T11:37:06.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:06 smithi188 ceph-mon[36925]: purged_snaps scrub starts 2024-02-18T11:37:06.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:06 smithi188 ceph-mon[36925]: purged_snaps scrub ok 2024-02-18T11:37:06.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:06 smithi188 ceph-mon[36925]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-02-18T11:37:06.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:06 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-18T11:37:06.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:06 smithi188 ceph-mon[36925]: osd.9 [v2:172.21.15.12:6826/1825928986,v1:172.21.15.12:6827/1825928986] boot 2024-02-18T11:37:06.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:06 smithi188 ceph-mon[36925]: osdmap e69: 10 total, 10 up, 10 in 2024-02-18T11:37:06.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:06 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-18T11:37:06.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:06 smithi012 ceph-mon[32460]: purged_snaps scrub starts 2024-02-18T11:37:06.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:06 smithi012 ceph-mon[32460]: purged_snaps scrub ok 2024-02-18T11:37:06.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:06 smithi012 ceph-mon[32460]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-02-18T11:37:06.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:06 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-18T11:37:06.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:06 smithi012 ceph-mon[32460]: osd.9 [v2:172.21.15.12:6826/1825928986,v1:172.21.15.12:6827/1825928986] boot 2024-02-18T11:37:06.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:06 smithi012 ceph-mon[32460]: osdmap e69: 10 total, 10 up, 10 in 2024-02-18T11:37:06.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:06 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-18T11:37:07.601 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:07 smithi158 ceph-mon[36626]: osdmap e70: 10 total, 10 up, 10 in 2024-02-18T11:37:07.601 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:07 smithi158 ceph-mon[36626]: Detected new or changed devices on smithi012 2024-02-18T11:37:07.601 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:07 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:07.601 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:07 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:07.601 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:07 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:37:07.601 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:07 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:37:07.602 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:07 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:37:07.602 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:07 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:37:07.602 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:07 smithi158 ceph-mon[36626]: Adjusting osd_memory_target on smithi012 to 4273M 2024-02-18T11:37:07.602 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:07 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:07.602 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:07 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:37:07.602 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:07 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:37:07.602 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:07 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:07.602 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:07 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:37:07.602 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:07 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:37:07.602 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:07 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:37:07.602 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:07 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:07.851 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:37:07.854+0000 7f2004943700 1 -- 172.21.15.158:0/3683638840 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f20001050f0 msgr2=0x7f20001074e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:37:07.851 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:37:07.854+0000 7f2004943700 1 --2- 172.21.15.158:0/3683638840 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f20001050f0 0x7f20001074e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1ff4009990 tx=0x7f1ff4009ca0 comp rx=0 tx=0).stop 2024-02-18T11:37:07.851 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:37:07.854+0000 7f2004943700 1 -- 172.21.15.158:0/3683638840 shutdown_connections 2024-02-18T11:37:07.851 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:37:07.854+0000 7f2004943700 1 --2- 172.21.15.158:0/3683638840 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2000107a20 0x7f2000109e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:37:07.852 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:37:07.854+0000 7f2004943700 1 --2- 172.21.15.158:0/3683638840 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20001027c0 0x7f2000104bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:37:07.852 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:37:07.854+0000 7f2004943700 1 --2- 172.21.15.158:0/3683638840 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f20001050f0 0x7f20001074e0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:37:07.852 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:37:07.854+0000 7f2004943700 1 -- 172.21.15.158:0/3683638840 >> 172.21.15.158:0/3683638840 conn(0x7f20000fc1b0 msgr2=0x7f20000fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:37:07.852 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:37:07.854+0000 7f2004943700 1 -- 172.21.15.158:0/3683638840 shutdown_connections 2024-02-18T11:37:07.852 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:37:07.854+0000 7f2004943700 1 -- 172.21.15.158:0/3683638840 wait complete. 2024-02-18T11:37:07.852 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:37:07.854+0000 7f2004943700 1 Processor -- start 2024-02-18T11:37:07.852 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:37:07.854+0000 7f2004943700 1 -- start start 2024-02-18T11:37:07.852 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:37:07.855+0000 7f2004943700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20001027c0 0x7f20001020d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:37:07.853 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:37:07.855+0000 7f2004943700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20001050f0 0x7f2000100770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:37:07.853 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:37:07.855+0000 7f2004943700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2000107a20 0x7f2000100cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:37:07.853 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:37:07.855+0000 7f2004943700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2000101450 con 0x7f20001027c0 2024-02-18T11:37:07.853 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:37:07.855+0000 7f2004943700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20001015c0 con 0x7f2000107a20 2024-02-18T11:37:07.853 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:37:07.855+0000 7f2004943700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20001a1d60 con 0x7f20001050f0 2024-02-18T11:37:07.853 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:37:07.855+0000 7f1ffed9d700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2000107a20 0x7f2000100cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:37:07.853 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:37:07.855+0000 7f1ffed9d700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2000107a20 0x7f2000100cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.158:3300/0 says I am v2:172.21.15.158:35862/0 (socket says 172.21.15.158:35862) 2024-02-18T11:37:07.853 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:37:07.855+0000 7f1ffed9d700 1 -- 172.21.15.158:0/3132434109 learned_addr learned my addr 172.21.15.158:0/3132434109 (peer_addr_for_me v2:172.21.15.158:0/0) 2024-02-18T11:37:07.854 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:37:07.855+0000 7f1ffe59c700 1 --2- 172.21.15.158:0/3132434109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20001027c0 0x7f20001020d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:37:07.854 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:37:07.855+0000 7f1ffdd9b700 1 --2- 172.21.15.158:0/3132434109 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20001050f0 0x7f2000100770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:37:07.854 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:37:07.855+0000 7f1ffed9d700 1 -- 172.21.15.158:0/3132434109 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20001050f0 msgr2=0x7f2000100770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:37:07.854 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:37:07.855+0000 7f1ffed9d700 1 --2- 172.21.15.158:0/3132434109 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20001050f0 0x7f2000100770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:37:07.854 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:37:07.855+0000 7f1ffed9d700 1 -- 172.21.15.158:0/3132434109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20001027c0 msgr2=0x7f20001020d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:37:07.854 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:37:07.855+0000 7f1ffed9d700 1 --2- 172.21.15.158:0/3132434109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20001027c0 0x7f20001020d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:37:07.854 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:37:07.855+0000 7f1ffed9d700 1 -- 172.21.15.158:0/3132434109 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ff40095f0 con 0x7f2000107a20 2024-02-18T11:37:07.854 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:37:07.855+0000 7f1ffe59c700 1 --2- 172.21.15.158:0/3132434109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20001027c0 0x7f20001020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:37:07.855 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:37:07.855+0000 7f1ffdd9b700 1 --2- 172.21.15.158:0/3132434109 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20001050f0 0x7f2000100770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:37:07.855 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:37:07.855+0000 7f1ffed9d700 1 --2- 172.21.15.158:0/3132434109 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2000107a20 0x7f2000100cb0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1ff800b9e0 tx=0x7f1ff800bda0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:37:07.855 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:37:07.855+0000 7f1fef7fe700 1 -- 172.21.15.158:0/3132434109 <== mon.2 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff800c9b0 con 0x7f2000107a20 2024-02-18T11:37:07.855 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:37:07.855+0000 7f1fef7fe700 1 -- 172.21.15.158:0/3132434109 <== mon.2 v2:172.21.15.158:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1ff8014440 con 0x7f2000107a20 2024-02-18T11:37:07.855 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:37:07.855+0000 7f2004943700 1 -- 172.21.15.158:0/3132434109 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20001a1ff0 con 0x7f2000107a20 2024-02-18T11:37:07.855 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:37:07.855+0000 7f1fef7fe700 1 -- 172.21.15.158:0/3132434109 <== mon.2 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff80125a0 con 0x7f2000107a20 2024-02-18T11:37:07.855 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:37:07.855+0000 7f2004943700 1 -- 172.21.15.158:0/3132434109 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20001a2460 con 0x7f2000107a20 2024-02-18T11:37:07.855 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:37:07.856+0000 7f2004943700 1 -- 172.21.15.158:0/3132434109 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20000fdd30 con 0x7f2000107a20 2024-02-18T11:37:07.856 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:07 smithi188 ceph-mon[36925]: osdmap e70: 10 total, 10 up, 10 in 2024-02-18T11:37:07.857 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:07 smithi188 ceph-mon[36925]: Detected new or changed devices on smithi012 2024-02-18T11:37:07.857 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:07 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:07.858 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:07 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:07.858 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:07 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:37:07.858 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:07 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:37:07.858 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:07 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:37:07.858 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:07 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:37:07.858 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:07 smithi188 ceph-mon[36925]: Adjusting osd_memory_target on smithi012 to 4273M 2024-02-18T11:37:07.859 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:07 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:07.859 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:07 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:37:07.859 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:07 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:37:07.859 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:07 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:07.859 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:07 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:37:07.859 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:07 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:37:07.859 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:07 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:37:07.860 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:07 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:07.861 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:37:07.858+0000 7f1fef7fe700 1 -- 172.21.15.158:0/3132434109 <== mon.2 v2:172.21.15.158:3300/0 4 ==== mgrmap(e 16) v1 ==== 137245+0+0 (secure 0 0 0) 0x7f1ff80145b0 con 0x7f2000107a20 2024-02-18T11:37:07.861 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:37:07.859+0000 7f1fef7fe700 1 --2- 172.21.15.158:0/3132434109 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f1fe80a27e0 0x7f1fe80a4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:37:07.861 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:37:07.859+0000 7f1fef7fe700 1 -- 172.21.15.158:0/3132434109 <== mon.2 v2:172.21.15.158:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f1ff8006e40 con 0x7f2000107a20 2024-02-18T11:37:07.861 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:37:07.859+0000 7f1ffe59c700 1 --2- 172.21.15.158:0/3132434109 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f1fe80a27e0 0x7f1fe80a4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:37:07.862 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:37:07.860+0000 7f1ffe59c700 1 --2- 172.21.15.158:0/3132434109 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f1fe80a27e0 0x7f1fe80a4ca0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1ff0005fd0 tx=0x7f1ff0005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:37:07.862 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:37:07.862+0000 7f1fef7fe700 1 -- 172.21.15.158:0/3132434109 <== mon.2 v2:172.21.15.158:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1ff8096ba0 con 0x7f2000107a20 2024-02-18T11:37:07.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:07 smithi012 ceph-mon[32460]: osdmap e70: 10 total, 10 up, 10 in 2024-02-18T11:37:07.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:07 smithi012 ceph-mon[32460]: Detected new or changed devices on smithi012 2024-02-18T11:37:07.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:07 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:07.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:07 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:07.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:07 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:37:07.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:07 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:37:07.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:07 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:37:07.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:07 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:37:07.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:07 smithi012 ceph-mon[32460]: Adjusting osd_memory_target on smithi012 to 4273M 2024-02-18T11:37:07.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:07 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:07.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:07 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:37:07.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:07 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:37:07.925 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:07 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:07.925 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:07 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:37:07.925 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:07 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:37:07.925 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:07 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:37:07.925 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:07 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:08.058 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:37:08.063+0000 7f2004943700 1 -- 172.21.15.158:0/3132434109 --> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi158:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f200002d950 con 0x7f1fe80a27e0 2024-02-18T11:37:08.996 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:08 smithi158 ceph-mon[36626]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-18T11:37:08.996 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:08 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T11:37:08.996 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:08 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T11:37:08.996 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:08 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:37:09.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:08 smithi188 ceph-mon[36925]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-18T11:37:09.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:08 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T11:37:09.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:08 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T11:37:09.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:08 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:37:09.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:08 smithi012 ceph-mon[32460]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-18T11:37:09.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:08 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T11:37:09.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:08 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T11:37:09.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:08 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:37:09.956 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:09 smithi158 ceph-mon[36626]: from='client.24317 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi158:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:37:10.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:09 smithi188 ceph-mon[36925]: from='client.24317 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi158:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:37:10.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:09 smithi012 ceph-mon[32460]: from='client.24317 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi158:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:37:11.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:10 smithi188 ceph-mon[36925]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-18T11:37:11.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:10 smithi012 ceph-mon[32460]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-18T11:37:11.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:10 smithi158 ceph-mon[36626]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-18T11:37:12.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:11 smithi188 ceph-mon[36925]: from='client.? 172.21.15.158:0/1319207877' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "518b4cbf-9d90-4e9a-8ac9-b5cb9b3dee4a"}]: dispatch 2024-02-18T11:37:12.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:11 smithi188 ceph-mon[36925]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "518b4cbf-9d90-4e9a-8ac9-b5cb9b3dee4a"}]: dispatch 2024-02-18T11:37:12.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:11 smithi188 ceph-mon[36925]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "518b4cbf-9d90-4e9a-8ac9-b5cb9b3dee4a"}]': finished 2024-02-18T11:37:12.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:11 smithi188 ceph-mon[36925]: osdmap e71: 11 total, 10 up, 11 in 2024-02-18T11:37:12.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:11 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-18T11:37:12.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:11 smithi188 ceph-mon[36925]: from='client.? 172.21.15.158:0/3739875843' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T11:37:12.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:11 smithi012 ceph-mon[32460]: from='client.? 172.21.15.158:0/1319207877' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "518b4cbf-9d90-4e9a-8ac9-b5cb9b3dee4a"}]: dispatch 2024-02-18T11:37:12.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:11 smithi012 ceph-mon[32460]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "518b4cbf-9d90-4e9a-8ac9-b5cb9b3dee4a"}]: dispatch 2024-02-18T11:37:12.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:11 smithi012 ceph-mon[32460]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "518b4cbf-9d90-4e9a-8ac9-b5cb9b3dee4a"}]': finished 2024-02-18T11:37:12.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:11 smithi012 ceph-mon[32460]: osdmap e71: 11 total, 10 up, 11 in 2024-02-18T11:37:12.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:11 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-18T11:37:12.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:11 smithi012 ceph-mon[32460]: from='client.? 172.21.15.158:0/3739875843' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T11:37:12.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:11 smithi158 ceph-mon[36626]: from='client.? 172.21.15.158:0/1319207877' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "518b4cbf-9d90-4e9a-8ac9-b5cb9b3dee4a"}]: dispatch 2024-02-18T11:37:12.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:11 smithi158 ceph-mon[36626]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "518b4cbf-9d90-4e9a-8ac9-b5cb9b3dee4a"}]: dispatch 2024-02-18T11:37:12.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:11 smithi158 ceph-mon[36626]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "518b4cbf-9d90-4e9a-8ac9-b5cb9b3dee4a"}]': finished 2024-02-18T11:37:12.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:11 smithi158 ceph-mon[36626]: osdmap e71: 11 total, 10 up, 11 in 2024-02-18T11:37:12.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:11 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-18T11:37:12.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:11 smithi158 ceph-mon[36626]: from='client.? 172.21.15.158:0/3739875843' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T11:37:13.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:12 smithi188 ceph-mon[36925]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-18T11:37:13.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:12 smithi012 ceph-mon[32460]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-18T11:37:13.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:12 smithi158 ceph-mon[36626]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-18T11:37:14.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:14 smithi158 ceph-mon[36626]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-18T11:37:15.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:14 smithi188 ceph-mon[36925]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-18T11:37:15.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:14 smithi012 ceph-mon[32460]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-18T11:37:17.093 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:16 smithi158 ceph-mon[36626]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-18T11:37:17.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:16 smithi188 ceph-mon[36925]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-18T11:37:17.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:16 smithi012 ceph-mon[32460]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-18T11:37:18.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:18 smithi158 ceph-mon[36626]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-18T11:37:18.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:18 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-02-18T11:37:18.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:18 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:37:19.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:18 smithi188 ceph-mon[36925]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-18T11:37:19.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:18 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-02-18T11:37:19.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:18 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:37:19.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:18 smithi012 ceph-mon[32460]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-18T11:37:19.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:18 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-02-18T11:37:19.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:18 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:37:20.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:19 smithi012 ceph-mon[32460]: Deploying daemon osd.10 on smithi158 2024-02-18T11:37:20.091 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:19 smithi158 ceph-mon[36626]: Deploying daemon osd.10 on smithi158 2024-02-18T11:37:20.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:19 smithi188 ceph-mon[36925]: Deploying daemon osd.10 on smithi158 2024-02-18T11:37:21.089 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:20 smithi012 ceph-mon[32460]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-18T11:37:21.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:20 smithi188 ceph-mon[36925]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-18T11:37:21.239 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:20 smithi158 ceph-mon[36626]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-18T11:37:23.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:22 smithi188 ceph-mon[36925]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-18T11:37:23.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:22 smithi012 ceph-mon[32460]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-18T11:37:23.198 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:22 smithi158 ceph-mon[36626]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-18T11:37:25.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:24 smithi188 ceph-mon[36925]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-18T11:37:25.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:24 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:25.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:24 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:25.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:24 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:37:25.105 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:24 smithi158 ceph-mon[36626]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-18T11:37:25.105 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:24 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:25.105 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:24 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:25.105 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:24 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:37:25.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:24 smithi012 ceph-mon[32460]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-18T11:37:25.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:24 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:25.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:24 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:25.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:24 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:37:27.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:26 smithi188 ceph-mon[36925]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-18T11:37:27.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:26 smithi012 ceph-mon[32460]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-18T11:37:27.238 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:26 smithi158 ceph-mon[36626]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-18T11:37:27.979 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:37:27.984+0000 7f1fef7fe700 1 -- 172.21.15.158:0/3132434109 <== mgr.14152 v2:172.21.15.12:6800/1153736685 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f200002d950 con 0x7f1fe80a27e0 2024-02-18T11:37:27.980 INFO:teuthology.orchestra.run.smithi158.stdout:Created osd(s) 10 on host 'smithi158' 2024-02-18T11:37:27.982 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:37:27.987+0000 7f2004943700 1 -- 172.21.15.158:0/3132434109 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f1fe80a27e0 msgr2=0x7f1fe80a4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:37:27.982 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:37:27.987+0000 7f2004943700 1 --2- 172.21.15.158:0/3132434109 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f1fe80a27e0 0x7f1fe80a4ca0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1ff0005fd0 tx=0x7f1ff0005dc0 comp rx=0 tx=0).stop 2024-02-18T11:37:27.982 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:37:27.987+0000 7f2004943700 1 -- 172.21.15.158:0/3132434109 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2000107a20 msgr2=0x7f2000100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:37:27.982 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:37:27.987+0000 7f2004943700 1 --2- 172.21.15.158:0/3132434109 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2000107a20 0x7f2000100cb0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1ff800b9e0 tx=0x7f1ff800bda0 comp rx=0 tx=0).stop 2024-02-18T11:37:27.983 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:37:27.987+0000 7f2004943700 1 -- 172.21.15.158:0/3132434109 shutdown_connections 2024-02-18T11:37:27.983 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:37:27.987+0000 7f2004943700 1 --2- 172.21.15.158:0/3132434109 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2000107a20 0x7f2000100cb0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:37:27.983 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:37:27.987+0000 7f2004943700 1 --2- 172.21.15.158:0/3132434109 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f1fe80a27e0 0x7f1fe80a4ca0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:37:27.983 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:37:27.987+0000 7f2004943700 1 --2- 172.21.15.158:0/3132434109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20001027c0 0x7f20001020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:37:27.983 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:37:27.987+0000 7f2004943700 1 --2- 172.21.15.158:0/3132434109 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20001050f0 0x7f2000100770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:37:27.984 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:37:27.987+0000 7f2004943700 1 -- 172.21.15.158:0/3132434109 >> 172.21.15.158:0/3132434109 conn(0x7f20000fc1b0 msgr2=0x7f2000103100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:37:27.984 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:37:27.987+0000 7f2004943700 1 -- 172.21.15.158:0/3132434109 shutdown_connections 2024-02-18T11:37:27.984 INFO:teuthology.orchestra.run.smithi158.stderr:2024-02-18T11:37:27.987+0000 7f2004943700 1 -- 172.21.15.158:0/3132434109 wait complete. 2024-02-18T11:37:28.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:28 smithi158 ceph-mon[36626]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-18T11:37:28.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:28 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:28.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:28 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:28.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:28 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:37:28.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:28 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:37:28.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:28 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:28.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:28 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:28.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:28 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:28.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:28 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:37:28.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:28 smithi158 ceph-mon[36626]: from='osd.10 [v2:172.21.15.158:6824/3239052167,v1:172.21.15.158:6825/3239052167]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-02-18T11:37:28.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:28 smithi158 ceph-mon[36626]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-02-18T11:37:28.423 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:28 smithi012 ceph-mon[32460]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-18T11:37:28.423 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:28 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:28.423 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:28 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:28.423 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:28 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:37:28.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:28 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:37:28.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:28 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:28.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:28 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:28.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:28 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:28.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:28 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:37:28.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:28 smithi012 ceph-mon[32460]: from='osd.10 [v2:172.21.15.158:6824/3239052167,v1:172.21.15.158:6825/3239052167]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-02-18T11:37:28.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:28 smithi012 ceph-mon[32460]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-02-18T11:37:28.595 DEBUG:teuthology.orchestra.run.smithi158:osd.10> sudo journalctl -f -n 0 -u ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@osd.10.service 2024-02-18T11:37:28.598 INFO:tasks.cephadm:Deploying osd.11 on smithi188 with /dev/vg_nvme/lv_1... 2024-02-18T11:37:28.598 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-02-18T11:37:28.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:28 smithi188 ceph-mon[36925]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-18T11:37:28.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:28 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:28.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:28 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:28.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:28 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:37:28.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:28 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:37:28.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:28 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:28.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:28 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:28.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:28 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:28.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:28 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:37:28.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:28 smithi188 ceph-mon[36925]: from='osd.10 [v2:172.21.15.158:6824/3239052167,v1:172.21.15.158:6825/3239052167]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-02-18T11:37:28.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:28 smithi188 ceph-mon[36925]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-02-18T11:37:28.844 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.c/config 2024-02-18T11:37:29.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:29 smithi158 ceph-mon[36626]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-02-18T11:37:29.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:29 smithi158 ceph-mon[36626]: from='osd.10 [v2:172.21.15.158:6824/3239052167,v1:172.21.15.158:6825/3239052167]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi158", "root=default"]}]: dispatch 2024-02-18T11:37:29.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:29 smithi158 ceph-mon[36626]: osdmap e72: 11 total, 10 up, 11 in 2024-02-18T11:37:29.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:29 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-18T11:37:29.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:29 smithi158 ceph-mon[36626]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi158", "root=default"]}]: dispatch 2024-02-18T11:37:29.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:29 smithi158 ceph-mon[36626]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-18T11:37:30.311 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:29 smithi188 ceph-mon[36925]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-02-18T11:37:30.311 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:29 smithi188 ceph-mon[36925]: from='osd.10 [v2:172.21.15.158:6824/3239052167,v1:172.21.15.158:6825/3239052167]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi158", "root=default"]}]: dispatch 2024-02-18T11:37:30.311 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:29 smithi188 ceph-mon[36925]: osdmap e72: 11 total, 10 up, 11 in 2024-02-18T11:37:30.311 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:29 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-18T11:37:30.311 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:29 smithi188 ceph-mon[36925]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi158", "root=default"]}]: dispatch 2024-02-18T11:37:30.311 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:29 smithi188 ceph-mon[36925]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-18T11:37:30.423 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:29 smithi012 ceph-mon[32460]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-02-18T11:37:30.423 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:29 smithi012 ceph-mon[32460]: from='osd.10 [v2:172.21.15.158:6824/3239052167,v1:172.21.15.158:6825/3239052167]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi158", "root=default"]}]: dispatch 2024-02-18T11:37:30.423 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:29 smithi012 ceph-mon[32460]: osdmap e72: 11 total, 10 up, 11 in 2024-02-18T11:37:30.423 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:29 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-18T11:37:30.423 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:29 smithi012 ceph-mon[32460]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi158", "root=default"]}]: dispatch 2024-02-18T11:37:30.423 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:29 smithi012 ceph-mon[32460]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-18T11:37:30.883 INFO:teuthology.orchestra.run.smithi188.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-02-18T11:37:30.883 INFO:teuthology.orchestra.run.smithi188.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-02-18T11:37:30.884 INFO:teuthology.orchestra.run.smithi188.stderr: stderr: 10+0 records in 2024-02-18T11:37:30.884 INFO:teuthology.orchestra.run.smithi188.stderr:10+0 records out 2024-02-18T11:37:30.884 INFO:teuthology.orchestra.run.smithi188.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113812 s, 921 MB/s 2024-02-18T11:37:30.884 INFO:teuthology.orchestra.run.smithi188.stderr:--> Zapping successful for: 2024-02-18T11:37:30.949 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:37:30 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:37:30.703+0000 7f56e2f33700 -1 osd.10 0 waiting for initial osdmap 2024-02-18T11:37:30.949 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:37:30 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:37:30.718+0000 7f56db51f700 -1 osd.10 73 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-18T11:37:31.215 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:30 smithi188 ceph-mon[36925]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi158", "root=default"]}]': finished 2024-02-18T11:37:31.215 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:30 smithi188 ceph-mon[36925]: osdmap e73: 11 total, 10 up, 11 in 2024-02-18T11:37:31.215 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:30 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-18T11:37:31.215 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:30 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-18T11:37:31.215 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:30 smithi188 ceph-mon[36925]: from='osd.10 ' entity='osd.10' 2024-02-18T11:37:31.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:30 smithi158 ceph-mon[36626]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi158", "root=default"]}]': finished 2024-02-18T11:37:31.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:30 smithi158 ceph-mon[36626]: osdmap e73: 11 total, 10 up, 11 in 2024-02-18T11:37:31.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:30 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-18T11:37:31.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:30 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-18T11:37:31.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:30 smithi158 ceph-mon[36626]: from='osd.10 ' entity='osd.10' 2024-02-18T11:37:31.423 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:30 smithi012 ceph-mon[32460]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi158", "root=default"]}]': finished 2024-02-18T11:37:31.423 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:30 smithi012 ceph-mon[32460]: osdmap e73: 11 total, 10 up, 11 in 2024-02-18T11:37:31.423 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:30 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-18T11:37:31.423 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:30 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-18T11:37:31.423 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:30 smithi012 ceph-mon[32460]: from='osd.10 ' entity='osd.10' 2024-02-18T11:37:31.593 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi188:vg_nvme/lv_1 2024-02-18T11:37:31.888 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.c/config 2024-02-18T11:37:32.124 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:31 smithi188 ceph-mon[36925]: purged_snaps scrub starts 2024-02-18T11:37:32.124 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:31 smithi188 ceph-mon[36925]: purged_snaps scrub ok 2024-02-18T11:37:32.124 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:31 smithi188 ceph-mon[36925]: osd.10 [v2:172.21.15.158:6824/3239052167,v1:172.21.15.158:6825/3239052167] boot 2024-02-18T11:37:32.124 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:31 smithi188 ceph-mon[36925]: osdmap e74: 11 total, 11 up, 11 in 2024-02-18T11:37:32.124 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:31 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-18T11:37:32.124 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:31 smithi188 ceph-mon[36925]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 555 MiB used, 983 GiB / 983 GiB avail 2024-02-18T11:37:32.124 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:31 smithi188 ceph-mon[36925]: osdmap e75: 11 total, 11 up, 11 in 2024-02-18T11:37:32.125 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:31 smithi188 ceph-mon[36925]: Detected new or changed devices on smithi158 2024-02-18T11:37:32.125 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:31 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:32.125 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:31 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:32.125 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:31 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:37:32.125 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:31 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:37:32.125 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:31 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:37:32.125 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:31 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:37:32.125 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:31 smithi188 ceph-mon[36925]: Adjusting osd_memory_target on smithi158 to 4270M 2024-02-18T11:37:32.125 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:31 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:32.125 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:31 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:37:32.125 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:31 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:37:32.125 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:31 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:32.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:31 smithi012 ceph-mon[32460]: purged_snaps scrub starts 2024-02-18T11:37:32.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:31 smithi012 ceph-mon[32460]: purged_snaps scrub ok 2024-02-18T11:37:32.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:31 smithi012 ceph-mon[32460]: osd.10 [v2:172.21.15.158:6824/3239052167,v1:172.21.15.158:6825/3239052167] boot 2024-02-18T11:37:32.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:31 smithi012 ceph-mon[32460]: osdmap e74: 11 total, 11 up, 11 in 2024-02-18T11:37:32.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:31 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-18T11:37:32.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:31 smithi012 ceph-mon[32460]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 555 MiB used, 983 GiB / 983 GiB avail 2024-02-18T11:37:32.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:31 smithi012 ceph-mon[32460]: osdmap e75: 11 total, 11 up, 11 in 2024-02-18T11:37:32.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:31 smithi012 ceph-mon[32460]: Detected new or changed devices on smithi158 2024-02-18T11:37:32.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:31 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:32.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:31 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:32.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:31 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:37:32.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:31 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:37:32.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:31 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:37:32.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:31 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:37:32.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:31 smithi012 ceph-mon[32460]: Adjusting osd_memory_target on smithi158 to 4270M 2024-02-18T11:37:32.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:31 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:32.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:31 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:37:32.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:31 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:37:32.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:31 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:32.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:31 smithi158 ceph-mon[36626]: purged_snaps scrub starts 2024-02-18T11:37:32.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:31 smithi158 ceph-mon[36626]: purged_snaps scrub ok 2024-02-18T11:37:32.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:31 smithi158 ceph-mon[36626]: osd.10 [v2:172.21.15.158:6824/3239052167,v1:172.21.15.158:6825/3239052167] boot 2024-02-18T11:37:32.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:31 smithi158 ceph-mon[36626]: osdmap e74: 11 total, 11 up, 11 in 2024-02-18T11:37:32.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:31 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-18T11:37:32.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:31 smithi158 ceph-mon[36626]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 555 MiB used, 983 GiB / 983 GiB avail 2024-02-18T11:37:32.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:31 smithi158 ceph-mon[36626]: osdmap e75: 11 total, 11 up, 11 in 2024-02-18T11:37:32.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:31 smithi158 ceph-mon[36626]: Detected new or changed devices on smithi158 2024-02-18T11:37:32.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:31 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:32.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:31 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:32.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:31 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:37:32.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:31 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:37:32.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:31 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:37:32.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:31 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:37:32.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:31 smithi158 ceph-mon[36626]: Adjusting osd_memory_target on smithi158 to 4270M 2024-02-18T11:37:32.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:31 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:32.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:31 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:37:32.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:31 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:37:32.246 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:31 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:33.566 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:37:33.569+0000 7ff03fccc700 1 -- 172.21.15.188:0/3230860322 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff038100be0 msgr2=0x7ff038101000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:37:33.566 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:37:33.569+0000 7ff03fccc700 1 --2- 172.21.15.188:0/3230860322 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff038100be0 0x7ff038101000 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff034009990 tx=0x7ff034009ca0 comp rx=0 tx=0).stop 2024-02-18T11:37:33.566 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:37:33.569+0000 7ff03fccc700 1 -- 172.21.15.188:0/3230860322 shutdown_connections 2024-02-18T11:37:33.566 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:37:33.569+0000 7ff03fccc700 1 --2- 172.21.15.188:0/3230860322 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff038102780 0x7ff038109030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:37:33.567 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:37:33.569+0000 7ff03fccc700 1 --2- 172.21.15.188:0/3230860322 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff038100be0 0x7ff038101000 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:37:33.567 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:37:33.569+0000 7ff03fccc700 1 --2- 172.21.15.188:0/3230860322 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff038101de0 0x7ff038102240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:37:33.567 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:37:33.569+0000 7ff03fccc700 1 -- 172.21.15.188:0/3230860322 >> 172.21.15.188:0/3230860322 conn(0x7ff0380fc180 msgr2=0x7ff0380fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:37:33.567 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:37:33.569+0000 7ff03fccc700 1 -- 172.21.15.188:0/3230860322 shutdown_connections 2024-02-18T11:37:33.567 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:37:33.569+0000 7ff03fccc700 1 -- 172.21.15.188:0/3230860322 wait complete. 2024-02-18T11:37:33.567 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:37:33.569+0000 7ff03fccc700 1 Processor -- start 2024-02-18T11:37:33.567 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:37:33.569+0000 7ff03fccc700 1 -- start start 2024-02-18T11:37:33.567 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:37:33.570+0000 7ff03fccc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff038100be0 0x7ff03806a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:37:33.567 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:37:33.570+0000 7ff03fccc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff038101de0 0x7ff03819ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:37:33.567 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:37:33.570+0000 7ff03fccc700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff038102780 0x7ff03819f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:37:33.567 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:37:33.570+0000 7ff03fccc700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff03819f840 con 0x7ff038100be0 2024-02-18T11:37:33.567 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:37:33.570+0000 7ff03fccc700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff03819f9b0 con 0x7ff038102780 2024-02-18T11:37:33.567 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:37:33.570+0000 7ff03fccc700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff03819fb70 con 0x7ff038101de0 2024-02-18T11:37:33.568 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:37:33.570+0000 7ff03d267700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff038101de0 0x7ff03819ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:37:33.568 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:37:33.570+0000 7ff03d267700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff038101de0 0x7ff03819ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:35542/0 (socket says 172.21.15.188:35542) 2024-02-18T11:37:33.568 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:37:33.570+0000 7ff03d267700 1 -- 172.21.15.188:0/34156981 learned_addr learned my addr 172.21.15.188:0/34156981 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-02-18T11:37:33.568 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:37:33.570+0000 7ff03da68700 1 --2- 172.21.15.188:0/34156981 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff038100be0 0x7ff03806a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:37:33.568 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:37:33.570+0000 7ff03d267700 1 -- 172.21.15.188:0/34156981 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff038102780 msgr2=0x7ff03819f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:37:33.568 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:37:33.570+0000 7ff03e269700 1 --2- 172.21.15.188:0/34156981 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff038102780 0x7ff03819f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:37:33.568 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:37:33.570+0000 7ff03d267700 1 --2- 172.21.15.188:0/34156981 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff038102780 0x7ff03819f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:37:33.568 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:37:33.570+0000 7ff03d267700 1 -- 172.21.15.188:0/34156981 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff038100be0 msgr2=0x7ff03806a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:37:33.569 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:37:33.570+0000 7ff03d267700 1 --2- 172.21.15.188:0/34156981 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff038100be0 0x7ff03806a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:37:33.569 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:37:33.570+0000 7ff03d267700 1 -- 172.21.15.188:0/34156981 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0340095f0 con 0x7ff038101de0 2024-02-18T11:37:33.569 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:37:33.570+0000 7ff03da68700 1 --2- 172.21.15.188:0/34156981 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff038100be0 0x7ff03806a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:37:33.569 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:37:33.570+0000 7ff03e269700 1 --2- 172.21.15.188:0/34156981 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff038102780 0x7ff03819f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:37:33.569 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:37:33.570+0000 7ff03d267700 1 --2- 172.21.15.188:0/34156981 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff038101de0 0x7ff03819ca60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff02800d950 tx=0x7ff02800dd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:37:33.569 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:37:33.570+0000 7ff02effd700 1 -- 172.21.15.188:0/34156981 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff028009b40 con 0x7ff038101de0 2024-02-18T11:37:33.569 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:37:33.570+0000 7ff03fccc700 1 -- 172.21.15.188:0/34156981 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff03819fe50 con 0x7ff038101de0 2024-02-18T11:37:33.569 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:37:33.570+0000 7ff02effd700 1 -- 172.21.15.188:0/34156981 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff028009ca0 con 0x7ff038101de0 2024-02-18T11:37:33.569 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:37:33.570+0000 7ff02effd700 1 -- 172.21.15.188:0/34156981 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02800f6b0 con 0x7ff038101de0 2024-02-18T11:37:33.569 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:37:33.570+0000 7ff03fccc700 1 -- 172.21.15.188:0/34156981 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0381a02c0 con 0x7ff038101de0 2024-02-18T11:37:33.569 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:37:33.571+0000 7ff03fccc700 1 -- 172.21.15.188:0/34156981 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0381080c0 con 0x7ff038101de0 2024-02-18T11:37:33.570 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:37:33.572+0000 7ff02effd700 1 -- 172.21.15.188:0/34156981 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 16) v1 ==== 137245+0+0 (secure 0 0 0) 0x7ff028010460 con 0x7ff038101de0 2024-02-18T11:37:33.570 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:37:33.573+0000 7ff02effd700 1 --2- 172.21.15.188:0/34156981 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7ff0240a27e0 0x7ff0240a4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:37:33.570 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:37:33.573+0000 7ff02effd700 1 -- 172.21.15.188:0/34156981 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6694+0+0 (secure 0 0 0) 0x7ff0280ce330 con 0x7ff038101de0 2024-02-18T11:37:33.570 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:37:33.574+0000 7ff03da68700 1 --2- 172.21.15.188:0/34156981 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7ff0240a27e0 0x7ff0240a4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:37:33.570 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:37:33.574+0000 7ff03da68700 1 --2- 172.21.15.188:0/34156981 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7ff0240a27e0 0x7ff0240a4ca0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff03400b5c0 tx=0x7ff034005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:37:33.571 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:37:33.577+0000 7ff02effd700 1 -- 172.21.15.188:0/34156981 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff0280976f0 con 0x7ff038101de0 2024-02-18T11:37:33.773 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:37:33.778+0000 7ff03fccc700 1 -- 172.21.15.188:0/34156981 --> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7ff03802d8b0 con 0x7ff0240a27e0 2024-02-18T11:37:33.773 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:33 smithi188 ceph-mon[36925]: osdmap e76: 11 total, 11 up, 11 in 2024-02-18T11:37:33.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:33 smithi012 ceph-mon[32460]: osdmap e76: 11 total, 11 up, 11 in 2024-02-18T11:37:33.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:33 smithi158 ceph-mon[36626]: osdmap e76: 11 total, 11 up, 11 in 2024-02-18T11:37:34.518 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:34 smithi158 ceph-mon[36626]: pgmap v237: 1 pgs: 1 remapped+peering; 577 KiB data, 556 MiB used, 983 GiB / 983 GiB avail 2024-02-18T11:37:34.647 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:34 smithi188 ceph-mon[36925]: pgmap v237: 1 pgs: 1 remapped+peering; 577 KiB data, 556 MiB used, 983 GiB / 983 GiB avail 2024-02-18T11:37:34.647 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:34 smithi188 ceph-mon[36925]: osdmap e77: 11 total, 11 up, 11 in 2024-02-18T11:37:34.647 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:34 smithi188 ceph-mon[36925]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:37:34.647 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:34 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T11:37:34.648 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:34 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T11:37:34.648 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:34 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:37:34.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:34 smithi012 ceph-mon[32460]: pgmap v237: 1 pgs: 1 remapped+peering; 577 KiB data, 556 MiB used, 983 GiB / 983 GiB avail 2024-02-18T11:37:34.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:34 smithi012 ceph-mon[32460]: osdmap e77: 11 total, 11 up, 11 in 2024-02-18T11:37:34.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:34 smithi012 ceph-mon[32460]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:37:34.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:34 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T11:37:34.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:34 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T11:37:34.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:34 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:37:34.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:34 smithi158 ceph-mon[36626]: osdmap e77: 11 total, 11 up, 11 in 2024-02-18T11:37:34.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:34 smithi158 ceph-mon[36626]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:37:34.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:34 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T11:37:34.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:34 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T11:37:34.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:34 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:37:35.735 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:35 smithi188 ceph-mon[36925]: osdmap e78: 11 total, 11 up, 11 in 2024-02-18T11:37:35.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:35 smithi012 ceph-mon[32460]: osdmap e78: 11 total, 11 up, 11 in 2024-02-18T11:37:35.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:35 smithi158 ceph-mon[36626]: osdmap e78: 11 total, 11 up, 11 in 2024-02-18T11:37:36.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:36 smithi188 ceph-mon[36925]: pgmap v240: 1 pgs: 1 remapped+peering; 577 KiB data, 556 MiB used, 983 GiB / 983 GiB avail 2024-02-18T11:37:36.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:36 smithi012 ceph-mon[32460]: pgmap v240: 1 pgs: 1 remapped+peering; 577 KiB data, 556 MiB used, 983 GiB / 983 GiB avail 2024-02-18T11:37:36.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:36 smithi158 ceph-mon[36626]: pgmap v240: 1 pgs: 1 remapped+peering; 577 KiB data, 556 MiB used, 983 GiB / 983 GiB avail 2024-02-18T11:37:37.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:37 smithi188 ceph-mon[36925]: from='client.? 172.21.15.188:0/1334058662' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "98aa3a0d-4b2f-46c8-9341-375d13794c3c"}]: dispatch 2024-02-18T11:37:37.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:37 smithi188 ceph-mon[36925]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "98aa3a0d-4b2f-46c8-9341-375d13794c3c"}]: dispatch 2024-02-18T11:37:37.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:37 smithi188 ceph-mon[36925]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "98aa3a0d-4b2f-46c8-9341-375d13794c3c"}]': finished 2024-02-18T11:37:37.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:37 smithi188 ceph-mon[36925]: osdmap e79: 12 total, 11 up, 12 in 2024-02-18T11:37:37.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:37 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-18T11:37:37.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:37 smithi188 ceph-mon[36925]: from='client.? 172.21.15.188:0/1504518097' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T11:37:37.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:37 smithi012 ceph-mon[32460]: from='client.? 172.21.15.188:0/1334058662' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "98aa3a0d-4b2f-46c8-9341-375d13794c3c"}]: dispatch 2024-02-18T11:37:37.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:37 smithi012 ceph-mon[32460]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "98aa3a0d-4b2f-46c8-9341-375d13794c3c"}]: dispatch 2024-02-18T11:37:37.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:37 smithi012 ceph-mon[32460]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "98aa3a0d-4b2f-46c8-9341-375d13794c3c"}]': finished 2024-02-18T11:37:37.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:37 smithi012 ceph-mon[32460]: osdmap e79: 12 total, 11 up, 12 in 2024-02-18T11:37:37.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:37 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-18T11:37:37.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:37 smithi012 ceph-mon[32460]: from='client.? 172.21.15.188:0/1504518097' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T11:37:37.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:37 smithi158 ceph-mon[36626]: from='client.? 172.21.15.188:0/1334058662' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "98aa3a0d-4b2f-46c8-9341-375d13794c3c"}]: dispatch 2024-02-18T11:37:37.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:37 smithi158 ceph-mon[36626]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "98aa3a0d-4b2f-46c8-9341-375d13794c3c"}]: dispatch 2024-02-18T11:37:37.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:37 smithi158 ceph-mon[36626]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "98aa3a0d-4b2f-46c8-9341-375d13794c3c"}]': finished 2024-02-18T11:37:37.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:37 smithi158 ceph-mon[36626]: osdmap e79: 12 total, 11 up, 12 in 2024-02-18T11:37:37.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:37 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-18T11:37:37.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:37 smithi158 ceph-mon[36626]: from='client.? 172.21.15.188:0/1504518097' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T11:37:38.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:38 smithi012 ceph-mon[32460]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 129 KiB/s, 0 objects/s recovering 2024-02-18T11:37:38.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:38 smithi158 ceph-mon[36626]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 129 KiB/s, 0 objects/s recovering 2024-02-18T11:37:39.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:38 smithi188 ceph-mon[36925]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 129 KiB/s, 0 objects/s recovering 2024-02-18T11:37:40.792 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:40 smithi188 ceph-mon[36925]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-18T11:37:40.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:40 smithi012 ceph-mon[32460]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-18T11:37:40.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:40 smithi158 ceph-mon[36626]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-18T11:37:42.886 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:42 smithi188 ceph-mon[36925]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-02-18T11:37:42.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:42 smithi012 ceph-mon[32460]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-02-18T11:37:42.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:42 smithi158 ceph-mon[36626]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-02-18T11:37:44.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:44 smithi012 ceph-mon[32460]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-02-18T11:37:44.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:44 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-02-18T11:37:44.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:44 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:37:44.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:44 smithi188 ceph-mon[36925]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-02-18T11:37:44.935 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:44 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-02-18T11:37:44.935 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:44 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:37:44.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:44 smithi158 ceph-mon[36626]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-02-18T11:37:44.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:44 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-02-18T11:37:44.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:44 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:37:45.899 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:45 smithi188 ceph-mon[36925]: Deploying daemon osd.11 on smithi188 2024-02-18T11:37:45.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:45 smithi012 ceph-mon[32460]: Deploying daemon osd.11 on smithi188 2024-02-18T11:37:45.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:45 smithi158 ceph-mon[36626]: Deploying daemon osd.11 on smithi188 2024-02-18T11:37:46.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:46 smithi188 ceph-mon[36925]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-18T11:37:46.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:46 smithi012 ceph-mon[32460]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-18T11:37:46.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:46 smithi158 ceph-mon[36626]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-18T11:37:48.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:48 smithi188 ceph-mon[36925]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-02-18T11:37:48.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:48 smithi012 ceph-mon[32460]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-02-18T11:37:48.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:48 smithi158 ceph-mon[36626]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-02-18T11:37:51.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:50 smithi012 ceph-mon[32460]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-02-18T11:37:51.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:50 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:51.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:50 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:51.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:50 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:37:51.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:50 smithi158 ceph-mon[36626]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-02-18T11:37:51.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:50 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:51.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:50 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:51.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:50 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:37:51.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:50 smithi188 ceph-mon[36925]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-02-18T11:37:51.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:50 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:51.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:50 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:51.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:50 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:37:53.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:52 smithi012 ceph-mon[32460]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-02-18T11:37:53.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:52 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:53.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:52 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:53.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:52 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:37:53.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:52 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:37:53.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:52 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:53.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:52 smithi158 ceph-mon[36626]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-02-18T11:37:53.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:52 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:53.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:52 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:53.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:52 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:37:53.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:52 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:37:53.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:52 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:53.327 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:52 smithi188 ceph-mon[36925]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-02-18T11:37:53.328 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:52 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:53.328 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:52 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:53.328 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:52 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:37:53.328 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:52 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:37:53.328 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:52 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:53.690 INFO:teuthology.orchestra.run.smithi188.stdout:Created osd(s) 11 on host 'smithi188' 2024-02-18T11:37:53.691 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:37:53.696+0000 7ff02effd700 1 -- 172.21.15.188:0/34156981 <== mgr.14152 v2:172.21.15.12:6800/1153736685 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7ff03802d8b0 con 0x7ff0240a27e0 2024-02-18T11:37:53.695 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:37:53.698+0000 7ff03fccc700 1 -- 172.21.15.188:0/34156981 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7ff0240a27e0 msgr2=0x7ff0240a4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:37:53.695 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:37:53.698+0000 7ff03fccc700 1 --2- 172.21.15.188:0/34156981 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7ff0240a27e0 0x7ff0240a4ca0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff03400b5c0 tx=0x7ff034005fb0 comp rx=0 tx=0).stop 2024-02-18T11:37:53.696 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:37:53.698+0000 7ff03fccc700 1 -- 172.21.15.188:0/34156981 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff038101de0 msgr2=0x7ff03819ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:37:53.696 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:37:53.698+0000 7ff03fccc700 1 --2- 172.21.15.188:0/34156981 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff038101de0 0x7ff03819ca60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff02800d950 tx=0x7ff02800dd10 comp rx=0 tx=0).stop 2024-02-18T11:37:53.696 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:37:53.698+0000 7ff03fccc700 1 -- 172.21.15.188:0/34156981 shutdown_connections 2024-02-18T11:37:53.696 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:37:53.698+0000 7ff03fccc700 1 --2- 172.21.15.188:0/34156981 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff038102780 0x7ff03819f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:37:53.696 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:37:53.698+0000 7ff03fccc700 1 --2- 172.21.15.188:0/34156981 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7ff0240a27e0 0x7ff0240a4ca0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:37:53.696 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:37:53.698+0000 7ff03fccc700 1 --2- 172.21.15.188:0/34156981 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff038100be0 0x7ff03806a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:37:53.696 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:37:53.698+0000 7ff03fccc700 1 --2- 172.21.15.188:0/34156981 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff038101de0 0x7ff03819ca60 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:37:53.697 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:37:53.698+0000 7ff03fccc700 1 -- 172.21.15.188:0/34156981 >> 172.21.15.188:0/34156981 conn(0x7ff0380fc180 msgr2=0x7ff038105200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:37:53.697 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:37:53.698+0000 7ff03fccc700 1 -- 172.21.15.188:0/34156981 shutdown_connections 2024-02-18T11:37:53.697 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:37:53.698+0000 7ff03fccc700 1 -- 172.21.15.188:0/34156981 wait complete. 2024-02-18T11:37:54.375 DEBUG:teuthology.orchestra.run.smithi188:osd.11> sudo journalctl -f -n 0 -u ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@osd.11.service 2024-02-18T11:37:54.377 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-02-18T11:37:54.378 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph osd stat -f json 2024-02-18T11:37:54.629 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T11:37:54.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:54 smithi012 ceph-mon[32460]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-02-18T11:37:54.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:54 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:54.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:54 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:54.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:54 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:37:54.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:54 smithi012 ceph-mon[32460]: from='osd.11 [v2:172.21.15.188:6824/1135773350,v1:172.21.15.188:6825/1135773350]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-02-18T11:37:54.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:54 smithi012 ceph-mon[32460]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-02-18T11:37:54.961 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:54 smithi188 ceph-mon[36925]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-02-18T11:37:54.961 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:54 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:54.961 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:54 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:54.961 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:54 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:37:54.961 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:54 smithi188 ceph-mon[36925]: from='osd.11 [v2:172.21.15.188:6824/1135773350,v1:172.21.15.188:6825/1135773350]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-02-18T11:37:54.961 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:54 smithi188 ceph-mon[36925]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-02-18T11:37:54.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:54 smithi158 ceph-mon[36626]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-02-18T11:37:54.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:54 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:54.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:54 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:54.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:54 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:37:54.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:54 smithi158 ceph-mon[36626]: from='osd.11 [v2:172.21.15.188:6824/1135773350,v1:172.21.15.188:6825/1135773350]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-02-18T11:37:54.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:54 smithi158 ceph-mon[36626]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-02-18T11:37:55.850 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:55 smithi012 ceph-mon[32460]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-02-18T11:37:55.850 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:55 smithi012 ceph-mon[32460]: from='osd.11 [v2:172.21.15.188:6824/1135773350,v1:172.21.15.188:6825/1135773350]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-02-18T11:37:55.850 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:55 smithi012 ceph-mon[32460]: osdmap e80: 12 total, 11 up, 12 in 2024-02-18T11:37:55.850 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:55 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-18T11:37:55.850 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:55 smithi012 ceph-mon[32460]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-02-18T11:37:55.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:55 smithi158 ceph-mon[36626]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-02-18T11:37:55.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:55 smithi158 ceph-mon[36626]: from='osd.11 [v2:172.21.15.188:6824/1135773350,v1:172.21.15.188:6825/1135773350]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-02-18T11:37:55.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:55 smithi158 ceph-mon[36626]: osdmap e80: 12 total, 11 up, 12 in 2024-02-18T11:37:55.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:55 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-18T11:37:55.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:55 smithi158 ceph-mon[36626]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-02-18T11:37:56.026 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:55 smithi188 ceph-mon[36925]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-02-18T11:37:56.026 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:55 smithi188 ceph-mon[36925]: from='osd.11 [v2:172.21.15.188:6824/1135773350,v1:172.21.15.188:6825/1135773350]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-02-18T11:37:56.026 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:55 smithi188 ceph-mon[36925]: osdmap e80: 12 total, 11 up, 12 in 2024-02-18T11:37:56.026 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:55 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-18T11:37:56.026 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:55 smithi188 ceph-mon[36925]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-02-18T11:37:56.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:56.072+0000 7f4b1e25a700 1 -- 172.21.15.12:0/426142274 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b18101de0 msgr2=0x7f4b18102240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:37:56.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:56.072+0000 7f4b1e25a700 1 --2- 172.21.15.12:0/426142274 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b18101de0 0x7f4b18102240 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c0099c0 tx=0x7f4b0c009cd0 comp rx=0 tx=0).stop 2024-02-18T11:37:56.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:56.072+0000 7f4b1e25a700 1 -- 172.21.15.12:0/426142274 shutdown_connections 2024-02-18T11:37:56.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:56.072+0000 7f4b1e25a700 1 --2- 172.21.15.12:0/426142274 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b18102780 0x7f4b18109030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:37:56.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:56.072+0000 7f4b1e25a700 1 --2- 172.21.15.12:0/426142274 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4b18100be0 0x7f4b18101000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:37:56.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:56.072+0000 7f4b1e25a700 1 --2- 172.21.15.12:0/426142274 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b18101de0 0x7f4b18102240 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:37:56.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:56.072+0000 7f4b1e25a700 1 -- 172.21.15.12:0/426142274 >> 172.21.15.12:0/426142274 conn(0x7f4b180fc180 msgr2=0x7f4b180fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:37:56.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:56.072+0000 7f4b1e25a700 1 -- 172.21.15.12:0/426142274 shutdown_connections 2024-02-18T11:37:56.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:56.072+0000 7f4b1e25a700 1 -- 172.21.15.12:0/426142274 wait complete. 2024-02-18T11:37:56.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:56.073+0000 7f4b1e25a700 1 Processor -- start 2024-02-18T11:37:56.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:56.073+0000 7f4b1e25a700 1 -- start start 2024-02-18T11:37:56.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:56.073+0000 7f4b1e25a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b18100be0 0x7f4b18196c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:37:56.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:56.073+0000 7f4b1e25a700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4b18101de0 0x7f4b18197190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:37:56.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:56.073+0000 7f4b1e25a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b18102780 0x7f4b1819e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:37:56.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:56.073+0000 7f4b1e25a700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b1819e7a0 con 0x7f4b18100be0 2024-02-18T11:37:56.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:56.073+0000 7f4b1e25a700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b1819e8e0 con 0x7f4b18101de0 2024-02-18T11:37:56.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:56.073+0000 7f4b1e25a700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b1819eaa0 con 0x7f4b18102780 2024-02-18T11:37:56.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:56.073+0000 7f4b177fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b18100be0 0x7f4b18196c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:37:56.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:56.073+0000 7f4b177fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b18100be0 0x7f4b18196c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41208/0 (socket says 172.21.15.12:41208) 2024-02-18T11:37:56.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:56.073+0000 7f4b177fe700 1 -- 172.21.15.12:0/1483140069 learned_addr learned my addr 172.21.15.12:0/1483140069 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:37:56.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:56.073+0000 7f4b17fff700 1 --2- 172.21.15.12:0/1483140069 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b18102780 0x7f4b1819e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:37:56.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:56.073+0000 7f4b16ffd700 1 --2- 172.21.15.12:0/1483140069 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4b18101de0 0x7f4b18197190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:37:56.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:56.073+0000 7f4b177fe700 1 -- 172.21.15.12:0/1483140069 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b18102780 msgr2=0x7f4b1819e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:37:56.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:56.073+0000 7f4b177fe700 1 --2- 172.21.15.12:0/1483140069 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b18102780 0x7f4b1819e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:37:56.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:56.073+0000 7f4b177fe700 1 -- 172.21.15.12:0/1483140069 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4b18101de0 msgr2=0x7f4b18197190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:37:56.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:56.073+0000 7f4b177fe700 1 --2- 172.21.15.12:0/1483140069 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4b18101de0 0x7f4b18197190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:37:56.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:56.073+0000 7f4b177fe700 1 -- 172.21.15.12:0/1483140069 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b0c0095f0 con 0x7f4b18100be0 2024-02-18T11:37:56.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:56.074+0000 7f4b177fe700 1 --2- 172.21.15.12:0/1483140069 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b18100be0 0x7f4b18196c50 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4b0800b500 tx=0x7f4b0800b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:37:56.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:56.074+0000 7f4b14ff9700 1 -- 172.21.15.12:0/1483140069 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b08011830 con 0x7f4b18100be0 2024-02-18T11:37:56.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:56.074+0000 7f4b14ff9700 1 -- 172.21.15.12:0/1483140069 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4b08011e70 con 0x7f4b18100be0 2024-02-18T11:37:56.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:56.074+0000 7f4b1e25a700 1 -- 172.21.15.12:0/1483140069 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b1819ed80 con 0x7f4b18100be0 2024-02-18T11:37:56.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:56.074+0000 7f4b14ff9700 1 -- 172.21.15.12:0/1483140069 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b0800f2e0 con 0x7f4b18100be0 2024-02-18T11:37:56.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:56.074+0000 7f4b1e25a700 1 -- 172.21.15.12:0/1483140069 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b1819f1c0 con 0x7f4b18100be0 2024-02-18T11:37:56.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:56.075+0000 7f4b1e25a700 1 -- 172.21.15.12:0/1483140069 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b18068970 con 0x7f4b18100be0 2024-02-18T11:37:56.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:56.076+0000 7f4b14ff9700 1 -- 172.21.15.12:0/1483140069 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) v1 ==== 137245+0+0 (secure 0 0 0) 0x7f4b080105f0 con 0x7f4b18100be0 2024-02-18T11:37:56.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:56.076+0000 7f4b14ff9700 1 --2- 172.21.15.12:0/1483140069 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f4b000a27e0 0x7f4b000a4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:37:56.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:56.077+0000 7f4b16ffd700 1 --2- 172.21.15.12:0/1483140069 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f4b000a27e0 0x7f4b000a4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:37:56.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:56.077+0000 7f4b14ff9700 1 -- 172.21.15.12:0/1483140069 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6847+0+0 (secure 0 0 0) 0x7f4b080cd9c0 con 0x7f4b18100be0 2024-02-18T11:37:56.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:56.077+0000 7f4b16ffd700 1 --2- 172.21.15.12:0/1483140069 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f4b000a27e0 0x7f4b000a4ca0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c0038c0 tx=0x7f4b0c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:37:56.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:56.081+0000 7f4b14ff9700 1 -- 172.21.15.12:0/1483140069 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4b08096d10 con 0x7f4b18100be0 2024-02-18T11:37:56.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:56.283+0000 7f4b1e25a700 1 -- 172.21.15.12:0/1483140069 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f4b1810c660 con 0x7f4b18100be0 2024-02-18T11:37:56.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:56.283+0000 7f4b14ff9700 1 -- 172.21.15.12:0/1483140069 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) v1 ==== 74+0+133 (secure 0 0 0) 0x7f4b0809a3d0 con 0x7f4b18100be0 2024-02-18T11:37:56.278 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:37:56.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:56.285+0000 7f4b1e25a700 1 -- 172.21.15.12:0/1483140069 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f4b000a27e0 msgr2=0x7f4b000a4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:37:56.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:56.285+0000 7f4b1e25a700 1 --2- 172.21.15.12:0/1483140069 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f4b000a27e0 0x7f4b000a4ca0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c0038c0 tx=0x7f4b0c005fb0 comp rx=0 tx=0).stop 2024-02-18T11:37:56.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:56.285+0000 7f4b1e25a700 1 -- 172.21.15.12:0/1483140069 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b18100be0 msgr2=0x7f4b18196c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:37:56.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:56.285+0000 7f4b1e25a700 1 --2- 172.21.15.12:0/1483140069 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b18100be0 0x7f4b18196c50 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4b0800b500 tx=0x7f4b0800b810 comp rx=0 tx=0).stop 2024-02-18T11:37:56.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:56.285+0000 7f4b1e25a700 1 -- 172.21.15.12:0/1483140069 shutdown_connections 2024-02-18T11:37:56.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:56.285+0000 7f4b1e25a700 1 --2- 172.21.15.12:0/1483140069 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b18102780 0x7f4b1819e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:37:56.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:56.285+0000 7f4b1e25a700 1 --2- 172.21.15.12:0/1483140069 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f4b000a27e0 0x7f4b000a4ca0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:37:56.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:56.285+0000 7f4b1e25a700 1 --2- 172.21.15.12:0/1483140069 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b18100be0 0x7f4b18196c50 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:37:56.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:56.285+0000 7f4b1e25a700 1 --2- 172.21.15.12:0/1483140069 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4b18101de0 0x7f4b18197190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:37:56.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:56.285+0000 7f4b1e25a700 1 -- 172.21.15.12:0/1483140069 >> 172.21.15.12:0/1483140069 conn(0x7f4b180fc180 msgr2=0x7f4b18105180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:37:56.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:56.285+0000 7f4b1e25a700 1 -- 172.21.15.12:0/1483140069 shutdown_connections 2024-02-18T11:37:56.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:56.285+0000 7f4b1e25a700 1 -- 172.21.15.12:0/1483140069 wait complete. 2024-02-18T11:37:56.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:56 smithi188 ceph-mon[36925]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-02-18T11:37:56.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:56 smithi188 ceph-mon[36925]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-02-18T11:37:56.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:56 smithi188 ceph-mon[36925]: osdmap e81: 12 total, 11 up, 12 in 2024-02-18T11:37:56.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:56 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-18T11:37:56.856 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:56 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-18T11:37:56.856 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:56 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/1483140069' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-18T11:37:56.856 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:56 smithi188 ceph-mon[36925]: from='osd.11 ' entity='osd.11' 2024-02-18T11:37:56.856 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:37:56 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:37:56.509+0000 7f1f4b8de700 -1 osd.11 0 waiting for initial osdmap 2024-02-18T11:37:56.856 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:37:56 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:37:56.534+0000 7f1f43eca700 -1 osd.11 81 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-18T11:37:56.870 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":81,"num_osds":12,"num_up_osds":11,"osd_up_since":1708256250,"num_in_osds":12,"osd_in_since":1708256256,"num_remapped_pgs":0} 2024-02-18T11:37:56.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:56 smithi158 ceph-mon[36626]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-02-18T11:37:56.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:56 smithi158 ceph-mon[36626]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-02-18T11:37:56.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:56 smithi158 ceph-mon[36626]: osdmap e81: 12 total, 11 up, 12 in 2024-02-18T11:37:56.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:56 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-18T11:37:56.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:56 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-18T11:37:56.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:56 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/1483140069' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-18T11:37:56.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:56 smithi158 ceph-mon[36626]: from='osd.11 ' entity='osd.11' 2024-02-18T11:37:57.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:56 smithi012 ceph-mon[32460]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-02-18T11:37:57.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:56 smithi012 ceph-mon[32460]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-02-18T11:37:57.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:56 smithi012 ceph-mon[32460]: osdmap e81: 12 total, 11 up, 12 in 2024-02-18T11:37:57.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:56 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-18T11:37:57.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:56 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-18T11:37:57.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:56 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/1483140069' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-18T11:37:57.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:56 smithi012 ceph-mon[32460]: from='osd.11 ' entity='osd.11' 2024-02-18T11:37:57.871 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph osd stat -f json 2024-02-18T11:37:57.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:57 smithi158 ceph-mon[36626]: purged_snaps scrub starts 2024-02-18T11:37:57.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:57 smithi158 ceph-mon[36626]: purged_snaps scrub ok 2024-02-18T11:37:57.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:57 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-18T11:37:57.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:57 smithi158 ceph-mon[36626]: osd.11 [v2:172.21.15.188:6824/1135773350,v1:172.21.15.188:6825/1135773350] boot 2024-02-18T11:37:57.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:57 smithi158 ceph-mon[36626]: osdmap e82: 12 total, 12 up, 12 in 2024-02-18T11:37:57.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:57 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-18T11:37:58.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:57 smithi188 ceph-mon[36925]: purged_snaps scrub starts 2024-02-18T11:37:58.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:57 smithi188 ceph-mon[36925]: purged_snaps scrub ok 2024-02-18T11:37:58.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:57 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-18T11:37:58.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:57 smithi188 ceph-mon[36925]: osd.11 [v2:172.21.15.188:6824/1135773350,v1:172.21.15.188:6825/1135773350] boot 2024-02-18T11:37:58.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:57 smithi188 ceph-mon[36925]: osdmap e82: 12 total, 12 up, 12 in 2024-02-18T11:37:58.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:57 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-18T11:37:58.116 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T11:37:58.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:57 smithi012 ceph-mon[32460]: purged_snaps scrub starts 2024-02-18T11:37:58.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:57 smithi012 ceph-mon[32460]: purged_snaps scrub ok 2024-02-18T11:37:58.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:57 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-18T11:37:58.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:57 smithi012 ceph-mon[32460]: osd.11 [v2:172.21.15.188:6824/1135773350,v1:172.21.15.188:6825/1135773350] boot 2024-02-18T11:37:58.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:57 smithi012 ceph-mon[32460]: osdmap e82: 12 total, 12 up, 12 in 2024-02-18T11:37:58.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:57 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-18T11:37:58.890 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:58 smithi012 ceph-mon[32460]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-02-18T11:37:58.891 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:58 smithi012 ceph-mon[32460]: Detected new or changed devices on smithi188 2024-02-18T11:37:58.891 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:58 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:58.891 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:58 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:58.891 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:58 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:37:58.891 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:58 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:37:58.891 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:58 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:37:58.891 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:58 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:37:58.891 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:58 smithi012 ceph-mon[32460]: Adjusting osd_memory_target on smithi188 to 4270M 2024-02-18T11:37:58.891 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:58 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:58.891 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:58 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:37:58.891 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:58 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:37:58.891 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:58 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:58.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:58 smithi158 ceph-mon[36626]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-02-18T11:37:58.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:58 smithi158 ceph-mon[36626]: Detected new or changed devices on smithi188 2024-02-18T11:37:58.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:58 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:58.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:58 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:58.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:58 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:37:58.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:58 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:37:58.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:58 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:37:58.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:58 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:37:58.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:58 smithi158 ceph-mon[36626]: Adjusting osd_memory_target on smithi188 to 4270M 2024-02-18T11:37:58.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:58 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:58.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:58 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:37:58.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:58 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:37:58.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:58 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:59.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:58 smithi188 ceph-mon[36925]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-02-18T11:37:59.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:58 smithi188 ceph-mon[36925]: Detected new or changed devices on smithi188 2024-02-18T11:37:59.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:58 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:59.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:58 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:59.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:58 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:37:59.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:58 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:37:59.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:58 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:37:59.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:58 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:37:59.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:58 smithi188 ceph-mon[36925]: Adjusting osd_memory_target on smithi188 to 4270M 2024-02-18T11:37:59.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:58 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:59.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:58 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:37:59.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:58 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:37:59.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:58 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:37:59.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:59.671+0000 7fefaa1a4700 1 -- 172.21.15.12:0/1359213510 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fefa4100380 msgr2=0x7fefa41007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:37:59.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:59.671+0000 7fefaa1a4700 1 --2- 172.21.15.12:0/1359213510 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fefa4100380 0x7fefa41007a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fef940099c0 tx=0x7fef94009cd0 comp rx=0 tx=0).stop 2024-02-18T11:37:59.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:59.671+0000 7fefaa1a4700 1 -- 172.21.15.12:0/1359213510 shutdown_connections 2024-02-18T11:37:59.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:59.671+0000 7fefaa1a4700 1 --2- 172.21.15.12:0/1359213510 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fefa4101f20 0x7fefa41087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:37:59.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:59.671+0000 7fefaa1a4700 1 --2- 172.21.15.12:0/1359213510 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fefa4100380 0x7fefa41007a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:37:59.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:59.671+0000 7fefaa1a4700 1 --2- 172.21.15.12:0/1359213510 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fefa4101580 0x7fefa41019e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:37:59.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:59.671+0000 7fefaa1a4700 1 -- 172.21.15.12:0/1359213510 >> 172.21.15.12:0/1359213510 conn(0x7fefa40fb900 msgr2=0x7fefa40fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:37:59.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:59.671+0000 7fefaa1a4700 1 -- 172.21.15.12:0/1359213510 shutdown_connections 2024-02-18T11:37:59.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:59.671+0000 7fefaa1a4700 1 -- 172.21.15.12:0/1359213510 wait complete. 2024-02-18T11:37:59.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:59.671+0000 7fefaa1a4700 1 Processor -- start 2024-02-18T11:37:59.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:59.672+0000 7fefaa1a4700 1 -- start start 2024-02-18T11:37:59.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:59.672+0000 7fefaa1a4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fefa4100380 0x7fefa4197cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:37:59.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:59.672+0000 7fefaa1a4700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fefa4101580 0x7fefa4198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:37:59.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:59.672+0000 7fefaa1a4700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fefa4101f20 0x7fefa419f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:37:59.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:59.672+0000 7fefaa1a4700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa419f840 con 0x7fefa4100380 2024-02-18T11:37:59.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:59.672+0000 7fefaa1a4700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa419f980 con 0x7fefa4101f20 2024-02-18T11:37:59.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:59.672+0000 7fefaa1a4700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa419fb40 con 0x7fefa4101580 2024-02-18T11:37:59.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:59.672+0000 7fefa37fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fefa4100380 0x7fefa4197cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:37:59.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:59.672+0000 7fefa37fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fefa4100380 0x7fefa4197cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41232/0 (socket says 172.21.15.12:41232) 2024-02-18T11:37:59.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:59.672+0000 7fefa37fe700 1 -- 172.21.15.12:0/3482703275 learned_addr learned my addr 172.21.15.12:0/3482703275 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:37:59.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:59.672+0000 7fefa2ffd700 1 --2- 172.21.15.12:0/3482703275 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fefa4101580 0x7fefa4198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:37:59.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:59.672+0000 7fefa3fff700 1 --2- 172.21.15.12:0/3482703275 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fefa4101f20 0x7fefa419f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:37:59.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:59.672+0000 7fefa37fe700 1 -- 172.21.15.12:0/3482703275 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fefa4101580 msgr2=0x7fefa4198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:37:59.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:59.672+0000 7fefa37fe700 1 --2- 172.21.15.12:0/3482703275 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fefa4101580 0x7fefa4198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:37:59.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:59.672+0000 7fefa37fe700 1 -- 172.21.15.12:0/3482703275 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fefa4101f20 msgr2=0x7fefa419f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:37:59.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:59.672+0000 7fefa37fe700 1 --2- 172.21.15.12:0/3482703275 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fefa4101f20 0x7fefa419f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:37:59.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:59.672+0000 7fefa37fe700 1 -- 172.21.15.12:0/3482703275 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef940095f0 con 0x7fefa4100380 2024-02-18T11:37:59.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:59.673+0000 7fefa37fe700 1 --2- 172.21.15.12:0/3482703275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fefa4100380 0x7fefa4197cf0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fef94006010 tx=0x7fef94004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:37:59.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:59.673+0000 7fefa0ff9700 1 -- 172.21.15.12:0/3482703275 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef9401d070 con 0x7fefa4100380 2024-02-18T11:37:59.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:59.673+0000 7fefa0ff9700 1 -- 172.21.15.12:0/3482703275 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fef94004050 con 0x7fefa4100380 2024-02-18T11:37:59.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:59.673+0000 7fefaa1a4700 1 -- 172.21.15.12:0/3482703275 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefa419fdc0 con 0x7fefa4100380 2024-02-18T11:37:59.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:59.673+0000 7fefa0ff9700 1 -- 172.21.15.12:0/3482703275 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef9400f460 con 0x7fefa4100380 2024-02-18T11:37:59.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:59.673+0000 7fefaa1a4700 1 -- 172.21.15.12:0/3482703275 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefa41a0180 con 0x7fefa4100380 2024-02-18T11:37:59.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:59.674+0000 7fefaa1a4700 1 -- 172.21.15.12:0/3482703275 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefa410b4e0 con 0x7fefa4100380 2024-02-18T11:37:59.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:59.675+0000 7fefa0ff9700 1 -- 172.21.15.12:0/3482703275 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) v1 ==== 137245+0+0 (secure 0 0 0) 0x7fef94022600 con 0x7fefa4100380 2024-02-18T11:37:59.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:59.675+0000 7fefa0ff9700 1 --2- 172.21.15.12:0/3482703275 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fef900a27e0 0x7fef900a4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:37:59.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:59.675+0000 7fefa2ffd700 1 --2- 172.21.15.12:0/3482703275 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fef900a27e0 0x7fef900a4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:37:59.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:59.675+0000 7fefa0ff9700 1 -- 172.21.15.12:0/3482703275 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fef940cf350 con 0x7fefa4100380 2024-02-18T11:37:59.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:59.676+0000 7fefa2ffd700 1 --2- 172.21.15.12:0/3482703275 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fef900a27e0 0x7fef900a4ca0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fef8c005950 tx=0x7fef8c00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:37:59.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:59.680+0000 7fefa0ff9700 1 -- 172.21.15.12:0/3482703275 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fef94098610 con 0x7fefa4100380 2024-02-18T11:37:59.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:59.885+0000 7fefaa1a4700 1 -- 172.21.15.12:0/3482703275 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fefa410b6a0 con 0x7fefa4100380 2024-02-18T11:37:59.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:59.886+0000 7fefa0ff9700 1 -- 172.21.15.12:0/3482703275 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v83) v1 ==== 74+0+133 (secure 0 0 0) 0x7fef9409bcd0 con 0x7fefa4100380 2024-02-18T11:37:59.880 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:37:59.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:59.888+0000 7fefaa1a4700 1 -- 172.21.15.12:0/3482703275 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fef900a27e0 msgr2=0x7fef900a4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:37:59.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:59.888+0000 7fefaa1a4700 1 --2- 172.21.15.12:0/3482703275 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fef900a27e0 0x7fef900a4ca0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fef8c005950 tx=0x7fef8c00a300 comp rx=0 tx=0).stop 2024-02-18T11:37:59.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:59.888+0000 7fefaa1a4700 1 -- 172.21.15.12:0/3482703275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fefa4100380 msgr2=0x7fefa4197cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:37:59.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:59.888+0000 7fefaa1a4700 1 --2- 172.21.15.12:0/3482703275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fefa4100380 0x7fefa4197cf0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fef94006010 tx=0x7fef94004730 comp rx=0 tx=0).stop 2024-02-18T11:37:59.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:59.888+0000 7fefaa1a4700 1 -- 172.21.15.12:0/3482703275 shutdown_connections 2024-02-18T11:37:59.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:59.888+0000 7fefaa1a4700 1 --2- 172.21.15.12:0/3482703275 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fefa4101f20 0x7fefa419f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:37:59.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:59.888+0000 7fefaa1a4700 1 --2- 172.21.15.12:0/3482703275 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fef900a27e0 0x7fef900a4ca0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:37:59.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:59.888+0000 7fefaa1a4700 1 --2- 172.21.15.12:0/3482703275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fefa4100380 0x7fefa4197cf0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:37:59.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:59.888+0000 7fefaa1a4700 1 --2- 172.21.15.12:0/3482703275 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fefa4101580 0x7fefa4198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:37:59.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:59.888+0000 7fefaa1a4700 1 -- 172.21.15.12:0/3482703275 >> 172.21.15.12:0/3482703275 conn(0x7fefa40fb900 msgr2=0x7fefa4106a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:37:59.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:59.888+0000 7fefaa1a4700 1 -- 172.21.15.12:0/3482703275 shutdown_connections 2024-02-18T11:37:59.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:37:59.888+0000 7fefaa1a4700 1 -- 172.21.15.12:0/3482703275 wait complete. 2024-02-18T11:37:59.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:37:59 smithi158 ceph-mon[36626]: osdmap e83: 12 total, 12 up, 12 in 2024-02-18T11:38:00.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:37:59 smithi188 ceph-mon[36925]: osdmap e83: 12 total, 12 up, 12 in 2024-02-18T11:38:00.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:37:59 smithi012 ceph-mon[32460]: osdmap e83: 12 total, 12 up, 12 in 2024-02-18T11:38:00.542 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":83,"num_osds":12,"num_up_osds":12,"osd_up_since":1708256277,"num_in_osds":12,"osd_in_since":1708256256,"num_remapped_pgs":0} 2024-02-18T11:38:00.543 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph osd dump --format=json 2024-02-18T11:38:00.781 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T11:38:01.029 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:00 smithi012 ceph-mon[32460]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-18T11:38:01.029 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:00 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/3482703275' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-18T11:38:01.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:00 smithi188 ceph-mon[36925]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-18T11:38:01.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:00 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/3482703275' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-18T11:38:01.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:00 smithi158 ceph-mon[36626]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-18T11:38:01.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:00 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/3482703275' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-18T11:38:02.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:02.428+0000 7fdeb5603700 1 -- 172.21.15.12:0/728267567 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdeb01026c0 msgr2=0x7fdeb010eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:02.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:02.428+0000 7fdeb5603700 1 --2- 172.21.15.12:0/728267567 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdeb01026c0 0x7fdeb010eb00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fdea80099c0 tx=0x7fdea8009cd0 comp rx=0 tx=0).stop 2024-02-18T11:38:02.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:02.428+0000 7fdeb5603700 1 -- 172.21.15.12:0/728267567 shutdown_connections 2024-02-18T11:38:02.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:02.428+0000 7fdeb5603700 1 --2- 172.21.15.12:0/728267567 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdeb01026c0 0x7fdeb010eb00 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:02.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:02.428+0000 7fdeb5603700 1 --2- 172.21.15.12:0/728267567 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdeb0107ce0 0x7fdeb01080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:02.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:02.428+0000 7fdeb5603700 1 --2- 172.21.15.12:0/728267567 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdeb0101d00 0x7fdeb0102180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:02.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:02.428+0000 7fdeb5603700 1 -- 172.21.15.12:0/728267567 >> 172.21.15.12:0/728267567 conn(0x7fdeb0078de0 msgr2=0x7fdeb00791f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:02.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:02.428+0000 7fdeb5603700 1 -- 172.21.15.12:0/728267567 shutdown_connections 2024-02-18T11:38:02.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:02.428+0000 7fdeb5603700 1 -- 172.21.15.12:0/728267567 wait complete. 2024-02-18T11:38:02.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:02.428+0000 7fdeb5603700 1 Processor -- start 2024-02-18T11:38:02.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:02.428+0000 7fdeb5603700 1 -- start start 2024-02-18T11:38:02.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:02.429+0000 7fdeb5603700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdeb0101d00 0x7fdeb0073fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:02.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:02.429+0000 7fdeb5603700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdeb01026c0 0x7fdeb00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:02.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:02.429+0000 7fdeb5603700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdeb0107ce0 0x7fdeb0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:02.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:02.429+0000 7fdeb5603700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb01a6430 con 0x7fdeb01026c0 2024-02-18T11:38:02.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:02.429+0000 7fdeb5603700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb01a65a0 con 0x7fdeb0101d00 2024-02-18T11:38:02.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:02.429+0000 7fdeb5603700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb01a6760 con 0x7fdeb0107ce0 2024-02-18T11:38:02.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:02.429+0000 7fdeae7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdeb01026c0 0x7fdeb00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:02.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:02.429+0000 7fdeae7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdeb01026c0 0x7fdeb00744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51522/0 (socket says 172.21.15.12:51522) 2024-02-18T11:38:02.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:02.429+0000 7fdeae7fc700 1 -- 172.21.15.12:0/61747542 learned_addr learned my addr 172.21.15.12:0/61747542 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:38:02.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:02.429+0000 7fdeaeffd700 1 --2- 172.21.15.12:0/61747542 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdeb0101d00 0x7fdeb0073fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:02.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:02.429+0000 7fdeaf7fe700 1 --2- 172.21.15.12:0/61747542 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdeb0107ce0 0x7fdeb0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:02.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:02.429+0000 7fdeae7fc700 1 -- 172.21.15.12:0/61747542 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdeb0107ce0 msgr2=0x7fdeb0074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:02.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:02.429+0000 7fdeae7fc700 1 --2- 172.21.15.12:0/61747542 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdeb0107ce0 0x7fdeb0074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:02.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:02.429+0000 7fdeae7fc700 1 -- 172.21.15.12:0/61747542 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdeb0101d00 msgr2=0x7fdeb0073fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:02.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:02.429+0000 7fdeae7fc700 1 --2- 172.21.15.12:0/61747542 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdeb0101d00 0x7fdeb0073fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:02.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:02.429+0000 7fdeae7fc700 1 -- 172.21.15.12:0/61747542 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdea80095f0 con 0x7fdeb01026c0 2024-02-18T11:38:02.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:02.429+0000 7fdeaf7fe700 1 --2- 172.21.15.12:0/61747542 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdeb0107ce0 0x7fdeb0074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:38:02.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:02.429+0000 7fdeae7fc700 1 --2- 172.21.15.12:0/61747542 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdeb01026c0 0x7fdeb00744f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fde9800c9b0 tx=0x7fde9800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:02.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:02.429+0000 7fdea7fff700 1 -- 172.21.15.12:0/61747542 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde98007b00 con 0x7fdeb01026c0 2024-02-18T11:38:02.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:02.430+0000 7fdeb5603700 1 -- 172.21.15.12:0/61747542 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdeb01a6a40 con 0x7fdeb01026c0 2024-02-18T11:38:02.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:02.430+0000 7fdea7fff700 1 -- 172.21.15.12:0/61747542 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fde98007c60 con 0x7fdeb01026c0 2024-02-18T11:38:02.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:02.430+0000 7fdea7fff700 1 -- 172.21.15.12:0/61747542 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde98018660 con 0x7fdeb01026c0 2024-02-18T11:38:02.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:02.430+0000 7fdeb5603700 1 -- 172.21.15.12:0/61747542 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdeb01a6f40 con 0x7fdeb01026c0 2024-02-18T11:38:02.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:02.431+0000 7fdeb5603700 1 -- 172.21.15.12:0/61747542 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdeb0068970 con 0x7fdeb01026c0 2024-02-18T11:38:02.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:02.432+0000 7fdea7fff700 1 -- 172.21.15.12:0/61747542 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) v1 ==== 137245+0+0 (secure 0 0 0) 0x7fde9800f450 con 0x7fdeb01026c0 2024-02-18T11:38:02.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:02.433+0000 7fdea7fff700 1 --2- 172.21.15.12:0/61747542 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fde9c0a27e0 0x7fde9c0a4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:02.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:02.433+0000 7fdeaeffd700 1 --2- 172.21.15.12:0/61747542 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fde9c0a27e0 0x7fde9c0a4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:02.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:02.433+0000 7fdea7fff700 1 -- 172.21.15.12:0/61747542 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fde980ce360 con 0x7fdeb01026c0 2024-02-18T11:38:02.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:02.433+0000 7fdeaeffd700 1 --2- 172.21.15.12:0/61747542 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fde9c0a27e0 0x7fde9c0a4ca0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fdea0009550 tx=0x7fdea00091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:02.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:02.437+0000 7fdea7fff700 1 -- 172.21.15.12:0/61747542 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fde98097580 con 0x7fdeb01026c0 2024-02-18T11:38:02.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:02.638+0000 7fdeb5603700 1 -- 172.21.15.12:0/61747542 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdeb00754e0 con 0x7fdeb01026c0 2024-02-18T11:38:02.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:02.639+0000 7fdea7fff700 1 -- 172.21.15.12:0/61747542 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+18466 (secure 0 0 0) 0x7fde9809ac40 con 0x7fdeb01026c0 2024-02-18T11:38:02.634 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:38:02.634 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":83,"fsid":"0c8bb0de-ce51-11ee-95ba-87774f69a715","created":"2024-02-18T11:30:49.110059+0000","modified":"2024-02-18T11:37:58.739798+0000","last_up_change":"2024-02-18T11:37:57.512337+0000","last_in_change":"2024-02-18T11:37:36.567969+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-18T11:34:15.281138+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"35f15ea6-deb8-48a1-a7dd-ab649ec13533","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":536522426},{"type":"v1","addr":"172.21.15.12:6803","nonce":536522426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":536522426},{"type":"v1","addr":"172.21.15.12:6805","nonce":536522426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":536522426},{"type":"v1","addr":"172.21.15.12:6809","nonce":536522426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":536522426},{"type":"v1","addr":"172.21.15.12:6807","nonce":536522426}]},"public_addr":"172.21.15.12:6803/536522426","cluster_addr":"172.21.15.12:6805/536522426","heartbeat_back_addr":"172.21.15.12:6809/536522426","heartbeat_front_addr":"172.21.15.12:6807/536522426","state":["exists","up"]},{"osd":1,"uuid":"2af937ef-d827-44e6-b805-70cffbfe1126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":4249111488},{"type":"v1","addr":"172.21.15.158:6801","nonce":4249111488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":4249111488},{"type":"v1","addr":"172.21.15.158:6803","nonce":4249111488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":4249111488},{"type":"v1","addr":"172.21.15.158:6807","nonce":4249111488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":4249111488},{"type":"v1","addr":"172.21.15.158:6805","nonce":4249111488}]},"public_addr":"172.21.15.158:6801/4249111488","cluster_addr":"172.21.15.158:6803/4249111488","heartbeat_back_addr":"172.21.15.158:6807/4249111488","heartbeat_front_addr":"172.21.15.158:6805/4249111488","state":["exists","up"]},{"osd":2,"uuid":"17b804dd-31df-45ae-a637-4a5f65bbaf26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":627976150},{"type":"v1","addr":"172.21.15.188:6801","nonce":627976150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":627976150},{"type":"v1","addr":"172.21.15.188:6803","nonce":627976150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":627976150},{"type":"v1","addr":"172.21.15.188:6807","nonce":627976150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":627976150},{"type":"v1","addr":"172.21.15.188:6805","nonce":627976150}]},"public_addr":"172.21.15.188:6801/627976150","cluster_addr":"172.21.15.188:6803/627976150","heartbeat_back_addr":"172.21.15.188:6807/627976150","heartbeat_front_addr":"172.21.15.188:6805/627976150","state":["exists","up"]},{"osd":3,"uuid":"3f9cc423-cdfa-4acf-aaa2-b58b32ccb9cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":3758512503},{"type":"v1","addr":"172.21.15.12:6811","nonce":3758512503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":3758512503},{"type":"v1","addr":"172.21.15.12:6813","nonce":3758512503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3758512503},{"type":"v1","addr":"172.21.15.12:6817","nonce":3758512503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":3758512503},{"type":"v1","addr":"172.21.15.12:6815","nonce":3758512503}]},"public_addr":"172.21.15.12:6811/3758512503","cluster_addr":"172.21.15.12:6813/3758512503","heartbeat_back_addr":"172.21.15.12:6817/3758512503","heartbeat_front_addr":"172.21.15.12:6815/3758512503","state":["exists","up"]},{"osd":4,"uuid":"a92e20ec-af1c-4ef4-a0cb-caba5fcc71aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":3892820997},{"type":"v1","addr":"172.21.15.158:6809","nonce":3892820997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":3892820997},{"type":"v1","addr":"172.21.15.158:6811","nonce":3892820997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":3892820997},{"type":"v1","addr":"172.21.15.158:6815","nonce":3892820997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":3892820997},{"type":"v1","addr":"172.21.15.158:6813","nonce":3892820997}]},"public_addr":"172.21.15.158:6809/3892820997","cluster_addr":"172.21.15.158:6811/3892820997","heartbeat_back_addr":"172.21.15.158:6815/3892820997","heartbeat_front_addr":"172.21.15.158:6813/3892820997","state":["exists","up"]},{"osd":5,"uuid":"d9c2e0c8-85f5-4ae2-bf3e-125c1be3a2db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3939495896},{"type":"v1","addr":"172.21.15.188:6809","nonce":3939495896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3939495896},{"type":"v1","addr":"172.21.15.188:6811","nonce":3939495896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3939495896},{"type":"v1","addr":"172.21.15.188:6815","nonce":3939495896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3939495896},{"type":"v1","addr":"172.21.15.188:6813","nonce":3939495896}]},"public_addr":"172.21.15.188:6809/3939495896","cluster_addr":"172.21.15.188:6811/3939495896","heartbeat_back_addr":"172.21.15.188:6815/3939495896","heartbeat_front_addr":"172.21.15.188:6813/3939495896","state":["exists","up"]},{"osd":6,"uuid":"4a523512-de6b-49f0-be81-00f2f00c1c10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":1206337767},{"type":"v1","addr":"172.21.15.12:6819","nonce":1206337767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":1206337767},{"type":"v1","addr":"172.21.15.12:6821","nonce":1206337767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":1206337767},{"type":"v1","addr":"172.21.15.12:6825","nonce":1206337767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":1206337767},{"type":"v1","addr":"172.21.15.12:6823","nonce":1206337767}]},"public_addr":"172.21.15.12:6819/1206337767","cluster_addr":"172.21.15.12:6821/1206337767","heartbeat_back_addr":"172.21.15.12:6825/1206337767","heartbeat_front_addr":"172.21.15.12:6823/1206337767","state":["exists","up"]},{"osd":7,"uuid":"363271a6-df93-44c9-b0e5-3d29768aa96c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":3283400032},{"type":"v1","addr":"172.21.15.158:6817","nonce":3283400032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":3283400032},{"type":"v1","addr":"172.21.15.158:6819","nonce":3283400032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":3283400032},{"type":"v1","addr":"172.21.15.158:6823","nonce":3283400032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":3283400032},{"type":"v1","addr":"172.21.15.158:6821","nonce":3283400032}]},"public_addr":"172.21.15.158:6817/3283400032","cluster_addr":"172.21.15.158:6819/3283400032","heartbeat_back_addr":"172.21.15.158:6823/3283400032","heartbeat_front_addr":"172.21.15.158:6821/3283400032","state":["exists","up"]},{"osd":8,"uuid":"9a089528-3e8e-45d3-9279-ddb54088b96a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2110170682},{"type":"v1","addr":"172.21.15.188:6817","nonce":2110170682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2110170682},{"type":"v1","addr":"172.21.15.188:6819","nonce":2110170682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":2110170682},{"type":"v1","addr":"172.21.15.188:6823","nonce":2110170682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2110170682},{"type":"v1","addr":"172.21.15.188:6821","nonce":2110170682}]},"public_addr":"172.21.15.188:6817/2110170682","cluster_addr":"172.21.15.188:6819/2110170682","heartbeat_back_addr":"172.21.15.188:6823/2110170682","heartbeat_front_addr":"172.21.15.188:6821/2110170682","state":["exists","up"]},{"osd":9,"uuid":"b12e2727-d649-4464-90e1-fafbc1754832","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1825928986},{"type":"v1","addr":"172.21.15.12:6827","nonce":1825928986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1825928986},{"type":"v1","addr":"172.21.15.12:6829","nonce":1825928986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":1825928986},{"type":"v1","addr":"172.21.15.12:6833","nonce":1825928986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1825928986},{"type":"v1","addr":"172.21.15.12:6831","nonce":1825928986}]},"public_addr":"172.21.15.12:6827/1825928986","cluster_addr":"172.21.15.12:6829/1825928986","heartbeat_back_addr":"172.21.15.12:6833/1825928986","heartbeat_front_addr":"172.21.15.12:6831/1825928986","state":["exists","up"]},{"osd":10,"uuid":"518b4cbf-9d90-4e9a-8ac9-b5cb9b3dee4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":3239052167},{"type":"v1","addr":"172.21.15.158:6825","nonce":3239052167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":3239052167},{"type":"v1","addr":"172.21.15.158:6827","nonce":3239052167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":3239052167},{"type":"v1","addr":"172.21.15.158:6831","nonce":3239052167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":3239052167},{"type":"v1","addr":"172.21.15.158:6829","nonce":3239052167}]},"public_addr":"172.21.15.158:6825/3239052167","cluster_addr":"172.21.15.158:6827/3239052167","heartbeat_back_addr":"172.21.15.158:6831/3239052167","heartbeat_front_addr":"172.21.15.158:6829/3239052167","state":["exists","up"]},{"osd":11,"uuid":"98aa3a0d-4b2f-46c8-9341-375d13794c3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1135773350},{"type":"v1","addr":"172.21.15.188:6825","nonce":1135773350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1135773350},{"type":"v1","addr":"172.21.15.188:6827","nonce":1135773350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1135773350},{"type":"v1","addr":"172.21.15.188:6831","nonce":1135773350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1135773350},{"type":"v1","addr":"172.21.15.188:6829","nonce":1135773350}]},"public_addr":"172.21.15.188:6825/1135773350","cluster_addr":"172.21.15.188:6827/1135773350","heartbeat_back_addr":"172.21.15.188:6831/1135773350","heartbeat_front_addr":"172.21.15.188:6829/1135773350","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:33:24.280879+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:33:47.859461+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:34:11.725283+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:34:35.707629+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:35:00.024197+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:35:24.499803+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:35:48.953870+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:36:13.642456+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:36:38.936457+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:37:04.021466+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:37:29.073178+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:37:54.863919+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:6801/4135274411":"2024-02-19T11:31:50.936928+0000","172.21.15.12:6800/4135274411":"2024-02-19T11:31:50.936928+0000","172.21.15.12:0/118428584":"2024-02-19T11:31:50.936928+0000","172.21.15.12:0/4276810915":"2024-02-19T11:31:50.936928+0000","172.21.15.12:0/3777545262":"2024-02-19T11:31:24.000830+0000","172.21.15.12:0/114707743":"2024-02-19T11:31:24.000830+0000","172.21.15.12:6800/3925204723":"2024-02-19T11:31:24.000830+0000","172.21.15.12:6801/3925204723":"2024-02-19T11:31:24.000830+0000","172.21.15.12:0/74881618":"2024-02-19T11:31:24.000830+0000","172.21.15.12:0/1614805375":"2024-02-19T11:31:50.936928+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-18T11:38:02.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:02.641+0000 7fdeb5603700 1 -- 172.21.15.12:0/61747542 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fde9c0a27e0 msgr2=0x7fde9c0a4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:02.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:02.641+0000 7fdeb5603700 1 --2- 172.21.15.12:0/61747542 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fde9c0a27e0 0x7fde9c0a4ca0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fdea0009550 tx=0x7fdea00091b0 comp rx=0 tx=0).stop 2024-02-18T11:38:02.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:02.641+0000 7fdeb5603700 1 -- 172.21.15.12:0/61747542 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdeb01026c0 msgr2=0x7fdeb00744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:02.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:02.641+0000 7fdeb5603700 1 --2- 172.21.15.12:0/61747542 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdeb01026c0 0x7fdeb00744f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fde9800c9b0 tx=0x7fde9800ccc0 comp rx=0 tx=0).stop 2024-02-18T11:38:02.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:02.641+0000 7fdeb5603700 1 -- 172.21.15.12:0/61747542 shutdown_connections 2024-02-18T11:38:02.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:02.641+0000 7fdeb5603700 1 --2- 172.21.15.12:0/61747542 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdeb0107ce0 0x7fdeb0074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:02.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:02.641+0000 7fdeb5603700 1 --2- 172.21.15.12:0/61747542 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdeb0101d00 0x7fdeb0073fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:02.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:02.641+0000 7fdeb5603700 1 --2- 172.21.15.12:0/61747542 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fde9c0a27e0 0x7fde9c0a4ca0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:02.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:02.641+0000 7fdeb5603700 1 --2- 172.21.15.12:0/61747542 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdeb01026c0 0x7fdeb00744f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:02.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:02.642+0000 7fdeb5603700 1 -- 172.21.15.12:0/61747542 >> 172.21.15.12:0/61747542 conn(0x7fdeb0078de0 msgr2=0x7fdeb01006e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:02.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:02.642+0000 7fdeb5603700 1 -- 172.21.15.12:0/61747542 shutdown_connections 2024-02-18T11:38:02.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:02.642+0000 7fdeb5603700 1 -- 172.21.15.12:0/61747542 wait complete. 2024-02-18T11:38:02.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:02 smithi012 ceph-mon[32460]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-18T11:38:02.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:02 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/61747542' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-18T11:38:03.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:02 smithi188 ceph-mon[36925]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-18T11:38:03.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:02 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/61747542' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-18T11:38:03.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:02 smithi158 ceph-mon[36626]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-18T11:38:03.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:02 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/61747542' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-18T11:38:03.334 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-18T11:34:15.281138+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-18T11:38:03.335 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-02-18T11:38:03.580 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T11:38:04.863 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:04 smithi012 ceph-mon[32460]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-18T11:38:05.047 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:04 smithi158 ceph-mon[36626]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-18T11:38:05.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:05.081+0000 7f1796e11700 1 -- 172.21.15.12:0/3301815338 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1790100e90 msgr2=0x7f1790101310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:05.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:05.081+0000 7f1796e11700 1 --2- 172.21.15.12:0/3301815338 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1790100e90 0x7f1790101310 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f177c009990 tx=0x7f177c009ca0 comp rx=0 tx=0).stop 2024-02-18T11:38:05.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:05.081+0000 7f1796e11700 1 -- 172.21.15.12:0/3301815338 shutdown_connections 2024-02-18T11:38:05.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:05.081+0000 7f1796e11700 1 --2- 172.21.15.12:0/3301815338 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1790101850 0x7f179010da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:05.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:05.081+0000 7f1796e11700 1 --2- 172.21.15.12:0/3301815338 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1790106da0 0x7f1790107180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:05.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:05.081+0000 7f1796e11700 1 --2- 172.21.15.12:0/3301815338 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1790100e90 0x7f1790101310 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:05.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:05.081+0000 7f1796e11700 1 -- 172.21.15.12:0/3301815338 >> 172.21.15.12:0/3301815338 conn(0x7f1790077a10 msgr2=0x7f1790077e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:05.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:05.081+0000 7f1796e11700 1 -- 172.21.15.12:0/3301815338 shutdown_connections 2024-02-18T11:38:05.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:05.081+0000 7f1796e11700 1 -- 172.21.15.12:0/3301815338 wait complete. 2024-02-18T11:38:05.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:05.082+0000 7f1796e11700 1 Processor -- start 2024-02-18T11:38:05.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:05.082+0000 7f1796e11700 1 -- start start 2024-02-18T11:38:05.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:05.082+0000 7f1796e11700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1790100e90 0x7f1790199090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:05.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:05.082+0000 7f1796e11700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1790101850 0x7f17901995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:05.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:05.082+0000 7f1796e11700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1790106da0 0x7f17901a4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:05.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:05.082+0000 7f1796e11700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1790199e70 con 0x7f1790100e90 2024-02-18T11:38:05.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:05.082+0000 7f1796e11700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17901a4650 con 0x7f1790101850 2024-02-18T11:38:05.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:05.082+0000 7f1796e11700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17901a4790 con 0x7f1790106da0 2024-02-18T11:38:05.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:05.082+0000 7f1794bad700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1790100e90 0x7f1790199090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:05.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:05.082+0000 7f1794bad700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1790100e90 0x7f1790199090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51550/0 (socket says 172.21.15.12:51550) 2024-02-18T11:38:05.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:05.082+0000 7f1794bad700 1 -- 172.21.15.12:0/734476140 learned_addr learned my addr 172.21.15.12:0/734476140 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:38:05.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:05.083+0000 7f17953ae700 1 --2- 172.21.15.12:0/734476140 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1790106da0 0x7f17901a4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:05.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:05.083+0000 7f178ffff700 1 --2- 172.21.15.12:0/734476140 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1790101850 0x7f17901995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:05.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:05.083+0000 7f1794bad700 1 -- 172.21.15.12:0/734476140 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1790106da0 msgr2=0x7f17901a4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:05.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:05.083+0000 7f1794bad700 1 --2- 172.21.15.12:0/734476140 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1790106da0 0x7f17901a4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:05.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:05.083+0000 7f1794bad700 1 -- 172.21.15.12:0/734476140 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1790101850 msgr2=0x7f17901995d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:05.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:05.083+0000 7f1794bad700 1 --2- 172.21.15.12:0/734476140 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1790101850 0x7f17901995d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:05.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:05.083+0000 7f1794bad700 1 -- 172.21.15.12:0/734476140 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f177c0095f0 con 0x7f1790100e90 2024-02-18T11:38:05.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:05.083+0000 7f17953ae700 1 --2- 172.21.15.12:0/734476140 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1790106da0 0x7f17901a4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:38:05.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:05.083+0000 7f178ffff700 1 --2- 172.21.15.12:0/734476140 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1790101850 0x7f17901995d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:38:05.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:05.083+0000 7f1794bad700 1 --2- 172.21.15.12:0/734476140 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1790100e90 0x7f1790199090 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f178400d9b0 tx=0x7f178400dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:05.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:05.083+0000 7f178dffb700 1 -- 172.21.15.12:0/734476140 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178400bb60 con 0x7f1790100e90 2024-02-18T11:38:05.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:05.083+0000 7f178dffb700 1 -- 172.21.15.12:0/734476140 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1784007e80 con 0x7f1790100e90 2024-02-18T11:38:05.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:05.083+0000 7f178dffb700 1 -- 172.21.15.12:0/734476140 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1784004e20 con 0x7f1790100e90 2024-02-18T11:38:05.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:05.083+0000 7f1796e11700 1 -- 172.21.15.12:0/734476140 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17901a4a40 con 0x7f1790100e90 2024-02-18T11:38:05.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:05.083+0000 7f1796e11700 1 -- 172.21.15.12:0/734476140 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17901a4ef0 con 0x7f1790100e90 2024-02-18T11:38:05.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:05.084+0000 7f1796e11700 1 -- 172.21.15.12:0/734476140 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17901091d0 con 0x7f1790100e90 2024-02-18T11:38:05.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:05.085+0000 7f178dffb700 1 -- 172.21.15.12:0/734476140 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) v1 ==== 137245+0+0 (secure 0 0 0) 0x7f1784004020 con 0x7f1790100e90 2024-02-18T11:38:05.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:05.086+0000 7f178dffb700 1 --2- 172.21.15.12:0/734476140 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f17800a27e0 0x7f17800a4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:05.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:05.086+0000 7f178ffff700 1 --2- 172.21.15.12:0/734476140 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f17800a27e0 0x7f17800a4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:05.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:05.086+0000 7f178dffb700 1 -- 172.21.15.12:0/734476140 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f17840cdf30 con 0x7f1790100e90 2024-02-18T11:38:05.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:05.086+0000 7f178ffff700 1 --2- 172.21.15.12:0/734476140 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f17800a27e0 0x7f17800a4ca0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f177c00b5c0 tx=0x7f177c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:05.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:05.090+0000 7f178dffb700 1 -- 172.21.15.12:0/734476140 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f17840970c0 con 0x7f1790100e90 2024-02-18T11:38:05.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:04 smithi188 ceph-mon[36925]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-18T11:38:05.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:05.290+0000 7f1796e11700 1 -- 172.21.15.12:0/734476140 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1790110cd0 con 0x7f1790100e90 2024-02-18T11:38:05.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:05.290+0000 7f178dffb700 1 -- 172.21.15.12:0/734476140 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7f17840160a0 con 0x7f1790100e90 2024-02-18T11:38:05.285 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-02-18T11:38:05.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:05.292+0000 7f1796e11700 1 -- 172.21.15.12:0/734476140 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f17800a27e0 msgr2=0x7f17800a4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:05.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:05.292+0000 7f1796e11700 1 --2- 172.21.15.12:0/734476140 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f17800a27e0 0x7f17800a4ca0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f177c00b5c0 tx=0x7f177c005fb0 comp rx=0 tx=0).stop 2024-02-18T11:38:05.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:05.292+0000 7f1796e11700 1 -- 172.21.15.12:0/734476140 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1790100e90 msgr2=0x7f1790199090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:05.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:05.292+0000 7f1796e11700 1 --2- 172.21.15.12:0/734476140 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1790100e90 0x7f1790199090 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f178400d9b0 tx=0x7f178400dd70 comp rx=0 tx=0).stop 2024-02-18T11:38:05.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:05.292+0000 7f1796e11700 1 -- 172.21.15.12:0/734476140 shutdown_connections 2024-02-18T11:38:05.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:05.292+0000 7f1796e11700 1 --2- 172.21.15.12:0/734476140 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1790106da0 0x7f17901a4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:05.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:05.292+0000 7f1796e11700 1 --2- 172.21.15.12:0/734476140 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f17800a27e0 0x7f17800a4ca0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:05.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:05.293+0000 7f1796e11700 1 --2- 172.21.15.12:0/734476140 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1790100e90 0x7f1790199090 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:05.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:05.293+0000 7f1796e11700 1 --2- 172.21.15.12:0/734476140 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1790101850 0x7f17901995d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:05.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:05.293+0000 7f1796e11700 1 -- 172.21.15.12:0/734476140 >> 172.21.15.12:0/734476140 conn(0x7f1790077a10 msgr2=0x7f17900ff910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:05.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:05.293+0000 7f1796e11700 1 -- 172.21.15.12:0/734476140 shutdown_connections 2024-02-18T11:38:05.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:05.293+0000 7f1796e11700 1 -- 172.21.15.12:0/734476140 wait complete. 2024-02-18T11:38:05.862 INFO:tasks.cephadm:Adding mds.a on smithi012 2024-02-18T11:38:05.862 INFO:tasks.cephadm:Adding mds.d on smithi012 2024-02-18T11:38:05.862 INFO:tasks.cephadm:Adding mds.g on smithi012 2024-02-18T11:38:05.863 INFO:tasks.cephadm:Adding mds.j on smithi012 2024-02-18T11:38:05.863 INFO:tasks.cephadm:Adding mds.b on smithi158 2024-02-18T11:38:05.863 INFO:tasks.cephadm:Adding mds.e on smithi158 2024-02-18T11:38:05.863 INFO:tasks.cephadm:Adding mds.h on smithi158 2024-02-18T11:38:05.863 INFO:tasks.cephadm:Adding mds.k on smithi158 2024-02-18T11:38:05.863 INFO:tasks.cephadm:Adding mds.c on smithi188 2024-02-18T11:38:05.864 INFO:tasks.cephadm:Adding mds.f on smithi188 2024-02-18T11:38:05.864 INFO:tasks.cephadm:Adding mds.i on smithi188 2024-02-18T11:38:05.864 INFO:tasks.cephadm:Adding mds.l on smithi188 2024-02-18T11:38:05.864 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph orch apply mds all '12;smithi012=a;smithi012=d;smithi012=g;smithi012=j;smithi158=b;smithi158=e;smithi158=h;smithi158=k;smithi188=c;smithi188=f;smithi188=i;smithi188=l' 2024-02-18T11:38:06.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:05 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/734476140' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-18T11:38:06.113 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.c/config 2024-02-18T11:38:06.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:05 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/734476140' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-18T11:38:06.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:05 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/734476140' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-18T11:38:06.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:06 smithi188 ceph-mon[36925]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-18T11:38:07.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:06 smithi012 ceph-mon[32460]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-18T11:38:07.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:06 smithi158 ceph-mon[36626]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-18T11:38:07.950 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:38:07.953+0000 7ff552d35700 1 -- 172.21.15.188:0/635206339 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff54c105090 msgr2=0x7ff54c107480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:07.950 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:38:07.953+0000 7ff552d35700 1 --2- 172.21.15.188:0/635206339 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff54c105090 0x7ff54c107480 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff53c009990 tx=0x7ff53c009ca0 comp rx=0 tx=0).stop 2024-02-18T11:38:07.950 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:38:07.953+0000 7ff552d35700 1 -- 172.21.15.188:0/635206339 shutdown_connections 2024-02-18T11:38:07.950 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:38:07.953+0000 7ff552d35700 1 --2- 172.21.15.188:0/635206339 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff54c1079c0 0x7ff54c109e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:07.950 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:38:07.953+0000 7ff552d35700 1 --2- 172.21.15.188:0/635206339 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff54c102760 0x7ff54c104b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:07.950 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:38:07.953+0000 7ff552d35700 1 --2- 172.21.15.188:0/635206339 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff54c105090 0x7ff54c107480 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:07.950 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:38:07.953+0000 7ff552d35700 1 -- 172.21.15.188:0/635206339 >> 172.21.15.188:0/635206339 conn(0x7ff54c0fc170 msgr2=0x7ff54c0fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:07.950 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:38:07.953+0000 7ff552d35700 1 -- 172.21.15.188:0/635206339 shutdown_connections 2024-02-18T11:38:07.950 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:38:07.953+0000 7ff552d35700 1 -- 172.21.15.188:0/635206339 wait complete. 2024-02-18T11:38:07.950 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:38:07.953+0000 7ff552d35700 1 Processor -- start 2024-02-18T11:38:07.950 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:38:07.953+0000 7ff552d35700 1 -- start start 2024-02-18T11:38:07.950 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:38:07.954+0000 7ff552d35700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff54c102760 0x7ff54c197cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:07.951 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:38:07.954+0000 7ff552d35700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff54c105090 0x7ff54c1981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:07.951 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:38:07.954+0000 7ff552d35700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff54c1079c0 0x7ff54c19f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:07.951 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:38:07.954+0000 7ff552d35700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c19f790 con 0x7ff54c105090 2024-02-18T11:38:07.951 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:38:07.954+0000 7ff552d35700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c19f8d0 con 0x7ff54c102760 2024-02-18T11:38:07.951 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:38:07.954+0000 7ff552d35700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c19fa90 con 0x7ff54c1079c0 2024-02-18T11:38:07.951 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:38:07.954+0000 7ff5512d2700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff54c1079c0 0x7ff54c19f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:07.951 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:38:07.954+0000 7ff5512d2700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff54c1079c0 0x7ff54c19f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:38690/0 (socket says 172.21.15.188:38690) 2024-02-18T11:38:07.951 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:38:07.954+0000 7ff5512d2700 1 -- 172.21.15.188:0/4046652221 learned_addr learned my addr 172.21.15.188:0/4046652221 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-02-18T11:38:07.951 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:38:07.954+0000 7ff543fff700 1 --2- 172.21.15.188:0/4046652221 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff54c105090 0x7ff54c1981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:07.951 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:38:07.954+0000 7ff550ad1700 1 --2- 172.21.15.188:0/4046652221 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff54c102760 0x7ff54c197cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:07.951 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:38:07.954+0000 7ff5512d2700 1 -- 172.21.15.188:0/4046652221 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff54c102760 msgr2=0x7ff54c197cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:07.951 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:38:07.954+0000 7ff5512d2700 1 --2- 172.21.15.188:0/4046652221 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff54c102760 0x7ff54c197cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:07.951 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:38:07.954+0000 7ff5512d2700 1 -- 172.21.15.188:0/4046652221 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff54c105090 msgr2=0x7ff54c1981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:07.951 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:38:07.954+0000 7ff5512d2700 1 --2- 172.21.15.188:0/4046652221 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff54c105090 0x7ff54c1981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:07.951 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:38:07.954+0000 7ff5512d2700 1 -- 172.21.15.188:0/4046652221 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff53c0095f0 con 0x7ff54c1079c0 2024-02-18T11:38:07.952 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:38:07.954+0000 7ff5512d2700 1 --2- 172.21.15.188:0/4046652221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff54c1079c0 0x7ff54c19f1c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff54400b480 tx=0x7ff54400b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:07.952 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:38:07.954+0000 7ff541ffb700 1 -- 172.21.15.188:0/4046652221 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff544011830 con 0x7ff54c1079c0 2024-02-18T11:38:07.952 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:38:07.954+0000 7ff541ffb700 1 -- 172.21.15.188:0/4046652221 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff544011e70 con 0x7ff54c1079c0 2024-02-18T11:38:07.952 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:38:07.954+0000 7ff552d35700 1 -- 172.21.15.188:0/4046652221 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff54c19fd70 con 0x7ff54c1079c0 2024-02-18T11:38:07.952 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:38:07.955+0000 7ff541ffb700 1 -- 172.21.15.188:0/4046652221 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff54400f2e0 con 0x7ff54c1079c0 2024-02-18T11:38:07.952 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:38:07.955+0000 7ff552d35700 1 -- 172.21.15.188:0/4046652221 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff54c1a01c0 con 0x7ff54c1079c0 2024-02-18T11:38:07.953 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:38:07.956+0000 7ff552d35700 1 -- 172.21.15.188:0/4046652221 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff54c0fdcf0 con 0x7ff54c1079c0 2024-02-18T11:38:07.953 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:38:07.956+0000 7ff541ffb700 1 -- 172.21.15.188:0/4046652221 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 16) v1 ==== 137245+0+0 (secure 0 0 0) 0x7ff544010610 con 0x7ff54c1079c0 2024-02-18T11:38:07.953 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:38:07.957+0000 7ff541ffb700 1 --2- 172.21.15.188:0/4046652221 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7ff5340a27e0 0x7ff5340a4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:07.953 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:38:07.957+0000 7ff541ffb700 1 -- 172.21.15.188:0/4046652221 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff5440cdac0 con 0x7ff54c1079c0 2024-02-18T11:38:07.953 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:38:07.957+0000 7ff550ad1700 1 --2- 172.21.15.188:0/4046652221 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7ff5340a27e0 0x7ff5340a4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:07.953 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:38:07.958+0000 7ff550ad1700 1 --2- 172.21.15.188:0/4046652221 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7ff5340a27e0 0x7ff5340a4ca0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff54c199120 tx=0x7ff548009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:07.956 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:38:07.962+0000 7ff541ffb700 1 -- 172.21.15.188:0/4046652221 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff544096cf0 con 0x7ff54c1079c0 2024-02-18T11:38:08.160 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:38:08.165+0000 7ff552d35700 1 -- 172.21.15.188:0/4046652221 --> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi012=a;smithi012=d;smithi012=g;smithi012=j;smithi158=b;smithi158=e;smithi158=h;smithi158=k;smithi188=c;smithi188=f;smithi188=i;smithi188=l", "target": ["mon-mgr", ""]}) v1 -- 0x7ff54c02da50 con 0x7ff5340a27e0 2024-02-18T11:38:08.168 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:38:08.173+0000 7ff541ffb700 1 -- 172.21.15.188:0/4046652221 <== mgr.14152 v2:172.21.15.12:6800/1153736685 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7ff54c02da50 con 0x7ff5340a27e0 2024-02-18T11:38:08.168 INFO:teuthology.orchestra.run.smithi188.stdout:Scheduled mds.all update... 2024-02-18T11:38:08.170 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:38:08.175+0000 7ff552d35700 1 -- 172.21.15.188:0/4046652221 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7ff5340a27e0 msgr2=0x7ff5340a4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:08.171 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:38:08.175+0000 7ff552d35700 1 --2- 172.21.15.188:0/4046652221 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7ff5340a27e0 0x7ff5340a4ca0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff54c199120 tx=0x7ff548009040 comp rx=0 tx=0).stop 2024-02-18T11:38:08.171 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:38:08.175+0000 7ff552d35700 1 -- 172.21.15.188:0/4046652221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff54c1079c0 msgr2=0x7ff54c19f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:08.171 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:38:08.175+0000 7ff552d35700 1 --2- 172.21.15.188:0/4046652221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff54c1079c0 0x7ff54c19f1c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff54400b480 tx=0x7ff54400b790 comp rx=0 tx=0).stop 2024-02-18T11:38:08.171 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:38:08.176+0000 7ff552d35700 1 -- 172.21.15.188:0/4046652221 shutdown_connections 2024-02-18T11:38:08.171 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:38:08.176+0000 7ff552d35700 1 --2- 172.21.15.188:0/4046652221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff54c1079c0 0x7ff54c19f1c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:08.171 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:38:08.176+0000 7ff552d35700 1 --2- 172.21.15.188:0/4046652221 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff54c102760 0x7ff54c197cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:08.171 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:38:08.176+0000 7ff552d35700 1 --2- 172.21.15.188:0/4046652221 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7ff5340a27e0 0x7ff5340a4ca0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:08.171 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:38:08.176+0000 7ff552d35700 1 --2- 172.21.15.188:0/4046652221 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff54c105090 0x7ff54c1981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:08.171 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:38:08.176+0000 7ff552d35700 1 -- 172.21.15.188:0/4046652221 >> 172.21.15.188:0/4046652221 conn(0x7ff54c0fc170 msgr2=0x7ff54c1030c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:08.171 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:38:08.176+0000 7ff552d35700 1 -- 172.21.15.188:0/4046652221 shutdown_connections 2024-02-18T11:38:08.171 INFO:teuthology.orchestra.run.smithi188.stderr:2024-02-18T11:38:08.176+0000 7ff552d35700 1 -- 172.21.15.188:0/4046652221 wait complete. 2024-02-18T11:38:08.795 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:08 smithi188 ceph-mon[36925]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-18T11:38:08.795 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:08 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:08.796 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:08 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:38:08.821 DEBUG:teuthology.orchestra.run.smithi012:mds.a> sudo journalctl -f -n 0 -u ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@mds.a.service 2024-02-18T11:38:08.825 DEBUG:teuthology.orchestra.run.smithi012:mds.d> sudo journalctl -f -n 0 -u ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@mds.d.service 2024-02-18T11:38:08.868 DEBUG:teuthology.orchestra.run.smithi012:mds.g> sudo journalctl -f -n 0 -u ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@mds.g.service 2024-02-18T11:38:08.911 DEBUG:teuthology.orchestra.run.smithi012:mds.j> sudo journalctl -f -n 0 -u ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@mds.j.service 2024-02-18T11:38:08.954 DEBUG:teuthology.orchestra.run.smithi158:mds.b> sudo journalctl -f -n 0 -u ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@mds.b.service 2024-02-18T11:38:08.956 DEBUG:teuthology.orchestra.run.smithi158:mds.e> sudo journalctl -f -n 0 -u ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@mds.e.service 2024-02-18T11:38:08.999 DEBUG:teuthology.orchestra.run.smithi158:mds.h> sudo journalctl -f -n 0 -u ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@mds.h.service 2024-02-18T11:38:09.020 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:08 smithi158 ceph-mon[36626]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-18T11:38:09.020 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:08 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:09.020 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:08 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:38:09.021 DEBUG:teuthology.orchestra.run.smithi158:mds.k> sudo journalctl -f -n 0 -u ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@mds.k.service 2024-02-18T11:38:09.064 DEBUG:teuthology.orchestra.run.smithi188:mds.c> sudo journalctl -f -n 0 -u ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@mds.c.service 2024-02-18T11:38:09.067 DEBUG:teuthology.orchestra.run.smithi188:mds.f> sudo journalctl -f -n 0 -u ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@mds.f.service 2024-02-18T11:38:09.110 DEBUG:teuthology.orchestra.run.smithi188:mds.i> sudo journalctl -f -n 0 -u ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@mds.i.service 2024-02-18T11:38:09.152 DEBUG:teuthology.orchestra.run.smithi188:mds.l> sudo journalctl -f -n 0 -u ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@mds.l.service 2024-02-18T11:38:09.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:08 smithi012 ceph-mon[32460]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-18T11:38:09.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:08 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:09.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:08 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:38:09.196 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-02-18T11:38:09.196 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-18T11:38:09.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.316+0000 7f3720fc9640 1 -- 172.21.15.12:0/625632302 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f371c0ff260 msgr2=0x7f371c0ff6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:09.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.316+0000 7f3720fc9640 1 --2- 172.21.15.12:0/625632302 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f371c0ff260 0x7f371c0ff6e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3704009860 tx=0x7f370402bb70 comp rx=0 tx=0).stop 2024-02-18T11:38:09.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.316+0000 7f3720fc9640 1 -- 172.21.15.12:0/625632302 shutdown_connections 2024-02-18T11:38:09.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.316+0000 7f3720fc9640 1 --2- 172.21.15.12:0/625632302 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f371c0ffc20 0x7f371c1097d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:09.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.316+0000 7f3720fc9640 1 --2- 172.21.15.12:0/625632302 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f371c0ff260 0x7f371c0ff6e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:09.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.316+0000 7f3720fc9640 1 --2- 172.21.15.12:0/625632302 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f371c1027f0 0x7f371c102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:09.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.316+0000 7f3720fc9640 1 -- 172.21.15.12:0/625632302 >> 172.21.15.12:0/625632302 conn(0x7f371c0fae00 msgr2=0x7f371c0fd240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:09.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.316+0000 7f3720fc9640 1 -- 172.21.15.12:0/625632302 shutdown_connections 2024-02-18T11:38:09.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.316+0000 7f3720fc9640 1 -- 172.21.15.12:0/625632302 wait complete. 2024-02-18T11:38:09.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.317+0000 7f3720fc9640 1 Processor -- start 2024-02-18T11:38:09.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.317+0000 7f3720fc9640 1 -- start start 2024-02-18T11:38:09.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.317+0000 7f3720fc9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f371c0ff260 0x7f371c1972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:09.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.317+0000 7f3720fc9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f371c0ffc20 0x7f371c1977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:09.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.317+0000 7f3720fc9640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f371c1027f0 0x7f371c19e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:09.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.317+0000 7f3720fc9640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f371c19ed90 con 0x7f371c0ff260 2024-02-18T11:38:09.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.318+0000 7f3720fc9640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f371c19ef00 con 0x7f371c1027f0 2024-02-18T11:38:09.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.318+0000 7f3720fc9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f371c19f0c0 con 0x7f371c0ffc20 2024-02-18T11:38:09.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.318+0000 7f371a59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f371c0ff260 0x7f371c1972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:09.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.318+0000 7f371a59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f371c0ff260 0x7f371c1972a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51580/0 (socket says 172.21.15.12:51580) 2024-02-18T11:38:09.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.318+0000 7f371a59c640 1 -- 172.21.15.12:0/3424588739 learned_addr learned my addr 172.21.15.12:0/3424588739 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:38:09.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.318+0000 7f3711d9b640 1 --2- 172.21.15.12:0/3424588739 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f371c0ffc20 0x7f371c1977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:09.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.318+0000 7f371ad9d640 1 --2- 172.21.15.12:0/3424588739 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f371c1027f0 0x7f371c19e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:09.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.318+0000 7f371a59c640 1 -- 172.21.15.12:0/3424588739 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f371c0ffc20 msgr2=0x7f371c1977e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:09.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.318+0000 7f371a59c640 1 --2- 172.21.15.12:0/3424588739 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f371c0ffc20 0x7f371c1977e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:09.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.318+0000 7f371a59c640 1 -- 172.21.15.12:0/3424588739 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f371c1027f0 msgr2=0x7f371c19e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:09.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.318+0000 7f371a59c640 1 --2- 172.21.15.12:0/3424588739 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f371c1027f0 0x7f371c19e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:09.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.318+0000 7f371a59c640 1 -- 172.21.15.12:0/3424588739 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3704009490 con 0x7f371c0ff260 2024-02-18T11:38:09.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.318+0000 7f371a59c640 1 --2- 172.21.15.12:0/3424588739 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f371c0ff260 0x7f371c1972a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f370c00e9c0 tx=0x7f370c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:09.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.318+0000 7f3713fff640 1 -- 172.21.15.12:0/3424588739 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f370c009700 con 0x7f371c0ff260 2024-02-18T11:38:09.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.318+0000 7f3720fc9640 1 -- 172.21.15.12:0/3424588739 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f371c19f3a0 con 0x7f371c0ff260 2024-02-18T11:38:09.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.318+0000 7f3713fff640 1 -- 172.21.15.12:0/3424588739 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f370c00cd80 con 0x7f371c0ff260 2024-02-18T11:38:09.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.318+0000 7f3713fff640 1 -- 172.21.15.12:0/3424588739 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f370c010430 con 0x7f371c0ff260 2024-02-18T11:38:09.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.318+0000 7f3720fc9640 1 -- 172.21.15.12:0/3424588739 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f371c19f880 con 0x7f371c0ff260 2024-02-18T11:38:09.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.319+0000 7f3720fc9640 1 -- 172.21.15.12:0/3424588739 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f371c1086b0 con 0x7f371c0ff260 2024-02-18T11:38:09.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.320+0000 7f3713fff640 1 -- 172.21.15.12:0/3424588739 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) v1 ==== 137245+0+0 (secure 0 0 0) 0x7f370c003680 con 0x7f371c0ff260 2024-02-18T11:38:09.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.321+0000 7f3713fff640 1 --2- 172.21.15.12:0/3424588739 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f37080a2750 0x7f37080a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:09.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.321+0000 7f3711d9b640 1 --2- 172.21.15.12:0/3424588739 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f37080a2750 0x7f37080a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:09.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.321+0000 7f3713fff640 1 -- 172.21.15.12:0/3424588739 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f370c014070 con 0x7f371c0ff260 2024-02-18T11:38:09.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.321+0000 7f3711d9b640 1 --2- 172.21.15.12:0/3424588739 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f37080a2750 0x7f37080a4c10 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f370402c180 tx=0x7f3704034c00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:09.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.326+0000 7f3713fff640 1 -- 172.21.15.12:0/3424588739 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f370c097730 con 0x7f371c0ff260 2024-02-18T11:38:09.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.507+0000 7f3720fc9640 1 -- 172.21.15.12:0/3424588739 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f371c10cc50 con 0x7f371c0ff260 2024-02-18T11:38:09.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.508+0000 7f3713fff640 1 -- 172.21.15.12:0/3424588739 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+18466 (secure 0 0 0) 0x7f370c09adf0 con 0x7f371c0ff260 2024-02-18T11:38:09.503 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:38:09.503 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":83,"fsid":"0c8bb0de-ce51-11ee-95ba-87774f69a715","created":"2024-02-18T11:30:49.110059+0000","modified":"2024-02-18T11:37:58.739798+0000","last_up_change":"2024-02-18T11:37:57.512337+0000","last_in_change":"2024-02-18T11:37:36.567969+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-18T11:34:15.281138+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"35f15ea6-deb8-48a1-a7dd-ab649ec13533","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":536522426},{"type":"v1","addr":"172.21.15.12:6803","nonce":536522426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":536522426},{"type":"v1","addr":"172.21.15.12:6805","nonce":536522426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":536522426},{"type":"v1","addr":"172.21.15.12:6809","nonce":536522426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":536522426},{"type":"v1","addr":"172.21.15.12:6807","nonce":536522426}]},"public_addr":"172.21.15.12:6803/536522426","cluster_addr":"172.21.15.12:6805/536522426","heartbeat_back_addr":"172.21.15.12:6809/536522426","heartbeat_front_addr":"172.21.15.12:6807/536522426","state":["exists","up"]},{"osd":1,"uuid":"2af937ef-d827-44e6-b805-70cffbfe1126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":4249111488},{"type":"v1","addr":"172.21.15.158:6801","nonce":4249111488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":4249111488},{"type":"v1","addr":"172.21.15.158:6803","nonce":4249111488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":4249111488},{"type":"v1","addr":"172.21.15.158:6807","nonce":4249111488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":4249111488},{"type":"v1","addr":"172.21.15.158:6805","nonce":4249111488}]},"public_addr":"172.21.15.158:6801/4249111488","cluster_addr":"172.21.15.158:6803/4249111488","heartbeat_back_addr":"172.21.15.158:6807/4249111488","heartbeat_front_addr":"172.21.15.158:6805/4249111488","state":["exists","up"]},{"osd":2,"uuid":"17b804dd-31df-45ae-a637-4a5f65bbaf26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":627976150},{"type":"v1","addr":"172.21.15.188:6801","nonce":627976150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":627976150},{"type":"v1","addr":"172.21.15.188:6803","nonce":627976150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":627976150},{"type":"v1","addr":"172.21.15.188:6807","nonce":627976150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":627976150},{"type":"v1","addr":"172.21.15.188:6805","nonce":627976150}]},"public_addr":"172.21.15.188:6801/627976150","cluster_addr":"172.21.15.188:6803/627976150","heartbeat_back_addr":"172.21.15.188:6807/627976150","heartbeat_front_addr":"172.21.15.188:6805/627976150","state":["exists","up"]},{"osd":3,"uuid":"3f9cc423-cdfa-4acf-aaa2-b58b32ccb9cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":3758512503},{"type":"v1","addr":"172.21.15.12:6811","nonce":3758512503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":3758512503},{"type":"v1","addr":"172.21.15.12:6813","nonce":3758512503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3758512503},{"type":"v1","addr":"172.21.15.12:6817","nonce":3758512503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":3758512503},{"type":"v1","addr":"172.21.15.12:6815","nonce":3758512503}]},"public_addr":"172.21.15.12:6811/3758512503","cluster_addr":"172.21.15.12:6813/3758512503","heartbeat_back_addr":"172.21.15.12:6817/3758512503","heartbeat_front_addr":"172.21.15.12:6815/3758512503","state":["exists","up"]},{"osd":4,"uuid":"a92e20ec-af1c-4ef4-a0cb-caba5fcc71aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":3892820997},{"type":"v1","addr":"172.21.15.158:6809","nonce":3892820997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":3892820997},{"type":"v1","addr":"172.21.15.158:6811","nonce":3892820997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":3892820997},{"type":"v1","addr":"172.21.15.158:6815","nonce":3892820997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":3892820997},{"type":"v1","addr":"172.21.15.158:6813","nonce":3892820997}]},"public_addr":"172.21.15.158:6809/3892820997","cluster_addr":"172.21.15.158:6811/3892820997","heartbeat_back_addr":"172.21.15.158:6815/3892820997","heartbeat_front_addr":"172.21.15.158:6813/3892820997","state":["exists","up"]},{"osd":5,"uuid":"d9c2e0c8-85f5-4ae2-bf3e-125c1be3a2db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3939495896},{"type":"v1","addr":"172.21.15.188:6809","nonce":3939495896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3939495896},{"type":"v1","addr":"172.21.15.188:6811","nonce":3939495896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3939495896},{"type":"v1","addr":"172.21.15.188:6815","nonce":3939495896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3939495896},{"type":"v1","addr":"172.21.15.188:6813","nonce":3939495896}]},"public_addr":"172.21.15.188:6809/3939495896","cluster_addr":"172.21.15.188:6811/3939495896","heartbeat_back_addr":"172.21.15.188:6815/3939495896","heartbeat_front_addr":"172.21.15.188:6813/3939495896","state":["exists","up"]},{"osd":6,"uuid":"4a523512-de6b-49f0-be81-00f2f00c1c10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":1206337767},{"type":"v1","addr":"172.21.15.12:6819","nonce":1206337767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":1206337767},{"type":"v1","addr":"172.21.15.12:6821","nonce":1206337767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":1206337767},{"type":"v1","addr":"172.21.15.12:6825","nonce":1206337767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":1206337767},{"type":"v1","addr":"172.21.15.12:6823","nonce":1206337767}]},"public_addr":"172.21.15.12:6819/1206337767","cluster_addr":"172.21.15.12:6821/1206337767","heartbeat_back_addr":"172.21.15.12:6825/1206337767","heartbeat_front_addr":"172.21.15.12:6823/1206337767","state":["exists","up"]},{"osd":7,"uuid":"363271a6-df93-44c9-b0e5-3d29768aa96c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":3283400032},{"type":"v1","addr":"172.21.15.158:6817","nonce":3283400032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":3283400032},{"type":"v1","addr":"172.21.15.158:6819","nonce":3283400032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":3283400032},{"type":"v1","addr":"172.21.15.158:6823","nonce":3283400032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":3283400032},{"type":"v1","addr":"172.21.15.158:6821","nonce":3283400032}]},"public_addr":"172.21.15.158:6817/3283400032","cluster_addr":"172.21.15.158:6819/3283400032","heartbeat_back_addr":"172.21.15.158:6823/3283400032","heartbeat_front_addr":"172.21.15.158:6821/3283400032","state":["exists","up"]},{"osd":8,"uuid":"9a089528-3e8e-45d3-9279-ddb54088b96a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2110170682},{"type":"v1","addr":"172.21.15.188:6817","nonce":2110170682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2110170682},{"type":"v1","addr":"172.21.15.188:6819","nonce":2110170682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":2110170682},{"type":"v1","addr":"172.21.15.188:6823","nonce":2110170682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2110170682},{"type":"v1","addr":"172.21.15.188:6821","nonce":2110170682}]},"public_addr":"172.21.15.188:6817/2110170682","cluster_addr":"172.21.15.188:6819/2110170682","heartbeat_back_addr":"172.21.15.188:6823/2110170682","heartbeat_front_addr":"172.21.15.188:6821/2110170682","state":["exists","up"]},{"osd":9,"uuid":"b12e2727-d649-4464-90e1-fafbc1754832","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1825928986},{"type":"v1","addr":"172.21.15.12:6827","nonce":1825928986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1825928986},{"type":"v1","addr":"172.21.15.12:6829","nonce":1825928986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":1825928986},{"type":"v1","addr":"172.21.15.12:6833","nonce":1825928986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1825928986},{"type":"v1","addr":"172.21.15.12:6831","nonce":1825928986}]},"public_addr":"172.21.15.12:6827/1825928986","cluster_addr":"172.21.15.12:6829/1825928986","heartbeat_back_addr":"172.21.15.12:6833/1825928986","heartbeat_front_addr":"172.21.15.12:6831/1825928986","state":["exists","up"]},{"osd":10,"uuid":"518b4cbf-9d90-4e9a-8ac9-b5cb9b3dee4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":3239052167},{"type":"v1","addr":"172.21.15.158:6825","nonce":3239052167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":3239052167},{"type":"v1","addr":"172.21.15.158:6827","nonce":3239052167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":3239052167},{"type":"v1","addr":"172.21.15.158:6831","nonce":3239052167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":3239052167},{"type":"v1","addr":"172.21.15.158:6829","nonce":3239052167}]},"public_addr":"172.21.15.158:6825/3239052167","cluster_addr":"172.21.15.158:6827/3239052167","heartbeat_back_addr":"172.21.15.158:6831/3239052167","heartbeat_front_addr":"172.21.15.158:6829/3239052167","state":["exists","up"]},{"osd":11,"uuid":"98aa3a0d-4b2f-46c8-9341-375d13794c3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1135773350},{"type":"v1","addr":"172.21.15.188:6825","nonce":1135773350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1135773350},{"type":"v1","addr":"172.21.15.188:6827","nonce":1135773350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1135773350},{"type":"v1","addr":"172.21.15.188:6831","nonce":1135773350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1135773350},{"type":"v1","addr":"172.21.15.188:6829","nonce":1135773350}]},"public_addr":"172.21.15.188:6825/1135773350","cluster_addr":"172.21.15.188:6827/1135773350","heartbeat_back_addr":"172.21.15.188:6831/1135773350","heartbeat_front_addr":"172.21.15.188:6829/1135773350","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:33:24.280879+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:33:47.859461+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:34:11.725283+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:34:35.707629+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:35:00.024197+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:35:24.499803+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:35:48.953870+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:36:13.642456+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:36:38.936457+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:37:04.021466+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:37:29.073178+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:37:54.863919+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:6801/4135274411":"2024-02-19T11:31:50.936928+0000","172.21.15.12:6800/4135274411":"2024-02-19T11:31:50.936928+0000","172.21.15.12:0/118428584":"2024-02-19T11:31:50.936928+0000","172.21.15.12:0/4276810915":"2024-02-19T11:31:50.936928+0000","172.21.15.12:0/3777545262":"2024-02-19T11:31:24.000830+0000","172.21.15.12:0/114707743":"2024-02-19T11:31:24.000830+0000","172.21.15.12:6800/3925204723":"2024-02-19T11:31:24.000830+0000","172.21.15.12:6801/3925204723":"2024-02-19T11:31:24.000830+0000","172.21.15.12:0/74881618":"2024-02-19T11:31:24.000830+0000","172.21.15.12:0/1614805375":"2024-02-19T11:31:50.936928+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-18T11:38:09.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.510+0000 7f3720fc9640 1 -- 172.21.15.12:0/3424588739 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f37080a2750 msgr2=0x7f37080a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:09.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.510+0000 7f3720fc9640 1 --2- 172.21.15.12:0/3424588739 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f37080a2750 0x7f37080a4c10 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f370402c180 tx=0x7f3704034c00 comp rx=0 tx=0).stop 2024-02-18T11:38:09.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.510+0000 7f3720fc9640 1 -- 172.21.15.12:0/3424588739 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f371c0ff260 msgr2=0x7f371c1972a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:09.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.510+0000 7f3720fc9640 1 --2- 172.21.15.12:0/3424588739 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f371c0ff260 0x7f371c1972a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f370c00e9c0 tx=0x7f370c00ee80 comp rx=0 tx=0).stop 2024-02-18T11:38:09.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.510+0000 7f3720fc9640 1 -- 172.21.15.12:0/3424588739 shutdown_connections 2024-02-18T11:38:09.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.510+0000 7f3720fc9640 1 --2- 172.21.15.12:0/3424588739 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f37080a2750 0x7f37080a4c10 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:09.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.510+0000 7f3720fc9640 1 --2- 172.21.15.12:0/3424588739 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f371c0ffc20 0x7f371c1977e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:09.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.510+0000 7f3720fc9640 1 --2- 172.21.15.12:0/3424588739 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f371c1027f0 0x7f371c19e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:09.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.510+0000 7f3720fc9640 1 --2- 172.21.15.12:0/3424588739 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f371c0ff260 0x7f371c1972a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:09.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.510+0000 7f3720fc9640 1 -- 172.21.15.12:0/3424588739 >> 172.21.15.12:0/3424588739 conn(0x7f371c0fae00 msgr2=0x7f371c105880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:09.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.510+0000 7f3720fc9640 1 -- 172.21.15.12:0/3424588739 shutdown_connections 2024-02-18T11:38:09.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.510+0000 7f3720fc9640 1 -- 172.21.15.12:0/3424588739 wait complete. 2024-02-18T11:38:09.516 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-18T11:34:15.281138+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-18T11:38:09.516 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-18T11:38:09.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.628+0000 7fa2ed088640 1 -- 172.21.15.12:0/3675619540 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2e8101c80 msgr2=0x7fa2e8108510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:09.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.628+0000 7fa2ed088640 1 --2- 172.21.15.12:0/3675619540 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2e8101c80 0x7fa2e8108510 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa2e0009ca0 tx=0x7fa2e002c0f0 comp rx=0 tx=0).stop 2024-02-18T11:38:09.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.628+0000 7fa2ed088640 1 -- 172.21.15.12:0/3675619540 shutdown_connections 2024-02-18T11:38:09.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.628+0000 7fa2ed088640 1 --2- 172.21.15.12:0/3675619540 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa2e81012c0 0x7fa2e8101740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:09.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.628+0000 7fa2ed088640 1 --2- 172.21.15.12:0/3675619540 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2e8101c80 0x7fa2e8108510 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:09.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.628+0000 7fa2ed088640 1 --2- 172.21.15.12:0/3675619540 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa2e81000d0 0x7fa2e81004f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:09.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.628+0000 7fa2ed088640 1 -- 172.21.15.12:0/3675619540 >> 172.21.15.12:0/3675619540 conn(0x7fa2e80fb660 msgr2=0x7fa2e80fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:09.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.628+0000 7fa2ed088640 1 -- 172.21.15.12:0/3675619540 shutdown_connections 2024-02-18T11:38:09.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.628+0000 7fa2ed088640 1 -- 172.21.15.12:0/3675619540 wait complete. 2024-02-18T11:38:09.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.629+0000 7fa2ed088640 1 Processor -- start 2024-02-18T11:38:09.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.629+0000 7fa2ed088640 1 -- start start 2024-02-18T11:38:09.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.629+0000 7fa2ed088640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2e81000d0 0x7fa2e8197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:09.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.629+0000 7fa2ed088640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa2e81012c0 0x7fa2e8197770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:09.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.629+0000 7fa2ed088640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa2e8101c80 0x7fa2e819e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:09.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.629+0000 7fa2ed088640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e819ed20 con 0x7fa2e81000d0 2024-02-18T11:38:09.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.629+0000 7fa2ed088640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e819ee90 con 0x7fa2e81012c0 2024-02-18T11:38:09.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.629+0000 7fa2ed088640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e819f050 con 0x7fa2e8101c80 2024-02-18T11:38:09.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.629+0000 7fa2e6d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2e81000d0 0x7fa2e8197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:09.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.629+0000 7fa2e6d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2e81000d0 0x7fa2e8197230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51600/0 (socket says 172.21.15.12:51600) 2024-02-18T11:38:09.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.629+0000 7fa2e6d9d640 1 -- 172.21.15.12:0/3218696960 learned_addr learned my addr 172.21.15.12:0/3218696960 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:38:09.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.629+0000 7fa2de59c640 1 --2- 172.21.15.12:0/3218696960 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa2e81012c0 0x7fa2e8197770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:09.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.630+0000 7fa2e759e640 1 --2- 172.21.15.12:0/3218696960 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa2e8101c80 0x7fa2e819e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:09.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.630+0000 7fa2e6d9d640 1 -- 172.21.15.12:0/3218696960 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa2e8101c80 msgr2=0x7fa2e819e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:09.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.630+0000 7fa2e6d9d640 1 --2- 172.21.15.12:0/3218696960 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa2e8101c80 0x7fa2e819e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:09.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.630+0000 7fa2e6d9d640 1 -- 172.21.15.12:0/3218696960 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa2e81012c0 msgr2=0x7fa2e8197770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:09.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.630+0000 7fa2e6d9d640 1 --2- 172.21.15.12:0/3218696960 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa2e81012c0 0x7fa2e8197770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:09.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.630+0000 7fa2e6d9d640 1 -- 172.21.15.12:0/3218696960 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2e0009900 con 0x7fa2e81000d0 2024-02-18T11:38:09.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.630+0000 7fa2e759e640 1 --2- 172.21.15.12:0/3218696960 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa2e8101c80 0x7fa2e819e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:38:09.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.630+0000 7fa2e6d9d640 1 --2- 172.21.15.12:0/3218696960 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2e81000d0 0x7fa2e8197230 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa2d800d950 tx=0x7fa2d800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:09.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.630+0000 7fa2e4d99640 1 -- 172.21.15.12:0/3218696960 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d8009c30 con 0x7fa2e81000d0 2024-02-18T11:38:09.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.630+0000 7fa2ed088640 1 -- 172.21.15.12:0/3218696960 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2e819f330 con 0x7fa2e81000d0 2024-02-18T11:38:09.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.630+0000 7fa2e4d99640 1 -- 172.21.15.12:0/3218696960 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa2d8009d90 con 0x7fa2e81000d0 2024-02-18T11:38:09.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.630+0000 7fa2e4d99640 1 -- 172.21.15.12:0/3218696960 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d800f850 con 0x7fa2e81000d0 2024-02-18T11:38:09.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.630+0000 7fa2ed088640 1 -- 172.21.15.12:0/3218696960 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2e819f890 con 0x7fa2e81000d0 2024-02-18T11:38:09.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.631+0000 7fa2ed088640 1 -- 172.21.15.12:0/3218696960 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2e810bae0 con 0x7fa2e81000d0 2024-02-18T11:38:09.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.632+0000 7fa2e4d99640 1 -- 172.21.15.12:0/3218696960 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) v1 ==== 137245+0+0 (secure 0 0 0) 0x7fa2d800f9b0 con 0x7fa2e81000d0 2024-02-18T11:38:09.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.633+0000 7fa2e4d99640 1 --2- 172.21.15.12:0/3218696960 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fa2d40a2750 0x7fa2d40a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:09.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.633+0000 7fa2de59c640 1 --2- 172.21.15.12:0/3218696960 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fa2d40a2750 0x7fa2d40a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:09.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.633+0000 7fa2e4d99640 1 -- 172.21.15.12:0/3218696960 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa2d80cde20 con 0x7fa2e81000d0 2024-02-18T11:38:09.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.633+0000 7fa2de59c640 1 --2- 172.21.15.12:0/3218696960 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fa2d40a2750 0x7fa2d40a4c10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa2e8198740 tx=0x7fa2d000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:09.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.637+0000 7fa2e4d99640 1 -- 172.21.15.12:0/3218696960 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa2d8096fb0 con 0x7fa2e81000d0 2024-02-18T11:38:09.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.818+0000 7fa2ed088640 1 -- 172.21.15.12:0/3218696960 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa2e819fd80 con 0x7fa2e81000d0 2024-02-18T11:38:09.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.819+0000 7fa2e4d99640 1 -- 172.21.15.12:0/3218696960 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa2d809a670 con 0x7fa2e81000d0 2024-02-18T11:38:09.813 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-02-18T11:38:09.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.820+0000 7fa2ed088640 1 -- 172.21.15.12:0/3218696960 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fa2d40a2750 msgr2=0x7fa2d40a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:09.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.820+0000 7fa2ed088640 1 --2- 172.21.15.12:0/3218696960 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fa2d40a2750 0x7fa2d40a4c10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa2e8198740 tx=0x7fa2d000b390 comp rx=0 tx=0).stop 2024-02-18T11:38:09.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.820+0000 7fa2ed088640 1 -- 172.21.15.12:0/3218696960 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2e81000d0 msgr2=0x7fa2e8197230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:09.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.820+0000 7fa2ed088640 1 --2- 172.21.15.12:0/3218696960 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2e81000d0 0x7fa2e8197230 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa2d800d950 tx=0x7fa2d800de10 comp rx=0 tx=0).stop 2024-02-18T11:38:09.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.820+0000 7fa2ed088640 1 -- 172.21.15.12:0/3218696960 shutdown_connections 2024-02-18T11:38:09.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.820+0000 7fa2ed088640 1 --2- 172.21.15.12:0/3218696960 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fa2d40a2750 0x7fa2d40a4c10 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:09.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.820+0000 7fa2ed088640 1 --2- 172.21.15.12:0/3218696960 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa2e8101c80 0x7fa2e819e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:09.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.820+0000 7fa2ed088640 1 --2- 172.21.15.12:0/3218696960 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa2e81012c0 0x7fa2e8197770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:09.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.820+0000 7fa2ed088640 1 --2- 172.21.15.12:0/3218696960 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2e81000d0 0x7fa2e8197230 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:09.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.821+0000 7fa2ed088640 1 -- 172.21.15.12:0/3218696960 >> 172.21.15.12:0/3218696960 conn(0x7fa2e80fb660 msgr2=0x7fa2e80fd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:09.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.821+0000 7fa2ed088640 1 -- 172.21.15.12:0/3218696960 shutdown_connections 2024-02-18T11:38:09.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.821+0000 7fa2ed088640 1 -- 172.21.15.12:0/3218696960 wait complete. 2024-02-18T11:38:09.825 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:09 smithi012 ceph-mon[32460]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi012=a;smithi012=d;smithi012=g;smithi012=j;smithi158=b;smithi158=e;smithi158=h;smithi158=k;smithi188=c;smithi188=f;smithi188=i;smithi188=l", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:38:09.825 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:09 smithi012 ceph-mon[32460]: Saving service mds.all spec with placement smithi012=a;smithi012=d;smithi012=g;smithi012=j;smithi158=b;smithi158=e;smithi158=h;smithi158=k;smithi188=c;smithi188=f;smithi188=i;smithi188=l;count:12 2024-02-18T11:38:09.826 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:09 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:38:09.826 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:09 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:38:09.826 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:09 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:09.826 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:09 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-18T11:38:09.826 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:09 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-18T11:38:09.826 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:09 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:38:09.826 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:09 smithi012 ceph-mon[32460]: Deploying daemon mds.l on smithi188 2024-02-18T11:38:09.826 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:09 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/3424588739' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-18T11:38:09.827 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-18T11:38:09.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.936+0000 7f098cd0c640 1 -- 172.21.15.12:0/1184067004 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09881000e0 msgr2=0x7f0988100500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:09.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.936+0000 7f098cd0c640 1 --2- 172.21.15.12:0/1184067004 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09881000e0 0x7f0988100500 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0978009900 tx=0x7f097802bc30 comp rx=0 tx=0).stop 2024-02-18T11:38:09.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.936+0000 7f098cd0c640 1 -- 172.21.15.12:0/1184067004 shutdown_connections 2024-02-18T11:38:09.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.936+0000 7f098cd0c640 1 --2- 172.21.15.12:0/1184067004 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09881012d0 0x7f0988101750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:09.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.936+0000 7f098cd0c640 1 --2- 172.21.15.12:0/1184067004 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0988101c90 0x7f0988108520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:09.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.936+0000 7f098cd0c640 1 --2- 172.21.15.12:0/1184067004 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09881000e0 0x7f0988100500 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:09.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.936+0000 7f098cd0c640 1 -- 172.21.15.12:0/1184067004 >> 172.21.15.12:0/1184067004 conn(0x7f09880fb650 msgr2=0x7f09880fda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:09.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.936+0000 7f098cd0c640 1 -- 172.21.15.12:0/1184067004 shutdown_connections 2024-02-18T11:38:09.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.936+0000 7f098cd0c640 1 -- 172.21.15.12:0/1184067004 wait complete. 2024-02-18T11:38:09.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.937+0000 7f098cd0c640 1 Processor -- start 2024-02-18T11:38:09.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.937+0000 7f098cd0c640 1 -- start start 2024-02-18T11:38:09.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.937+0000 7f098cd0c640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f09881000e0 0x7f09881972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:09.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.937+0000 7f098cd0c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09881012d0 0x7f09881977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:09.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.937+0000 7f098cd0c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0988101c90 0x7f098819e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:09.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.937+0000 7f098cd0c640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f098819ed90 con 0x7f0988101c90 2024-02-18T11:38:09.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.937+0000 7f098cd0c640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f098819ef00 con 0x7f09881000e0 2024-02-18T11:38:09.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.937+0000 7f098cd0c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f098819f0c0 con 0x7f09881012d0 2024-02-18T11:38:09.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.937+0000 7f0986d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0988101c90 0x7f098819e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:09.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.937+0000 7f0986d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0988101c90 0x7f098819e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51620/0 (socket says 172.21.15.12:51620) 2024-02-18T11:38:09.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.937+0000 7f0986d9d640 1 -- 172.21.15.12:0/3846383154 learned_addr learned my addr 172.21.15.12:0/3846383154 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:38:09.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.937+0000 7f0985d9b640 1 --2- 172.21.15.12:0/3846383154 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09881012d0 0x7f09881977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:09.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.937+0000 7f098659c640 1 --2- 172.21.15.12:0/3846383154 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f09881000e0 0x7f09881972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:09.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.937+0000 7f0986d9d640 1 -- 172.21.15.12:0/3846383154 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09881012d0 msgr2=0x7f09881977e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:09.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.937+0000 7f0986d9d640 1 --2- 172.21.15.12:0/3846383154 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09881012d0 0x7f09881977e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:09.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.937+0000 7f0986d9d640 1 -- 172.21.15.12:0/3846383154 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f09881000e0 msgr2=0x7f09881972a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:09.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.937+0000 7f0986d9d640 1 --2- 172.21.15.12:0/3846383154 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f09881000e0 0x7f09881972a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:09.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.937+0000 7f0986d9d640 1 -- 172.21.15.12:0/3846383154 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0978009560 con 0x7f0988101c90 2024-02-18T11:38:09.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.937+0000 7f0986d9d640 1 --2- 172.21.15.12:0/3846383154 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0988101c90 0x7f098819e850 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f098000eaa0 tx=0x7f098000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:09.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.938+0000 7f09777fe640 1 -- 172.21.15.12:0/3846383154 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f098000cdb0 con 0x7f0988101c90 2024-02-18T11:38:09.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.938+0000 7f09777fe640 1 -- 172.21.15.12:0/3846383154 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0980005020 con 0x7f0988101c90 2024-02-18T11:38:09.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.938+0000 7f098cd0c640 1 -- 172.21.15.12:0/3846383154 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f098819f3a0 con 0x7f0988101c90 2024-02-18T11:38:09.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.938+0000 7f09777fe640 1 -- 172.21.15.12:0/3846383154 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09800116b0 con 0x7f0988101c90 2024-02-18T11:38:09.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.938+0000 7f098cd0c640 1 -- 172.21.15.12:0/3846383154 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f098819f860 con 0x7f0988101c90 2024-02-18T11:38:09.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.939+0000 7f098cd0c640 1 -- 172.21.15.12:0/3846383154 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f098810baf0 con 0x7f0988101c90 2024-02-18T11:38:09.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.940+0000 7f09777fe640 1 -- 172.21.15.12:0/3846383154 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) v1 ==== 137245+0+0 (secure 0 0 0) 0x7f0980005520 con 0x7f0988101c90 2024-02-18T11:38:09.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.940+0000 7f09777fe640 1 --2- 172.21.15.12:0/3846383154 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f09700a2750 0x7f09700a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:09.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.940+0000 7f09777fe640 1 -- 172.21.15.12:0/3846383154 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f09800ce670 con 0x7f0988101c90 2024-02-18T11:38:09.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.940+0000 7f098659c640 1 --2- 172.21.15.12:0/3846383154 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f09700a2750 0x7f09700a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:09.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.941+0000 7f098659c640 1 --2- 172.21.15.12:0/3846383154 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f09700a2750 0x7f09700a4c10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0978005ab0 tx=0x7f0978005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:09.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:09.944+0000 7f09777fe640 1 -- 172.21.15.12:0/3846383154 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0980097950 con 0x7f0988101c90 2024-02-18T11:38:10.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:09 smithi188 ceph-mon[36925]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi012=a;smithi012=d;smithi012=g;smithi012=j;smithi158=b;smithi158=e;smithi158=h;smithi158=k;smithi188=c;smithi188=f;smithi188=i;smithi188=l", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:38:10.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:09 smithi188 ceph-mon[36925]: Saving service mds.all spec with placement smithi012=a;smithi012=d;smithi012=g;smithi012=j;smithi158=b;smithi158=e;smithi158=h;smithi158=k;smithi188=c;smithi188=f;smithi188=i;smithi188=l;count:12 2024-02-18T11:38:10.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:09 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:38:10.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:09 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:38:10.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:09 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:10.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:09 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-18T11:38:10.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:09 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-18T11:38:10.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:09 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:38:10.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:09 smithi188 ceph-mon[36925]: Deploying daemon mds.l on smithi188 2024-02-18T11:38:10.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:09 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/3424588739' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-18T11:38:10.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:10.162+0000 7f098cd0c640 1 -- 172.21.15.12:0/3846383154 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f098810bc80 con 0x7f0988101c90 2024-02-18T11:38:10.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:10.162+0000 7f09777fe640 1 -- 172.21.15.12:0/3846383154 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+535 (secure 0 0 0) 0x7f098009b010 con 0x7f0988101c90 2024-02-18T11:38:10.157 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:38:10.157 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-02-18T11:38:10.157 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 1 2024-02-18T11:38:10.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:10.164+0000 7f098cd0c640 1 -- 172.21.15.12:0/3846383154 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f09700a2750 msgr2=0x7f09700a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:10.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:10.164+0000 7f098cd0c640 1 --2- 172.21.15.12:0/3846383154 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f09700a2750 0x7f09700a4c10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0978005ab0 tx=0x7f0978005fd0 comp rx=0 tx=0).stop 2024-02-18T11:38:10.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:10.164+0000 7f098cd0c640 1 -- 172.21.15.12:0/3846383154 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0988101c90 msgr2=0x7f098819e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:10.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:10.164+0000 7f098cd0c640 1 --2- 172.21.15.12:0/3846383154 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0988101c90 0x7f098819e850 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f098000eaa0 tx=0x7f098000ef60 comp rx=0 tx=0).stop 2024-02-18T11:38:10.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:10.164+0000 7f098cd0c640 1 -- 172.21.15.12:0/3846383154 shutdown_connections 2024-02-18T11:38:10.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:10.164+0000 7f098cd0c640 1 --2- 172.21.15.12:0/3846383154 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f09700a2750 0x7f09700a4c10 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:10.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:10.164+0000 7f098cd0c640 1 --2- 172.21.15.12:0/3846383154 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09881012d0 0x7f09881977e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:10.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:10.164+0000 7f098cd0c640 1 --2- 172.21.15.12:0/3846383154 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0988101c90 0x7f098819e850 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:10.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:10.164+0000 7f098cd0c640 1 --2- 172.21.15.12:0/3846383154 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f09881000e0 0x7f09881972a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:10.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:10.164+0000 7f098cd0c640 1 -- 172.21.15.12:0/3846383154 >> 172.21.15.12:0/3846383154 conn(0x7f09880fb650 msgr2=0x7f09880fd140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:10.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:10.165+0000 7f098cd0c640 1 -- 172.21.15.12:0/3846383154 shutdown_connections 2024-02-18T11:38:10.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:10.165+0000 7f098cd0c640 1 -- 172.21.15.12:0/3846383154 wait complete. 2024-02-18T11:38:10.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:09 smithi158 ceph-mon[36626]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi012=a;smithi012=d;smithi012=g;smithi012=j;smithi158=b;smithi158=e;smithi158=h;smithi158=k;smithi188=c;smithi188=f;smithi188=i;smithi188=l", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:38:10.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:09 smithi158 ceph-mon[36626]: Saving service mds.all spec with placement smithi012=a;smithi012=d;smithi012=g;smithi012=j;smithi158=b;smithi158=e;smithi158=h;smithi158=k;smithi188=c;smithi188=f;smithi188=i;smithi188=l;count:12 2024-02-18T11:38:10.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:09 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:38:10.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:09 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:38:10.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:09 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:10.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:09 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-18T11:38:10.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:09 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-18T11:38:10.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:09 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:38:10.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:09 smithi158 ceph-mon[36626]: Deploying daemon mds.l on smithi188 2024-02-18T11:38:10.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:09 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/3424588739' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-18T11:38:11.072 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:10 smithi188 ceph-mon[36925]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-18T11:38:11.073 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:10 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/3218696960' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-18T11:38:11.073 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:10 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/3846383154' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-18T11:38:11.073 INFO:journalctl@ceph.mds.l.smithi188.stdout:Feb 18 11:38:10 smithi188 systemd[1]: Starting Ceph mds.l for 0c8bb0de-ce51-11ee-95ba-87774f69a715... 2024-02-18T11:38:11.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:10 smithi012 ceph-mon[32460]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-18T11:38:11.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:10 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/3218696960' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-18T11:38:11.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:10 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/3846383154' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-18T11:38:11.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:10 smithi158 ceph-mon[36626]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-18T11:38:11.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:10 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/3218696960' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-18T11:38:11.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:10 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/3846383154' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-18T11:38:11.325 INFO:journalctl@ceph.mds.l.smithi188.stdout:Feb 18 11:38:11 smithi188 podman[66972]: 2024-02-18 11:38:11.078306109 +0000 UTC m=+0.104991558 container create c3ce165c7789c08dc5b5574447e32f69e2f2e65fe24760a6ba65a38bbbe2b7b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-l, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-02-18T11:38:11.325 INFO:journalctl@ceph.mds.l.smithi188.stdout:Feb 18 11:38:11 smithi188 podman[66972]: 2024-02-18 11:38:10.991147717 +0000 UTC m=+0.017833122 image pull 53c33ee3bf62dc411b650f86eedbbee7698efb1001b4c063271dcd8d65b41165 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1 2024-02-18T11:38:11.325 INFO:journalctl@ceph.mds.l.smithi188.stdout:Feb 18 11:38:11 smithi188 podman[66972]: 2024-02-18 11:38:11.305994659 +0000 UTC m=+0.332680068 container init c3ce165c7789c08dc5b5574447e32f69e2f2e65fe24760a6ba65a38bbbe2b7b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-l, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-02-18T11:38:11.325 INFO:journalctl@ceph.mds.l.smithi188.stdout:Feb 18 11:38:11 smithi188 podman[66972]: 2024-02-18 11:38:11.30873306 +0000 UTC m=+0.335418458 container start c3ce165c7789c08dc5b5574447e32f69e2f2e65fe24760a6ba65a38bbbe2b7b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-l, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-02-18T11:38:11.325 INFO:journalctl@ceph.mds.l.smithi188.stdout:Feb 18 11:38:11 smithi188 ceph-mds[66991]: -- 172.21.15.188:0/3489419781 <== mon.1 v2:172.21.15.188:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562faa3721a0 con 0x562faa348800 2024-02-18T11:38:11.604 INFO:journalctl@ceph.mds.l.smithi188.stdout:Feb 18 11:38:11 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-l[66987]: starting mds.l at 2024-02-18T11:38:11.604 INFO:journalctl@ceph.mds.l.smithi188.stdout:Feb 18 11:38:11 smithi188 bash[66972]: c3ce165c7789c08dc5b5574447e32f69e2f2e65fe24760a6ba65a38bbbe2b7b2 2024-02-18T11:38:11.604 INFO:journalctl@ceph.mds.l.smithi188.stdout:Feb 18 11:38:11 smithi188 systemd[1]: Started Ceph mds.l for 0c8bb0de-ce51-11ee-95ba-87774f69a715. 2024-02-18T11:38:12.171 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-18T11:38:12.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:12.292+0000 7f8c43815640 1 -- 172.21.15.12:0/1785284531 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c3c0ffa50 msgr2=0x7f8c3c101e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:12.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:12.292+0000 7f8c43815640 1 --2- 172.21.15.12:0/1785284531 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c3c0ffa50 0x7f8c3c101e80 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8c34009900 tx=0x7f8c3402bc30 comp rx=0 tx=0).stop 2024-02-18T11:38:12.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:12.292+0000 7f8c43815640 1 -- 172.21.15.12:0/1785284531 shutdown_connections 2024-02-18T11:38:12.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:12.292+0000 7f8c43815640 1 --2- 172.21.15.12:0/1785284531 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c3c104d70 0x7f8c3c1051f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:12.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:12.292+0000 7f8c43815640 1 --2- 172.21.15.12:0/1785284531 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8c3c1023c0 0x7f8c3c104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:12.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:12.292+0000 7f8c43815640 1 --2- 172.21.15.12:0/1785284531 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c3c0ffa50 0x7f8c3c101e80 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:12.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:12.292+0000 7f8c43815640 1 -- 172.21.15.12:0/1785284531 >> 172.21.15.12:0/1785284531 conn(0x7f8c3c0fb650 msgr2=0x7f8c3c0fda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:12.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:12.292+0000 7f8c43815640 1 -- 172.21.15.12:0/1785284531 shutdown_connections 2024-02-18T11:38:12.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:12.292+0000 7f8c43815640 1 -- 172.21.15.12:0/1785284531 wait complete. 2024-02-18T11:38:12.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:12.292+0000 7f8c43815640 1 Processor -- start 2024-02-18T11:38:12.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:12.292+0000 7f8c43815640 1 -- start start 2024-02-18T11:38:12.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:12.293+0000 7f8c43815640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c3c0ffa50 0x7f8c3c19d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:12.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:12.293+0000 7f8c43815640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c3c1023c0 0x7f8c3c19dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:12.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:12.293+0000 7f8c43815640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8c3c104d70 0x7f8c3c1a4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:12.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:12.293+0000 7f8c43815640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3c1a5350 con 0x7f8c3c1023c0 2024-02-18T11:38:12.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:12.293+0000 7f8c43815640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3c1a54c0 con 0x7f8c3c104d70 2024-02-18T11:38:12.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:12.293+0000 7f8c43815640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3c1a5680 con 0x7f8c3c0ffa50 2024-02-18T11:38:12.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:12.293+0000 7f8c40db0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c3c1023c0 0x7f8c3c19dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:12.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:12.293+0000 7f8c40db0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c3c1023c0 0x7f8c3c19dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46834/0 (socket says 172.21.15.12:46834) 2024-02-18T11:38:12.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:12.293+0000 7f8c40db0640 1 -- 172.21.15.12:0/1551410600 learned_addr learned my addr 172.21.15.12:0/1551410600 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:38:12.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:12.293+0000 7f8c41db2640 1 --2- 172.21.15.12:0/1551410600 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8c3c104d70 0x7f8c3c1a4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:12.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:12.293+0000 7f8c415b1640 1 --2- 172.21.15.12:0/1551410600 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c3c0ffa50 0x7f8c3c19d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:12.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:12.293+0000 7f8c40db0640 1 -- 172.21.15.12:0/1551410600 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c3c0ffa50 msgr2=0x7f8c3c19d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:12.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:12.293+0000 7f8c40db0640 1 --2- 172.21.15.12:0/1551410600 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c3c0ffa50 0x7f8c3c19d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:12.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:12.293+0000 7f8c40db0640 1 -- 172.21.15.12:0/1551410600 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8c3c104d70 msgr2=0x7f8c3c1a4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:12.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:12.293+0000 7f8c40db0640 1 --2- 172.21.15.12:0/1551410600 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8c3c104d70 0x7f8c3c1a4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:12.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:12.293+0000 7f8c40db0640 1 -- 172.21.15.12:0/1551410600 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c34009560 con 0x7f8c3c1023c0 2024-02-18T11:38:12.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:12.293+0000 7f8c415b1640 1 --2- 172.21.15.12:0/1551410600 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c3c0ffa50 0x7f8c3c19d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:38:12.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:12.293+0000 7f8c40db0640 1 --2- 172.21.15.12:0/1551410600 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c3c1023c0 0x7f8c3c19dda0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c00d950 tx=0x7f8c2c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:12.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:12.293+0000 7f8c327fc640 1 -- 172.21.15.12:0/1551410600 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c2c009c30 con 0x7f8c3c1023c0 2024-02-18T11:38:12.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:12.293+0000 7f8c327fc640 1 -- 172.21.15.12:0/1551410600 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8c2c009d90 con 0x7f8c3c1023c0 2024-02-18T11:38:12.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:12.293+0000 7f8c43815640 1 -- 172.21.15.12:0/1551410600 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c3c1a5960 con 0x7f8c3c1023c0 2024-02-18T11:38:12.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:12.293+0000 7f8c327fc640 1 -- 172.21.15.12:0/1551410600 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c2c00f7e0 con 0x7f8c3c1023c0 2024-02-18T11:38:12.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:12.294+0000 7f8c43815640 1 -- 172.21.15.12:0/1551410600 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c3c1a5e70 con 0x7f8c3c1023c0 2024-02-18T11:38:12.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:12.295+0000 7f8c43815640 1 -- 172.21.15.12:0/1551410600 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c3c0fd140 con 0x7f8c3c1023c0 2024-02-18T11:38:12.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:12.296+0000 7f8c327fc640 1 -- 172.21.15.12:0/1551410600 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) v1 ==== 137245+0+0 (secure 0 0 0) 0x7f8c2c010ad0 con 0x7f8c3c1023c0 2024-02-18T11:38:12.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:12.296+0000 7f8c327fc640 1 --2- 172.21.15.12:0/1551410600 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f8c280a2750 0x7f8c280a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:12.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:12.296+0000 7f8c415b1640 1 --2- 172.21.15.12:0/1551410600 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f8c280a2750 0x7f8c280a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:12.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:12.296+0000 7f8c327fc640 1 -- 172.21.15.12:0/1551410600 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f8c2c0cdd30 con 0x7f8c3c1023c0 2024-02-18T11:38:12.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:12.297+0000 7f8c415b1640 1 --2- 172.21.15.12:0/1551410600 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f8c280a2750 0x7f8c280a4c10 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8c34006010 tx=0x7f8c3402c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:12.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:12.301+0000 7f8c327fc640 1 -- 172.21.15.12:0/1551410600 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8c2c096f60 con 0x7f8c3c1023c0 2024-02-18T11:38:12.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:12 smithi158 ceph-mon[36626]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-18T11:38:12.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:12 smithi158 ceph-mon[36626]: mds.? [v2:172.21.15.188:6832/1207202465,v1:172.21.15.188:6833/1207202465] up:boot 2024-02-18T11:38:12.495 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:12 smithi158 ceph-mon[36626]: fsmap 1 up:standby 2024-02-18T11:38:12.495 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:12 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-18T11:38:12.495 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:12 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:12.495 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:12 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:12.495 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:12 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:12.495 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:12 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-18T11:38:12.495 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:12 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-18T11:38:12.495 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:12 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:38:12.495 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:12 smithi158 ceph-mon[36626]: Deploying daemon mds.k on smithi158 2024-02-18T11:38:12.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:12.524+0000 7f8c43815640 1 -- 172.21.15.12:0/1551410600 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8c3c113210 con 0x7f8c3c1023c0 2024-02-18T11:38:12.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:12.524+0000 7f8c327fc640 1 -- 172.21.15.12:0/1551410600 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1328 (secure 0 0 0) 0x7f8c2c09a620 con 0x7f8c3c1023c0 2024-02-18T11:38:12.519 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:38:12.519 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1207202465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1207202465},{"type":"v1","addr":"172.21.15.188:6833","nonce":1207202465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-02-18T11:38:12.519 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 2 2024-02-18T11:38:12.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:12.526+0000 7f8c43815640 1 -- 172.21.15.12:0/1551410600 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f8c280a2750 msgr2=0x7f8c280a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:12.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:12.526+0000 7f8c43815640 1 --2- 172.21.15.12:0/1551410600 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f8c280a2750 0x7f8c280a4c10 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8c34006010 tx=0x7f8c3402c7e0 comp rx=0 tx=0).stop 2024-02-18T11:38:12.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:12.526+0000 7f8c43815640 1 -- 172.21.15.12:0/1551410600 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c3c1023c0 msgr2=0x7f8c3c19dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:12.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:12.526+0000 7f8c43815640 1 --2- 172.21.15.12:0/1551410600 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c3c1023c0 0x7f8c3c19dda0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c00d950 tx=0x7f8c2c00de10 comp rx=0 tx=0).stop 2024-02-18T11:38:12.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:12.526+0000 7f8c43815640 1 -- 172.21.15.12:0/1551410600 shutdown_connections 2024-02-18T11:38:12.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:12.526+0000 7f8c43815640 1 --2- 172.21.15.12:0/1551410600 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f8c280a2750 0x7f8c280a4c10 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:12.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:12.526+0000 7f8c43815640 1 --2- 172.21.15.12:0/1551410600 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8c3c104d70 0x7f8c3c1a4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:12.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:12.526+0000 7f8c43815640 1 --2- 172.21.15.12:0/1551410600 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c3c1023c0 0x7f8c3c19dda0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:12.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:12.526+0000 7f8c43815640 1 --2- 172.21.15.12:0/1551410600 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c3c0ffa50 0x7f8c3c19d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:12.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:12.526+0000 7f8c43815640 1 -- 172.21.15.12:0/1551410600 >> 172.21.15.12:0/1551410600 conn(0x7f8c3c0fb650 msgr2=0x7f8c3c102d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:12.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:12.526+0000 7f8c43815640 1 -- 172.21.15.12:0/1551410600 shutdown_connections 2024-02-18T11:38:12.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:12.526+0000 7f8c43815640 1 -- 172.21.15.12:0/1551410600 wait complete. 2024-02-18T11:38:12.531 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:12 smithi012 ceph-mon[32460]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-18T11:38:12.531 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:12 smithi012 ceph-mon[32460]: mds.? [v2:172.21.15.188:6832/1207202465,v1:172.21.15.188:6833/1207202465] up:boot 2024-02-18T11:38:12.531 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:12 smithi012 ceph-mon[32460]: fsmap 1 up:standby 2024-02-18T11:38:12.531 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:12 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-18T11:38:12.531 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:12 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:12.531 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:12 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:12.532 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:12 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:12.532 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:12 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-18T11:38:12.532 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:12 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-18T11:38:12.532 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:12 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:38:12.532 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:12 smithi012 ceph-mon[32460]: Deploying daemon mds.k on smithi158 2024-02-18T11:38:12.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:12 smithi188 ceph-mon[36925]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-18T11:38:12.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:12 smithi188 ceph-mon[36925]: mds.? [v2:172.21.15.188:6832/1207202465,v1:172.21.15.188:6833/1207202465] up:boot 2024-02-18T11:38:12.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:12 smithi188 ceph-mon[36925]: fsmap 1 up:standby 2024-02-18T11:38:12.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:12 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-18T11:38:12.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:12 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:12.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:12 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:12.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:12 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:12.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:12 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-18T11:38:12.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:12 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-18T11:38:12.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:12 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:38:12.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:12 smithi188 ceph-mon[36925]: Deploying daemon mds.k on smithi158 2024-02-18T11:38:13.386 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:13 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/1551410600' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-18T11:38:13.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:13 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/1551410600' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-18T11:38:13.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:13 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/1551410600' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-18T11:38:14.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:14 smithi158 ceph-mon[36626]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-18T11:38:14.534 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-18T11:38:14.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:14 smithi188 ceph-mon[36925]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-18T11:38:14.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:14.643+0000 7f36359c9640 1 -- 172.21.15.12:0/2640717798 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3630104630 msgr2=0x7f3630106a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:14.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:14.643+0000 7f36359c9640 1 --2- 172.21.15.12:0/2640717798 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3630104630 0x7f3630106a20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3620009900 tx=0x7f362002bc30 comp rx=0 tx=0).stop 2024-02-18T11:38:14.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:14.643+0000 7f36359c9640 1 -- 172.21.15.12:0/2640717798 shutdown_connections 2024-02-18T11:38:14.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:14.643+0000 7f36359c9640 1 --2- 172.21.15.12:0/2640717798 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3630106f60 0x7f3630109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:14.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:14.643+0000 7f36359c9640 1 --2- 172.21.15.12:0/2640717798 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3630104630 0x7f3630106a20 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:14.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:14.643+0000 7f36359c9640 1 --2- 172.21.15.12:0/2640717798 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3630101d00 0x7f36301040f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:14.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:14.643+0000 7f36359c9640 1 -- 172.21.15.12:0/2640717798 >> 172.21.15.12:0/2640717798 conn(0x7f36300fb650 msgr2=0x7f36300fda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:14.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:14.644+0000 7f36359c9640 1 -- 172.21.15.12:0/2640717798 shutdown_connections 2024-02-18T11:38:14.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:14.644+0000 7f36359c9640 1 -- 172.21.15.12:0/2640717798 wait complete. 2024-02-18T11:38:14.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:14.644+0000 7f36359c9640 1 Processor -- start 2024-02-18T11:38:14.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:14.644+0000 7f36359c9640 1 -- start start 2024-02-18T11:38:14.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:14.644+0000 7f36359c9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3630101d00 0x7f3630101670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:14.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:14.644+0000 7f36359c9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3630104630 0x7f36300ffcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:14.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:14.644+0000 7f36359c9640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3630106f60 0x7f36301001f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:14.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:14.644+0000 7f36359c9640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3630100950 con 0x7f3630101d00 2024-02-18T11:38:14.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:14.644+0000 7f36359c9640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3630100ac0 con 0x7f3630106f60 2024-02-18T11:38:14.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:14.644+0000 7f36359c9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36301a12f0 con 0x7f3630104630 2024-02-18T11:38:14.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:14.645+0000 7f362effd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3630101d00 0x7f3630101670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:14.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:14.645+0000 7f362effd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3630101d00 0x7f3630101670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46850/0 (socket says 172.21.15.12:46850) 2024-02-18T11:38:14.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:14.645+0000 7f362effd640 1 -- 172.21.15.12:0/59905020 learned_addr learned my addr 172.21.15.12:0/59905020 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:38:14.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:14.645+0000 7f362f7fe640 1 --2- 172.21.15.12:0/59905020 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3630106f60 0x7f36301001f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:14.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:14.645+0000 7f362e7fc640 1 --2- 172.21.15.12:0/59905020 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3630104630 0x7f36300ffcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:14.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:14.645+0000 7f362effd640 1 -- 172.21.15.12:0/59905020 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3630104630 msgr2=0x7f36300ffcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:14.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:14.645+0000 7f362effd640 1 --2- 172.21.15.12:0/59905020 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3630104630 0x7f36300ffcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:14.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:14.645+0000 7f362effd640 1 -- 172.21.15.12:0/59905020 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3630106f60 msgr2=0x7f36301001f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:14.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:14.645+0000 7f362effd640 1 --2- 172.21.15.12:0/59905020 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3630106f60 0x7f36301001f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:14.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:14.645+0000 7f362effd640 1 -- 172.21.15.12:0/59905020 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3620009560 con 0x7f3630101d00 2024-02-18T11:38:14.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:14.645+0000 7f362f7fe640 1 --2- 172.21.15.12:0/59905020 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3630106f60 0x7f36301001f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-18T11:38:14.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:14.645+0000 7f362effd640 1 --2- 172.21.15.12:0/59905020 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3630101d00 0x7f3630101670 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f361800d980 tx=0x7f361800de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:14.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:14.645+0000 7f36349c7640 1 -- 172.21.15.12:0/59905020 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3618009a30 con 0x7f3630101d00 2024-02-18T11:38:14.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:14.645+0000 7f36359c9640 1 -- 172.21.15.12:0/59905020 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36301a15a0 con 0x7f3630101d00 2024-02-18T11:38:14.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:14.645+0000 7f36349c7640 1 -- 172.21.15.12:0/59905020 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3618009b90 con 0x7f3630101d00 2024-02-18T11:38:14.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:14.645+0000 7f36349c7640 1 -- 172.21.15.12:0/59905020 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3618004a00 con 0x7f3630101d00 2024-02-18T11:38:14.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:14.645+0000 7f36359c9640 1 -- 172.21.15.12:0/59905020 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36301a1ad0 con 0x7f3630101d00 2024-02-18T11:38:14.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:14.646+0000 7f36359c9640 1 -- 172.21.15.12:0/59905020 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36300fd140 con 0x7f3630101d00 2024-02-18T11:38:14.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:14.647+0000 7f36349c7640 1 -- 172.21.15.12:0/59905020 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) v1 ==== 137245+0+0 (secure 0 0 0) 0x7f3618009d00 con 0x7f3630101d00 2024-02-18T11:38:14.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:14.648+0000 7f36349c7640 1 --2- 172.21.15.12:0/59905020 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f361c0a2750 0x7f361c0a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:14.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:14.648+0000 7f362e7fc640 1 --2- 172.21.15.12:0/59905020 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f361c0a2750 0x7f361c0a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:14.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:14.648+0000 7f36349c7640 1 -- 172.21.15.12:0/59905020 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f36180cda70 con 0x7f3630101d00 2024-02-18T11:38:14.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:14.648+0000 7f362e7fc640 1 --2- 172.21.15.12:0/59905020 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f361c0a2750 0x7f361c0a4c10 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3620005ab0 tx=0x7f3620005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:14.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:14.652+0000 7f36349c7640 1 -- 172.21.15.12:0/59905020 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3618096ca0 con 0x7f3630101d00 2024-02-18T11:38:14.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:14 smithi012 ceph-mon[32460]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-18T11:38:14.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:14.879+0000 7f36359c9640 1 -- 172.21.15.12:0/59905020 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f363010cc10 con 0x7f3630101d00 2024-02-18T11:38:14.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:14.879+0000 7f36349c7640 1 -- 172.21.15.12:0/59905020 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2122 (secure 0 0 0) 0x7f36180ba030 con 0x7f3630101d00 2024-02-18T11:38:14.874 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:38:14.874 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1207202465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1207202465},{"type":"v1","addr":"172.21.15.188:6833","nonce":1207202465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6833/2636348323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6832","nonce":2636348323},{"type":"v1","addr":"172.21.15.158:6833","nonce":2636348323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-02-18T11:38:14.874 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 3 2024-02-18T11:38:14.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:14.881+0000 7f36359c9640 1 -- 172.21.15.12:0/59905020 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f361c0a2750 msgr2=0x7f361c0a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:14.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:14.881+0000 7f36359c9640 1 --2- 172.21.15.12:0/59905020 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f361c0a2750 0x7f361c0a4c10 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3620005ab0 tx=0x7f3620005fd0 comp rx=0 tx=0).stop 2024-02-18T11:38:14.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:14.881+0000 7f36359c9640 1 -- 172.21.15.12:0/59905020 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3630101d00 msgr2=0x7f3630101670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:14.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:14.881+0000 7f36359c9640 1 --2- 172.21.15.12:0/59905020 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3630101d00 0x7f3630101670 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f361800d980 tx=0x7f361800de40 comp rx=0 tx=0).stop 2024-02-18T11:38:14.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:14.881+0000 7f36359c9640 1 -- 172.21.15.12:0/59905020 shutdown_connections 2024-02-18T11:38:14.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:14.881+0000 7f36359c9640 1 --2- 172.21.15.12:0/59905020 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f361c0a2750 0x7f361c0a4c10 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:14.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:14.881+0000 7f36359c9640 1 --2- 172.21.15.12:0/59905020 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3630104630 0x7f36300ffcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:14.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:14.881+0000 7f36359c9640 1 --2- 172.21.15.12:0/59905020 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3630106f60 0x7f36301001f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:14.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:14.881+0000 7f36359c9640 1 --2- 172.21.15.12:0/59905020 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3630101d00 0x7f3630101670 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:14.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:14.881+0000 7f36359c9640 1 -- 172.21.15.12:0/59905020 >> 172.21.15.12:0/59905020 conn(0x7f36300fb650 msgr2=0x7f3630102660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:14.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:14.881+0000 7f36359c9640 1 -- 172.21.15.12:0/59905020 shutdown_connections 2024-02-18T11:38:14.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:14.881+0000 7f36359c9640 1 -- 172.21.15.12:0/59905020 wait complete. 2024-02-18T11:38:15.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:15 smithi188 ceph-mon[36925]: mds.? [v2:172.21.15.158:6832/2636348323,v1:172.21.15.158:6833/2636348323] up:boot 2024-02-18T11:38:15.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:15 smithi188 ceph-mon[36925]: fsmap 2 up:standby 2024-02-18T11:38:15.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:15 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-02-18T11:38:15.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:15 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:15.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:15 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:15.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:15 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:15.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:15 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-18T11:38:15.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:15 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-18T11:38:15.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:15 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:38:15.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:15 smithi188 ceph-mon[36925]: Deploying daemon mds.j on smithi012 2024-02-18T11:38:15.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:15 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/59905020' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-18T11:38:15.639 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:15 smithi158 ceph-mon[36626]: mds.? [v2:172.21.15.158:6832/2636348323,v1:172.21.15.158:6833/2636348323] up:boot 2024-02-18T11:38:15.639 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:15 smithi158 ceph-mon[36626]: fsmap 2 up:standby 2024-02-18T11:38:15.639 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:15 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-02-18T11:38:15.639 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:15 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:15.640 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:15 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:15.640 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:15 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:15.640 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:15 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-18T11:38:15.640 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:15 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-18T11:38:15.640 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:15 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:38:15.640 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:15 smithi158 ceph-mon[36626]: Deploying daemon mds.j on smithi012 2024-02-18T11:38:15.640 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:15 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/59905020' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-18T11:38:15.641 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:15 smithi012 ceph-mon[32460]: mds.? [v2:172.21.15.158:6832/2636348323,v1:172.21.15.158:6833/2636348323] up:boot 2024-02-18T11:38:15.642 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:15 smithi012 ceph-mon[32460]: fsmap 2 up:standby 2024-02-18T11:38:15.642 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:15 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-02-18T11:38:15.642 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:15 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:15.642 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:15 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:15.642 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:15 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:15.642 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:15 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-18T11:38:15.642 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:15 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-18T11:38:15.642 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:15 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:38:15.642 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:15 smithi012 ceph-mon[32460]: Deploying daemon mds.j on smithi012 2024-02-18T11:38:15.642 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:15 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/59905020' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-18T11:38:16.406 INFO:journalctl@ceph.mds.j.smithi012.stdout:Feb 18 11:38:16 smithi012 systemd[1]: Starting Ceph mds.j for 0c8bb0de-ce51-11ee-95ba-87774f69a715... 2024-02-18T11:38:16.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:16 smithi012 ceph-mon[32460]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-18T11:38:16.673 INFO:journalctl@ceph.mds.j.smithi012.stdout:Feb 18 11:38:16 smithi012 podman[71183]: 2024-02-18 11:38:16.60924269 +0000 UTC m=+0.096481638 container create b73432e950b98e782de80edc57cda36c7a2abf37e1ec54f05409689099cf5e45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-j, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-02-18T11:38:16.673 INFO:journalctl@ceph.mds.j.smithi012.stdout:Feb 18 11:38:16 smithi012 podman[71183]: 2024-02-18 11:38:16.530169703 +0000 UTC m=+0.017408651 image pull 53c33ee3bf62dc411b650f86eedbbee7698efb1001b4c063271dcd8d65b41165 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1 2024-02-18T11:38:16.744 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:16 smithi158 ceph-mon[36626]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-18T11:38:16.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:16 smithi188 ceph-mon[36925]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-18T11:38:16.888 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-18T11:38:16.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:17.004+0000 7f06ad1e2640 1 -- 172.21.15.12:0/1479562535 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f06a8106ec0 msgr2=0x7f06a8109310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:16.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:17.004+0000 7f06ad1e2640 1 --2- 172.21.15.12:0/1479562535 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f06a8106ec0 0x7f06a8109310 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f06a0009ca0 tx=0x7f06a002c0f0 comp rx=0 tx=0).stop 2024-02-18T11:38:16.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:17.004+0000 7f06ad1e2640 1 -- 172.21.15.12:0/1479562535 shutdown_connections 2024-02-18T11:38:16.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:17.004+0000 7f06ad1e2640 1 --2- 172.21.15.12:0/1479562535 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06a8104590 0x7f06a8106980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:16.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:17.004+0000 7f06ad1e2640 1 --2- 172.21.15.12:0/1479562535 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f06a8106ec0 0x7f06a8109310 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:16.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:17.004+0000 7f06ad1e2640 1 --2- 172.21.15.12:0/1479562535 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f06a8101c60 0x7f06a8104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:16.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:17.004+0000 7f06ad1e2640 1 -- 172.21.15.12:0/1479562535 >> 172.21.15.12:0/1479562535 conn(0x7f06a80fb650 msgr2=0x7f06a80fda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:16.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:17.004+0000 7f06ad1e2640 1 -- 172.21.15.12:0/1479562535 shutdown_connections 2024-02-18T11:38:16.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:17.004+0000 7f06ad1e2640 1 -- 172.21.15.12:0/1479562535 wait complete. 2024-02-18T11:38:16.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:17.005+0000 7f06ad1e2640 1 Processor -- start 2024-02-18T11:38:17.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:17.005+0000 7f06ad1e2640 1 -- start start 2024-02-18T11:38:17.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:17.005+0000 7f06ad1e2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f06a8101c60 0x7f06a8197210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:17.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:17.005+0000 7f06ad1e2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06a8104590 0x7f06a8197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:17.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:17.005+0000 7f06ad1e2640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f06a8106ec0 0x7f06a819e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:17.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:17.005+0000 7f06ad1e2640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a819ed00 con 0x7f06a8101c60 2024-02-18T11:38:17.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:17.005+0000 7f06ad1e2640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a819ee70 con 0x7f06a8106ec0 2024-02-18T11:38:17.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:17.005+0000 7f06ad1e2640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a819f030 con 0x7f06a8104590 2024-02-18T11:38:17.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:17.005+0000 7f06a6d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f06a8101c60 0x7f06a8197210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:17.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:17.005+0000 7f06a6d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f06a8101c60 0x7f06a8197210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46894/0 (socket says 172.21.15.12:46894) 2024-02-18T11:38:17.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:17.005+0000 7f06a6d9d640 1 -- 172.21.15.12:0/881159580 learned_addr learned my addr 172.21.15.12:0/881159580 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:38:17.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:17.005+0000 7f06a659c640 1 --2- 172.21.15.12:0/881159580 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06a8104590 0x7f06a8197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:17.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:17.005+0000 7f06a759e640 1 --2- 172.21.15.12:0/881159580 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f06a8106ec0 0x7f06a819e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:17.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:17.005+0000 7f06a6d9d640 1 -- 172.21.15.12:0/881159580 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06a8104590 msgr2=0x7f06a8197750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:17.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:17.005+0000 7f06a6d9d640 1 --2- 172.21.15.12:0/881159580 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06a8104590 0x7f06a8197750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:17.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:17.005+0000 7f06a6d9d640 1 -- 172.21.15.12:0/881159580 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f06a8106ec0 msgr2=0x7f06a819e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:17.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:17.005+0000 7f06a6d9d640 1 --2- 172.21.15.12:0/881159580 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f06a8106ec0 0x7f06a819e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:17.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:17.005+0000 7f06a6d9d640 1 -- 172.21.15.12:0/881159580 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06a0009900 con 0x7f06a8101c60 2024-02-18T11:38:17.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:17.005+0000 7f06a759e640 1 --2- 172.21.15.12:0/881159580 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f06a8106ec0 0x7f06a819e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:38:17.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:17.006+0000 7f06a6d9d640 1 --2- 172.21.15.12:0/881159580 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f06a8101c60 0x7f06a8197210 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f069800e9f0 tx=0x7f069800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:17.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:17.008+0000 7f068ffff640 1 -- 172.21.15.12:0/881159580 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0698009700 con 0x7f06a8101c60 2024-02-18T11:38:17.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:17.008+0000 7f06ad1e2640 1 -- 172.21.15.12:0/881159580 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06a819f310 con 0x7f06a8101c60 2024-02-18T11:38:17.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:17.008+0000 7f06ad1e2640 1 -- 172.21.15.12:0/881159580 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06a819f870 con 0x7f06a8101c60 2024-02-18T11:38:17.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:17.008+0000 7f068ffff640 1 -- 172.21.15.12:0/881159580 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f069800cd80 con 0x7f06a8101c60 2024-02-18T11:38:17.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:17.008+0000 7f068ffff640 1 -- 172.21.15.12:0/881159580 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0698018920 con 0x7f06a8101c60 2024-02-18T11:38:17.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:17.009+0000 7f06ad1e2640 1 -- 172.21.15.12:0/881159580 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06a80fd140 con 0x7f06a8101c60 2024-02-18T11:38:17.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:17.011+0000 7f068ffff640 1 -- 172.21.15.12:0/881159580 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) v1 ==== 137245+0+0 (secure 0 0 0) 0x7f0698018a80 con 0x7f06a8101c60 2024-02-18T11:38:17.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:17.011+0000 7f068ffff640 1 --2- 172.21.15.12:0/881159580 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f06900a2750 0x7f06900a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:17.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:17.011+0000 7f06a659c640 1 --2- 172.21.15.12:0/881159580 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f06900a2750 0x7f06900a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:17.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:17.011+0000 7f068ffff640 1 -- 172.21.15.12:0/881159580 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f0698014070 con 0x7f06a8101c60 2024-02-18T11:38:17.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:17.012+0000 7f06a659c640 1 --2- 172.21.15.12:0/881159580 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f06900a2750 0x7f06900a4c10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f06a8198720 tx=0x7f069c00b590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:17.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:17.017+0000 7f068ffff640 1 -- 172.21.15.12:0/881159580 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0698052d20 con 0x7f06a8101c60 2024-02-18T11:38:17.173 INFO:journalctl@ceph.mds.j.smithi012.stdout:Feb 18 11:38:16 smithi012 podman[71183]: 2024-02-18 11:38:16.854479381 +0000 UTC m=+0.341718325 container init b73432e950b98e782de80edc57cda36c7a2abf37e1ec54f05409689099cf5e45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-j, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-02-18T11:38:17.173 INFO:journalctl@ceph.mds.j.smithi012.stdout:Feb 18 11:38:16 smithi012 podman[71183]: 2024-02-18 11:38:16.857100724 +0000 UTC m=+0.344339670 container start b73432e950b98e782de80edc57cda36c7a2abf37e1ec54f05409689099cf5e45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-j, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-02-18T11:38:17.173 INFO:journalctl@ceph.mds.j.smithi012.stdout:Feb 18 11:38:16 smithi012 ceph-mds[71202]: -- 172.21.15.12:0/2327765280 <== mon.0 v2:172.21.15.12:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f4a3fa81a0 con 0x55f4a3f7f000 2024-02-18T11:38:17.173 INFO:journalctl@ceph.mds.j.smithi012.stdout:Feb 18 11:38:16 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-j[71198]: starting mds.j at 2024-02-18T11:38:17.173 INFO:journalctl@ceph.mds.j.smithi012.stdout:Feb 18 11:38:16 smithi012 bash[71183]: b73432e950b98e782de80edc57cda36c7a2abf37e1ec54f05409689099cf5e45 2024-02-18T11:38:17.173 INFO:journalctl@ceph.mds.j.smithi012.stdout:Feb 18 11:38:16 smithi012 systemd[1]: Started Ceph mds.j for 0c8bb0de-ce51-11ee-95ba-87774f69a715. 2024-02-18T11:38:17.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:17.239+0000 7f06ad1e2640 1 -- 172.21.15.12:0/881159580 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f06a819f4a0 con 0x7f06a8101c60 2024-02-18T11:38:17.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:17.240+0000 7f068ffff640 1 -- 172.21.15.12:0/881159580 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2122 (secure 0 0 0) 0x7f06a819f4a0 con 0x7f06a8101c60 2024-02-18T11:38:17.234 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:38:17.235 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1207202465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1207202465},{"type":"v1","addr":"172.21.15.188:6833","nonce":1207202465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6833/2636348323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6832","nonce":2636348323},{"type":"v1","addr":"172.21.15.158:6833","nonce":2636348323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-02-18T11:38:17.235 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 3 2024-02-18T11:38:17.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:17.242+0000 7f06ad1e2640 1 -- 172.21.15.12:0/881159580 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f06900a2750 msgr2=0x7f06900a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:17.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:17.242+0000 7f06ad1e2640 1 --2- 172.21.15.12:0/881159580 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f06900a2750 0x7f06900a4c10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f06a8198720 tx=0x7f069c00b590 comp rx=0 tx=0).stop 2024-02-18T11:38:17.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:17.242+0000 7f06ad1e2640 1 -- 172.21.15.12:0/881159580 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f06a8101c60 msgr2=0x7f06a8197210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:17.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:17.242+0000 7f06ad1e2640 1 --2- 172.21.15.12:0/881159580 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f06a8101c60 0x7f06a8197210 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f069800e9f0 tx=0x7f069800eeb0 comp rx=0 tx=0).stop 2024-02-18T11:38:17.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:17.242+0000 7f06ad1e2640 1 -- 172.21.15.12:0/881159580 shutdown_connections 2024-02-18T11:38:17.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:17.242+0000 7f06ad1e2640 1 --2- 172.21.15.12:0/881159580 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f06900a2750 0x7f06900a4c10 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:17.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:17.242+0000 7f06ad1e2640 1 --2- 172.21.15.12:0/881159580 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06a8104590 0x7f06a8197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:17.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:17.242+0000 7f06ad1e2640 1 --2- 172.21.15.12:0/881159580 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f06a8106ec0 0x7f06a819e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:17.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:17.242+0000 7f06ad1e2640 1 --2- 172.21.15.12:0/881159580 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f06a8101c60 0x7f06a8197210 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:17.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:17.242+0000 7f06ad1e2640 1 -- 172.21.15.12:0/881159580 >> 172.21.15.12:0/881159580 conn(0x7f06a80fb650 msgr2=0x7f06a81025a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:17.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:17.242+0000 7f06ad1e2640 1 -- 172.21.15.12:0/881159580 shutdown_connections 2024-02-18T11:38:17.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:17.242+0000 7f06ad1e2640 1 -- 172.21.15.12:0/881159580 wait complete. 2024-02-18T11:38:18.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:17 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:18.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:17 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:18.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:17 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:18.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:17 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-18T11:38:18.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:17 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-18T11:38:18.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:17 smithi012 ceph-mon[32460]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-18T11:38:18.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:17 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:38:18.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:17 smithi012 ceph-mon[32460]: Deploying daemon mds.d on smithi012 2024-02-18T11:38:18.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:17 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/881159580' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-18T11:38:18.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:17 smithi012 ceph-mon[32460]: mds.? [v2:172.21.15.12:6834/1788335302,v1:172.21.15.12:6835/1788335302] up:boot 2024-02-18T11:38:18.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:17 smithi012 ceph-mon[32460]: fsmap 3 up:standby 2024-02-18T11:38:18.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:17 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-02-18T11:38:18.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:17 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:18.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:17 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:18.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:17 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:18.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:17 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-18T11:38:18.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:17 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-18T11:38:18.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:17 smithi158 ceph-mon[36626]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-18T11:38:18.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:17 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:38:18.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:17 smithi158 ceph-mon[36626]: Deploying daemon mds.d on smithi012 2024-02-18T11:38:18.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:17 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/881159580' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-18T11:38:18.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:17 smithi158 ceph-mon[36626]: mds.? [v2:172.21.15.12:6834/1788335302,v1:172.21.15.12:6835/1788335302] up:boot 2024-02-18T11:38:18.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:17 smithi158 ceph-mon[36626]: fsmap 3 up:standby 2024-02-18T11:38:18.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:17 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-02-18T11:38:18.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:17 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:18.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:17 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:18.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:17 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:18.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:17 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-18T11:38:18.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:17 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-18T11:38:18.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:17 smithi188 ceph-mon[36925]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-18T11:38:18.355 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:17 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:38:18.355 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:17 smithi188 ceph-mon[36925]: Deploying daemon mds.d on smithi012 2024-02-18T11:38:18.355 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:17 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/881159580' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-18T11:38:18.355 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:17 smithi188 ceph-mon[36925]: mds.? [v2:172.21.15.12:6834/1788335302,v1:172.21.15.12:6835/1788335302] up:boot 2024-02-18T11:38:18.355 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:17 smithi188 ceph-mon[36925]: fsmap 3 up:standby 2024-02-18T11:38:18.355 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:17 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-02-18T11:38:19.173 INFO:journalctl@ceph.mds.d.smithi012.stdout:Feb 18 11:38:19 smithi012 systemd[1]: Starting Ceph mds.d for 0c8bb0de-ce51-11ee-95ba-87774f69a715... 2024-02-18T11:38:19.249 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-18T11:38:19.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:19.361+0000 7f57b8c9d640 1 -- 172.21.15.12:0/3901320659 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f57b40ff890 msgr2=0x7f57b40ffcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:19.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:19.361+0000 7f57b8c9d640 1 --2- 172.21.15.12:0/3901320659 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f57b40ff890 0x7f57b40ffcb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f57a4009900 tx=0x7f57a402bc30 comp rx=0 tx=0).stop 2024-02-18T11:38:19.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:19.361+0000 7f57b8c9d640 1 -- 172.21.15.12:0/3901320659 shutdown_connections 2024-02-18T11:38:19.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:19.361+0000 7f57b8c9d640 1 --2- 172.21.15.12:0/3901320659 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57b4100a80 0x7f57b4100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:19.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:19.361+0000 7f57b8c9d640 1 --2- 172.21.15.12:0/3901320659 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f57b4101440 0x7f57b4107cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:19.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:19.361+0000 7f57b8c9d640 1 --2- 172.21.15.12:0/3901320659 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f57b40ff890 0x7f57b40ffcb0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:19.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:19.361+0000 7f57b8c9d640 1 -- 172.21.15.12:0/3901320659 >> 172.21.15.12:0/3901320659 conn(0x7f57b40fae00 msgr2=0x7f57b40fd240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:19.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:19.361+0000 7f57b8c9d640 1 -- 172.21.15.12:0/3901320659 shutdown_connections 2024-02-18T11:38:19.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:19.361+0000 7f57b8c9d640 1 -- 172.21.15.12:0/3901320659 wait complete. 2024-02-18T11:38:19.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:19.362+0000 7f57b8c9d640 1 Processor -- start 2024-02-18T11:38:19.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:19.362+0000 7f57b8c9d640 1 -- start start 2024-02-18T11:38:19.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:19.362+0000 7f57b8c9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f57b40ff890 0x7f57b41972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:19.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:19.362+0000 7f57b8c9d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57b4100a80 0x7f57b4197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:19.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:19.362+0000 7f57b8c9d640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f57b4101440 0x7f57b419e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:19.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:19.362+0000 7f57b8c9d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b419ede0 con 0x7f57b40ff890 2024-02-18T11:38:19.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:19.362+0000 7f57b8c9d640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b419ef50 con 0x7f57b4101440 2024-02-18T11:38:19.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:19.362+0000 7f57b8c9d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b419f110 con 0x7f57b4100a80 2024-02-18T11:38:19.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:19.362+0000 7f57b259c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f57b40ff890 0x7f57b41972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:19.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:19.362+0000 7f57b259c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f57b40ff890 0x7f57b41972f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46908/0 (socket says 172.21.15.12:46908) 2024-02-18T11:38:19.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:19.362+0000 7f57b259c640 1 -- 172.21.15.12:0/2664161990 learned_addr learned my addr 172.21.15.12:0/2664161990 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:38:19.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:19.362+0000 7f57b2d9d640 1 --2- 172.21.15.12:0/2664161990 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f57b4101440 0x7f57b419e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:19.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:19.363+0000 7f57abfff640 1 --2- 172.21.15.12:0/2664161990 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57b4100a80 0x7f57b4197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:19.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:19.363+0000 7f57b259c640 1 -- 172.21.15.12:0/2664161990 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57b4100a80 msgr2=0x7f57b4197830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:19.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:19.363+0000 7f57b259c640 1 --2- 172.21.15.12:0/2664161990 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57b4100a80 0x7f57b4197830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:19.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:19.363+0000 7f57b259c640 1 -- 172.21.15.12:0/2664161990 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f57b4101440 msgr2=0x7f57b419e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:19.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:19.363+0000 7f57b259c640 1 --2- 172.21.15.12:0/2664161990 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f57b4101440 0x7f57b419e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:19.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:19.363+0000 7f57b259c640 1 -- 172.21.15.12:0/2664161990 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a4009560 con 0x7f57b40ff890 2024-02-18T11:38:19.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:19.363+0000 7f57b2d9d640 1 --2- 172.21.15.12:0/2664161990 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f57b4101440 0x7f57b419e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:38:19.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:19.363+0000 7f57abfff640 1 --2- 172.21.15.12:0/2664161990 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57b4100a80 0x7f57b4197830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:38:19.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:19.363+0000 7f57b259c640 1 --2- 172.21.15.12:0/2664161990 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f57b40ff890 0x7f57b41972f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f57a4000c00 tx=0x7f57a4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:19.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:19.363+0000 7f57ab7fe640 1 -- 172.21.15.12:0/2664161990 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a4004df0 con 0x7f57b40ff890 2024-02-18T11:38:19.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:19.363+0000 7f57ab7fe640 1 -- 172.21.15.12:0/2664161990 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f57a4005430 con 0x7f57b40ff890 2024-02-18T11:38:19.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:19.363+0000 7f57ab7fe640 1 -- 172.21.15.12:0/2664161990 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a403e450 con 0x7f57b40ff890 2024-02-18T11:38:19.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:19.363+0000 7f57b8c9d640 1 -- 172.21.15.12:0/2664161990 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57b419f390 con 0x7f57b40ff890 2024-02-18T11:38:19.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:19.363+0000 7f57b8c9d640 1 -- 172.21.15.12:0/2664161990 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57b419f8a0 con 0x7f57b40ff890 2024-02-18T11:38:19.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:19.364+0000 7f57b8c9d640 1 -- 172.21.15.12:0/2664161990 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57b410bb40 con 0x7f57b40ff890 2024-02-18T11:38:19.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:19.365+0000 7f57ab7fe640 1 -- 172.21.15.12:0/2664161990 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) v1 ==== 137245+0+0 (secure 0 0 0) 0x7f57a40055a0 con 0x7f57b40ff890 2024-02-18T11:38:19.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:19.366+0000 7f57ab7fe640 1 --2- 172.21.15.12:0/2664161990 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f57a00a2750 0x7f57a00a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:19.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:19.366+0000 7f57abfff640 1 --2- 172.21.15.12:0/2664161990 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f57a00a2750 0x7f57a00a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:19.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:19.366+0000 7f57ab7fe640 1 -- 172.21.15.12:0/2664161990 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f57a40f0740 con 0x7f57b40ff890 2024-02-18T11:38:19.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:19.366+0000 7f57abfff640 1 --2- 172.21.15.12:0/2664161990 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f57a00a2750 0x7f57a00a4c10 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f57b4198800 tx=0x7f579c005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:19.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:19.370+0000 7f57ab7fe640 1 -- 172.21.15.12:0/2664161990 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f57a40b98d0 con 0x7f57b40ff890 2024-02-18T11:38:19.577 INFO:journalctl@ceph.mds.d.smithi012.stdout:Feb 18 11:38:19 smithi012 podman[71535]: 2024-02-18 11:38:19.217164506 +0000 UTC m=+0.018464621 image pull 53c33ee3bf62dc411b650f86eedbbee7698efb1001b4c063271dcd8d65b41165 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1 2024-02-18T11:38:19.577 INFO:journalctl@ceph.mds.d.smithi012.stdout:Feb 18 11:38:19 smithi012 podman[71535]: 2024-02-18 11:38:19.347500792 +0000 UTC m=+0.148800894 container create bb1d1e660e10b85641096fd76db7438805f424e2e9f306662311ee2655839cee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-d, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-02-18T11:38:19.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:19.589+0000 7f57b8c9d640 1 -- 172.21.15.12:0/2664161990 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f57b410bd20 con 0x7f57b40ff890 2024-02-18T11:38:19.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:19.589+0000 7f57ab7fe640 1 -- 172.21.15.12:0/2664161990 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2913 (secure 0 0 0) 0x7f57a40bcf90 con 0x7f57b40ff890 2024-02-18T11:38:19.584 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:38:19.584 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1788335302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1788335302},{"type":"v1","addr":"172.21.15.12:6835","nonce":1788335302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1207202465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1207202465},{"type":"v1","addr":"172.21.15.188:6833","nonce":1207202465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6833/2636348323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6832","nonce":2636348323},{"type":"v1","addr":"172.21.15.158:6833","nonce":2636348323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-02-18T11:38:19.584 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 4 2024-02-18T11:38:19.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:19.591+0000 7f57b8c9d640 1 -- 172.21.15.12:0/2664161990 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f57a00a2750 msgr2=0x7f57a00a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:19.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:19.591+0000 7f57b8c9d640 1 --2- 172.21.15.12:0/2664161990 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f57a00a2750 0x7f57a00a4c10 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f57b4198800 tx=0x7f579c005e50 comp rx=0 tx=0).stop 2024-02-18T11:38:19.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:19.591+0000 7f57b8c9d640 1 -- 172.21.15.12:0/2664161990 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f57b40ff890 msgr2=0x7f57b41972f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:19.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:19.591+0000 7f57b8c9d640 1 --2- 172.21.15.12:0/2664161990 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f57b40ff890 0x7f57b41972f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f57a4000c00 tx=0x7f57a4004270 comp rx=0 tx=0).stop 2024-02-18T11:38:19.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:19.591+0000 7f57b8c9d640 1 -- 172.21.15.12:0/2664161990 shutdown_connections 2024-02-18T11:38:19.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:19.591+0000 7f57b8c9d640 1 --2- 172.21.15.12:0/2664161990 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f57a00a2750 0x7f57a00a4c10 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:19.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:19.591+0000 7f57b8c9d640 1 --2- 172.21.15.12:0/2664161990 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57b4100a80 0x7f57b4197830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:19.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:19.591+0000 7f57b8c9d640 1 --2- 172.21.15.12:0/2664161990 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f57b4101440 0x7f57b419e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:19.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:19.591+0000 7f57b8c9d640 1 --2- 172.21.15.12:0/2664161990 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f57b40ff890 0x7f57b41972f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:19.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:19.591+0000 7f57b8c9d640 1 -- 172.21.15.12:0/2664161990 >> 172.21.15.12:0/2664161990 conn(0x7f57b40fae00 msgr2=0x7f57b41086c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:19.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:19.592+0000 7f57b8c9d640 1 -- 172.21.15.12:0/2664161990 shutdown_connections 2024-02-18T11:38:19.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:19.592+0000 7f57b8c9d640 1 -- 172.21.15.12:0/2664161990 wait complete. 2024-02-18T11:38:19.923 INFO:journalctl@ceph.mds.d.smithi012.stdout:Feb 18 11:38:19 smithi012 podman[71535]: 2024-02-18 11:38:19.583389269 +0000 UTC m=+0.384689372 container init bb1d1e660e10b85641096fd76db7438805f424e2e9f306662311ee2655839cee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-d, org.label-schema.build-date=20240214, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-02-18T11:38:19.923 INFO:journalctl@ceph.mds.d.smithi012.stdout:Feb 18 11:38:19 smithi012 podman[71535]: 2024-02-18 11:38:19.586656628 +0000 UTC m=+0.387956739 container start bb1d1e660e10b85641096fd76db7438805f424e2e9f306662311ee2655839cee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-d, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-02-18T11:38:19.923 INFO:journalctl@ceph.mds.d.smithi012.stdout:Feb 18 11:38:19 smithi012 ceph-mds[71593]: -- 172.21.15.12:0/620492389 <== mon.0 v2:172.21.15.12:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a3e17ac1a0 con 0x55a3e1783000 2024-02-18T11:38:19.923 INFO:journalctl@ceph.mds.d.smithi012.stdout:Feb 18 11:38:19 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-d[71589]: starting mds.d at 2024-02-18T11:38:19.923 INFO:journalctl@ceph.mds.d.smithi012.stdout:Feb 18 11:38:19 smithi012 bash[71535]: bb1d1e660e10b85641096fd76db7438805f424e2e9f306662311ee2655839cee 2024-02-18T11:38:19.923 INFO:journalctl@ceph.mds.d.smithi012.stdout:Feb 18 11:38:19 smithi012 systemd[1]: Started Ceph mds.d for 0c8bb0de-ce51-11ee-95ba-87774f69a715. 2024-02-18T11:38:20.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:20 smithi188 ceph-mon[36925]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-18T11:38:20.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:20 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/2664161990' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-18T11:38:20.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:20 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:20.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:20 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:20.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:20 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:20.355 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:20 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-18T11:38:20.355 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:20 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-18T11:38:20.355 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:20 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:38:20.355 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:20 smithi188 ceph-mon[36925]: Deploying daemon mds.e on smithi158 2024-02-18T11:38:20.355 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:20 smithi158 ceph-mon[36626]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-18T11:38:20.355 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:20 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/2664161990' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-18T11:38:20.355 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:20 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:20.355 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:20 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:20.356 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:20 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:20.356 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:20 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-18T11:38:20.356 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:20 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-18T11:38:20.356 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:20 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:38:20.356 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:20 smithi158 ceph-mon[36626]: Deploying daemon mds.e on smithi158 2024-02-18T11:38:20.423 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:20 smithi012 ceph-mon[32460]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-18T11:38:20.423 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:20 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/2664161990' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-18T11:38:20.423 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:20 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:20.423 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:20 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:20.423 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:20 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:20.423 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:20 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-18T11:38:20.423 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:20 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-18T11:38:20.423 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:20 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:38:20.423 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:20 smithi012 ceph-mon[32460]: Deploying daemon mds.e on smithi158 2024-02-18T11:38:21.325 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:21 smithi012 ceph-mon[32460]: mds.? [v2:172.21.15.12:6836/2144692455,v1:172.21.15.12:6837/2144692455] up:boot 2024-02-18T11:38:21.325 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:21 smithi012 ceph-mon[32460]: fsmap 4 up:standby 2024-02-18T11:38:21.325 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:21 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-02-18T11:38:21.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:21 smithi188 ceph-mon[36925]: mds.? [v2:172.21.15.12:6836/2144692455,v1:172.21.15.12:6837/2144692455] up:boot 2024-02-18T11:38:21.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:21 smithi188 ceph-mon[36925]: fsmap 4 up:standby 2024-02-18T11:38:21.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:21 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-02-18T11:38:21.374 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:21 smithi158 ceph-mon[36626]: mds.? [v2:172.21.15.12:6836/2144692455,v1:172.21.15.12:6837/2144692455] up:boot 2024-02-18T11:38:21.374 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:21 smithi158 ceph-mon[36626]: fsmap 4 up:standby 2024-02-18T11:38:21.375 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:21 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-02-18T11:38:21.598 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-18T11:38:21.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:21.722+0000 7f09ca840640 1 -- 172.21.15.12:0/3283408410 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09c41045f0 msgr2=0x7f09c41069e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:21.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:21.722+0000 7f09ca840640 1 --2- 172.21.15.12:0/3283408410 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09c41045f0 0x7f09c41069e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f09b4009900 tx=0x7f09b402bc30 comp rx=0 tx=0).stop 2024-02-18T11:38:21.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:21.722+0000 7f09ca840640 1 -- 172.21.15.12:0/3283408410 shutdown_connections 2024-02-18T11:38:21.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:21.722+0000 7f09ca840640 1 --2- 172.21.15.12:0/3283408410 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f09c4106f20 0x7f09c4109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:21.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:21.722+0000 7f09ca840640 1 --2- 172.21.15.12:0/3283408410 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09c41045f0 0x7f09c41069e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:21.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:21.722+0000 7f09ca840640 1 --2- 172.21.15.12:0/3283408410 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09c4101cc0 0x7f09c41040b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:21.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:21.722+0000 7f09ca840640 1 -- 172.21.15.12:0/3283408410 >> 172.21.15.12:0/3283408410 conn(0x7f09c40fb650 msgr2=0x7f09c40fda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:21.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:21.722+0000 7f09ca840640 1 -- 172.21.15.12:0/3283408410 shutdown_connections 2024-02-18T11:38:21.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:21.722+0000 7f09ca840640 1 -- 172.21.15.12:0/3283408410 wait complete. 2024-02-18T11:38:21.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:21.723+0000 7f09ca840640 1 Processor -- start 2024-02-18T11:38:21.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:21.723+0000 7f09ca840640 1 -- start start 2024-02-18T11:38:21.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:21.723+0000 7f09ca840640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f09c4101cc0 0x7f09c41972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:21.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:21.723+0000 7f09ca840640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09c41045f0 0x7f09c41977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:21.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:21.723+0000 7f09ca840640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09c4106f20 0x7f09c419e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:21.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:21.723+0000 7f09ca840640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c419eda0 con 0x7f09c41045f0 2024-02-18T11:38:21.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:21.723+0000 7f09ca840640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c419ef10 con 0x7f09c4101cc0 2024-02-18T11:38:21.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:21.723+0000 7f09ca840640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c419f0d0 con 0x7f09c4106f20 2024-02-18T11:38:21.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:21.723+0000 7f09c37fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09c41045f0 0x7f09c41977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:21.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:21.723+0000 7f09c37fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09c41045f0 0x7f09c41977f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54042/0 (socket says 172.21.15.12:54042) 2024-02-18T11:38:21.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:21.723+0000 7f09c37fe640 1 -- 172.21.15.12:0/2152422491 learned_addr learned my addr 172.21.15.12:0/2152422491 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:38:21.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:21.723+0000 7f09c3fff640 1 --2- 172.21.15.12:0/2152422491 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f09c4101cc0 0x7f09c41972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:21.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:21.723+0000 7f09c8ddd640 1 --2- 172.21.15.12:0/2152422491 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09c4106f20 0x7f09c419e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:21.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:21.724+0000 7f09c37fe640 1 -- 172.21.15.12:0/2152422491 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09c4106f20 msgr2=0x7f09c419e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:21.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:21.724+0000 7f09c37fe640 1 --2- 172.21.15.12:0/2152422491 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09c4106f20 0x7f09c419e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:21.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:21.724+0000 7f09c37fe640 1 -- 172.21.15.12:0/2152422491 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f09c4101cc0 msgr2=0x7f09c41972b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:21.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:21.724+0000 7f09c37fe640 1 --2- 172.21.15.12:0/2152422491 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f09c4101cc0 0x7f09c41972b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:21.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:21.724+0000 7f09c37fe640 1 -- 172.21.15.12:0/2152422491 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09b4009560 con 0x7f09c41045f0 2024-02-18T11:38:21.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:21.724+0000 7f09c37fe640 1 --2- 172.21.15.12:0/2152422491 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09c41045f0 0x7f09c41977f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f09b4005ab0 tx=0x7f09b4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:21.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:21.724+0000 7f09c17fa640 1 -- 172.21.15.12:0/2152422491 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b4004aa0 con 0x7f09c41045f0 2024-02-18T11:38:21.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:21.724+0000 7f09ca840640 1 -- 172.21.15.12:0/2152422491 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09c419f350 con 0x7f09c41045f0 2024-02-18T11:38:21.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:21.724+0000 7f09c17fa640 1 -- 172.21.15.12:0/2152422491 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f09b40052d0 con 0x7f09c41045f0 2024-02-18T11:38:21.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:21.724+0000 7f09c17fa640 1 -- 172.21.15.12:0/2152422491 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b4003e80 con 0x7f09c41045f0 2024-02-18T11:38:21.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:21.724+0000 7f09ca840640 1 -- 172.21.15.12:0/2152422491 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09c419f8b0 con 0x7f09c41045f0 2024-02-18T11:38:21.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:21.725+0000 7f09ca840640 1 -- 172.21.15.12:0/2152422491 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09c40fd140 con 0x7f09c41045f0 2024-02-18T11:38:21.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:21.727+0000 7f09c17fa640 1 -- 172.21.15.12:0/2152422491 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) v1 ==== 137245+0+0 (secure 0 0 0) 0x7f09b4005440 con 0x7f09c41045f0 2024-02-18T11:38:21.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:21.727+0000 7f09c17fa640 1 --2- 172.21.15.12:0/2152422491 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f09b00a2750 0x7f09b00a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:21.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:21.728+0000 7f09c17fa640 1 -- 172.21.15.12:0/2152422491 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f09b40f0b40 con 0x7f09c41045f0 2024-02-18T11:38:21.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:21.728+0000 7f09c3fff640 1 --2- 172.21.15.12:0/2152422491 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f09b00a2750 0x7f09b00a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:21.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:21.728+0000 7f09c3fff640 1 --2- 172.21.15.12:0/2152422491 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f09b00a2750 0x7f09b00a4c10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f09ac005fd0 tx=0x7f09ac00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:21.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:21.731+0000 7f09c17fa640 1 -- 172.21.15.12:0/2152422491 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f09b40b9cd0 con 0x7f09c41045f0 2024-02-18T11:38:21.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:21.952+0000 7f09ca840640 1 -- 172.21.15.12:0/2152422491 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f09c410cbf0 con 0x7f09c41045f0 2024-02-18T11:38:21.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:21.952+0000 7f09c17fa640 1 -- 172.21.15.12:0/2152422491 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3704 (secure 0 0 0) 0x7f09b40bd390 con 0x7f09c41045f0 2024-02-18T11:38:21.947 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:38:21.947 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1788335302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1788335302},{"type":"v1","addr":"172.21.15.12:6835","nonce":1788335302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/2144692455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":2144692455},{"type":"v1","addr":"172.21.15.12:6837","nonce":2144692455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1207202465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1207202465},{"type":"v1","addr":"172.21.15.188:6833","nonce":1207202465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6833/2636348323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6832","nonce":2636348323},{"type":"v1","addr":"172.21.15.158:6833","nonce":2636348323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-02-18T11:38:21.947 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 5 2024-02-18T11:38:21.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:21.955+0000 7f09ca840640 1 -- 172.21.15.12:0/2152422491 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f09b00a2750 msgr2=0x7f09b00a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:21.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:21.955+0000 7f09ca840640 1 --2- 172.21.15.12:0/2152422491 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f09b00a2750 0x7f09b00a4c10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f09ac005fd0 tx=0x7f09ac00b390 comp rx=0 tx=0).stop 2024-02-18T11:38:21.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:21.955+0000 7f09ca840640 1 -- 172.21.15.12:0/2152422491 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09c41045f0 msgr2=0x7f09c41977f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:21.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:21.955+0000 7f09ca840640 1 --2- 172.21.15.12:0/2152422491 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09c41045f0 0x7f09c41977f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f09b4005ab0 tx=0x7f09b4004270 comp rx=0 tx=0).stop 2024-02-18T11:38:21.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:21.955+0000 7f09ca840640 1 -- 172.21.15.12:0/2152422491 shutdown_connections 2024-02-18T11:38:21.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:21.955+0000 7f09ca840640 1 --2- 172.21.15.12:0/2152422491 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f09b00a2750 0x7f09b00a4c10 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:21.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:21.955+0000 7f09ca840640 1 --2- 172.21.15.12:0/2152422491 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09c4106f20 0x7f09c419e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:21.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:21.955+0000 7f09ca840640 1 --2- 172.21.15.12:0/2152422491 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09c41045f0 0x7f09c41977f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:21.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:21.955+0000 7f09ca840640 1 --2- 172.21.15.12:0/2152422491 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f09c4101cc0 0x7f09c41972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:21.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:21.955+0000 7f09ca840640 1 -- 172.21.15.12:0/2152422491 >> 172.21.15.12:0/2152422491 conn(0x7f09c40fb650 msgr2=0x7f09c41026e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:21.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:21.955+0000 7f09ca840640 1 -- 172.21.15.12:0/2152422491 shutdown_connections 2024-02-18T11:38:21.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:21.955+0000 7f09ca840640 1 -- 172.21.15.12:0/2152422491 wait complete. 2024-02-18T11:38:21.994 INFO:journalctl@ceph.mds.e.smithi158.stdout:Feb 18 11:38:21 smithi158 systemd[1]: Starting Ceph mds.e for 0c8bb0de-ce51-11ee-95ba-87774f69a715... 2024-02-18T11:38:22.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:22 smithi012 ceph-mon[32460]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-18T11:38:22.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:22 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/2152422491' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-18T11:38:22.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:22 smithi188 ceph-mon[36925]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-18T11:38:22.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:22 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/2152422491' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-18T11:38:22.419 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:22 smithi158 ceph-mon[36626]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-18T11:38:22.419 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:22 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/2152422491' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-18T11:38:22.419 INFO:journalctl@ceph.mds.e.smithi158.stdout:Feb 18 11:38:22 smithi158 podman[66188]: 2024-02-18 11:38:22.045419852 +0000 UTC m=+0.017707279 image pull 53c33ee3bf62dc411b650f86eedbbee7698efb1001b4c063271dcd8d65b41165 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1 2024-02-18T11:38:22.419 INFO:journalctl@ceph.mds.e.smithi158.stdout:Feb 18 11:38:22 smithi158 podman[66188]: 2024-02-18 11:38:22.181134707 +0000 UTC m=+0.153422124 container create b0f4c55b33cd7cee45e3933153f3253031e7d02a5f357e8105efccbf8c21225f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-e, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, io.buildah.version=1.33.5) 2024-02-18T11:38:22.420 INFO:journalctl@ceph.mds.e.smithi158.stdout:Feb 18 11:38:22 smithi158 podman[66188]: 2024-02-18 11:38:22.425238608 +0000 UTC m=+0.397526023 container init b0f4c55b33cd7cee45e3933153f3253031e7d02a5f357e8105efccbf8c21225f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-e, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, ceph=True, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-18T11:38:22.744 INFO:journalctl@ceph.mds.e.smithi158.stdout:Feb 18 11:38:22 smithi158 podman[66188]: 2024-02-18 11:38:22.428135287 +0000 UTC m=+0.400422703 container start b0f4c55b33cd7cee45e3933153f3253031e7d02a5f357e8105efccbf8c21225f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-e, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-02-18T11:38:22.744 INFO:journalctl@ceph.mds.e.smithi158.stdout:Feb 18 11:38:22 smithi158 ceph-mds[66207]: -- 172.21.15.158:0/203380770 <== mon.2 v2:172.21.15.158:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559e9a0f61a0 con 0x559e9a0cd000 2024-02-18T11:38:22.744 INFO:journalctl@ceph.mds.e.smithi158.stdout:Feb 18 11:38:22 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-e[66203]: starting mds.e at 2024-02-18T11:38:22.745 INFO:journalctl@ceph.mds.e.smithi158.stdout:Feb 18 11:38:22 smithi158 bash[66188]: b0f4c55b33cd7cee45e3933153f3253031e7d02a5f357e8105efccbf8c21225f 2024-02-18T11:38:22.745 INFO:journalctl@ceph.mds.e.smithi158.stdout:Feb 18 11:38:22 smithi158 systemd[1]: Started Ceph mds.e for 0c8bb0de-ce51-11ee-95ba-87774f69a715. 2024-02-18T11:38:23.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:23 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:23.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:23 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:23.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:23 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:23.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:23 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-18T11:38:23.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:23 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-18T11:38:23.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:23 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:38:23.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:23 smithi188 ceph-mon[36925]: Deploying daemon mds.f on smithi188 2024-02-18T11:38:23.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:23 smithi188 ceph-mon[36925]: mds.? [v2:172.21.15.158:6834/3673253642,v1:172.21.15.158:6835/3673253642] up:boot 2024-02-18T11:38:23.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:23 smithi188 ceph-mon[36925]: fsmap 5 up:standby 2024-02-18T11:38:23.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:23 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-02-18T11:38:23.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:23 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:23.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:23 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:23.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:23 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:23.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:23 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-18T11:38:23.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:23 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-18T11:38:23.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:23 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:38:23.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:23 smithi012 ceph-mon[32460]: Deploying daemon mds.f on smithi188 2024-02-18T11:38:23.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:23 smithi012 ceph-mon[32460]: mds.? [v2:172.21.15.158:6834/3673253642,v1:172.21.15.158:6835/3673253642] up:boot 2024-02-18T11:38:23.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:23 smithi012 ceph-mon[32460]: fsmap 5 up:standby 2024-02-18T11:38:23.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:23 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-02-18T11:38:23.961 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-18T11:38:23.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:23 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:23.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:23 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:23.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:23 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:23.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:23 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-18T11:38:23.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:23 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-18T11:38:23.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:23 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:38:23.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:23 smithi158 ceph-mon[36626]: Deploying daemon mds.f on smithi188 2024-02-18T11:38:23.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:23 smithi158 ceph-mon[36626]: mds.? [v2:172.21.15.158:6834/3673253642,v1:172.21.15.158:6835/3673253642] up:boot 2024-02-18T11:38:23.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:23 smithi158 ceph-mon[36626]: fsmap 5 up:standby 2024-02-18T11:38:23.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:23 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-02-18T11:38:24.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:24.083+0000 7fe0093d0640 1 -- 172.21.15.12:0/552074716 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe004101d10 msgr2=0x7fe0041085a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:24.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:24.083+0000 7fe0093d0640 1 --2- 172.21.15.12:0/552074716 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe004101d10 0x7fe0041085a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdffc009ca0 tx=0x7fdffc02c0f0 comp rx=0 tx=0).stop 2024-02-18T11:38:24.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:24.083+0000 7fe0093d0640 1 -- 172.21.15.12:0/552074716 shutdown_connections 2024-02-18T11:38:24.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:24.083+0000 7fe0093d0640 1 --2- 172.21.15.12:0/552074716 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe004101d10 0x7fe0041085a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:24.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:24.083+0000 7fe0093d0640 1 --2- 172.21.15.12:0/552074716 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe004101350 0x7fe0041017d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:24.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:24.083+0000 7fe0093d0640 1 --2- 172.21.15.12:0/552074716 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe004100160 0x7fe004100580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:24.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:24.083+0000 7fe0093d0640 1 -- 172.21.15.12:0/552074716 >> 172.21.15.12:0/552074716 conn(0x7fe0040fb6f0 msgr2=0x7fe0040fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:24.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:24.083+0000 7fe0093d0640 1 -- 172.21.15.12:0/552074716 shutdown_connections 2024-02-18T11:38:24.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:24.083+0000 7fe0093d0640 1 -- 172.21.15.12:0/552074716 wait complete. 2024-02-18T11:38:24.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:24.084+0000 7fe0093d0640 1 Processor -- start 2024-02-18T11:38:24.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:24.084+0000 7fe0093d0640 1 -- start start 2024-02-18T11:38:24.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:24.084+0000 7fe0093d0640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe004100160 0x7fe00406a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:24.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:24.084+0000 7fe0093d0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe004101350 0x7fe00419bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:24.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:24.084+0000 7fe0093d0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe004101d10 0x7fe00419e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:24.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:24.084+0000 7fe0093d0640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe00419ed40 con 0x7fe004101350 2024-02-18T11:38:24.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:24.084+0000 7fe0093d0640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe00419eeb0 con 0x7fe004100160 2024-02-18T11:38:24.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:24.084+0000 7fe0093d0640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe00419f070 con 0x7fe004101d10 2024-02-18T11:38:24.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:24.084+0000 7fdffbfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe004101350 0x7fe00419bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:24.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:24.084+0000 7fdffbfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe004101350 0x7fe00419bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54052/0 (socket says 172.21.15.12:54052) 2024-02-18T11:38:24.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:24.084+0000 7fdffbfff640 1 -- 172.21.15.12:0/3942870776 learned_addr learned my addr 172.21.15.12:0/3942870776 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:38:24.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:24.084+0000 7fe002ffd640 1 --2- 172.21.15.12:0/3942870776 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe004100160 0x7fe00406a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:24.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:24.084+0000 7fe0037fe640 1 --2- 172.21.15.12:0/3942870776 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe004101d10 0x7fe00419e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:24.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:24.084+0000 7fdffbfff640 1 -- 172.21.15.12:0/3942870776 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe004101d10 msgr2=0x7fe00419e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:24.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:24.084+0000 7fdffbfff640 1 --2- 172.21.15.12:0/3942870776 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe004101d10 0x7fe00419e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:24.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:24.084+0000 7fdffbfff640 1 -- 172.21.15.12:0/3942870776 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe004100160 msgr2=0x7fe00406a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:24.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:24.084+0000 7fdffbfff640 1 --2- 172.21.15.12:0/3942870776 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe004100160 0x7fe00406a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:24.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:24.084+0000 7fdffbfff640 1 -- 172.21.15.12:0/3942870776 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdffc009900 con 0x7fe004101350 2024-02-18T11:38:24.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:24.085+0000 7fdffbfff640 1 --2- 172.21.15.12:0/3942870776 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe004101350 0x7fe00419bf40 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdfec00c890 tx=0x7fdfec00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:24.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:24.085+0000 7fe000ff9640 1 -- 172.21.15.12:0/3942870776 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfec004220 con 0x7fe004101350 2024-02-18T11:38:24.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:24.085+0000 7fe0093d0640 1 -- 172.21.15.12:0/3942870776 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe00419f2f0 con 0x7fe004101350 2024-02-18T11:38:24.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:24.085+0000 7fe000ff9640 1 -- 172.21.15.12:0/3942870776 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdfec007e90 con 0x7fe004101350 2024-02-18T11:38:24.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:24.085+0000 7fe000ff9640 1 -- 172.21.15.12:0/3942870776 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfec004a00 con 0x7fe004101350 2024-02-18T11:38:24.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:24.085+0000 7fe0093d0640 1 -- 172.21.15.12:0/3942870776 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe00419f850 con 0x7fe004101350 2024-02-18T11:38:24.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:24.086+0000 7fe0093d0640 1 -- 172.21.15.12:0/3942870776 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe004107480 con 0x7fe004101350 2024-02-18T11:38:24.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:24.087+0000 7fe000ff9640 1 -- 172.21.15.12:0/3942870776 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) v1 ==== 137245+0+0 (secure 0 0 0) 0x7fdfec0079b0 con 0x7fe004101350 2024-02-18T11:38:24.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:24.088+0000 7fe000ff9640 1 --2- 172.21.15.12:0/3942870776 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fdff00a2750 0x7fdff00a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:24.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:24.088+0000 7fe002ffd640 1 --2- 172.21.15.12:0/3942870776 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fdff00a2750 0x7fdff00a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:24.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:24.088+0000 7fe000ff9640 1 -- 172.21.15.12:0/3942870776 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fdfec0cde40 con 0x7fe004101350 2024-02-18T11:38:24.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:24.088+0000 7fe002ffd640 1 --2- 172.21.15.12:0/3942870776 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fdff00a2750 0x7fdff00a4c10 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fdff4005fd0 tx=0x7fdff400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:24.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:24.092+0000 7fe000ff9640 1 -- 172.21.15.12:0/3942870776 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdfec096fd0 con 0x7fe004101350 2024-02-18T11:38:24.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:24.310+0000 7fe0093d0640 1 -- 172.21.15.12:0/3942870776 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe00410bb00 con 0x7fe004101350 2024-02-18T11:38:24.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:24.311+0000 7fe000ff9640 1 -- 172.21.15.12:0/3942870776 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4498 (secure 0 0 0) 0x7fe00410bb00 con 0x7fe004101350 2024-02-18T11:38:24.305 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:38:24.305 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1788335302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1788335302},{"type":"v1","addr":"172.21.15.12:6835","nonce":1788335302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/2144692455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":2144692455},{"type":"v1","addr":"172.21.15.12:6837","nonce":2144692455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1207202465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1207202465},{"type":"v1","addr":"172.21.15.188:6833","nonce":1207202465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6833/2636348323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6832","nonce":2636348323},{"type":"v1","addr":"172.21.15.158:6833","nonce":2636348323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24446,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6835/3673253642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":3673253642},{"type":"v1","addr":"172.21.15.158:6835","nonce":3673253642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2024-02-18T11:38:24.306 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 6 2024-02-18T11:38:24.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:24.313+0000 7fe0093d0640 1 -- 172.21.15.12:0/3942870776 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fdff00a2750 msgr2=0x7fdff00a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:24.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:24.313+0000 7fe0093d0640 1 --2- 172.21.15.12:0/3942870776 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fdff00a2750 0x7fdff00a4c10 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fdff4005fd0 tx=0x7fdff400b390 comp rx=0 tx=0).stop 2024-02-18T11:38:24.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:24.313+0000 7fe0093d0640 1 -- 172.21.15.12:0/3942870776 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe004101350 msgr2=0x7fe00419bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:24.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:24.313+0000 7fe0093d0640 1 --2- 172.21.15.12:0/3942870776 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe004101350 0x7fe00419bf40 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdfec00c890 tx=0x7fdfec00cd50 comp rx=0 tx=0).stop 2024-02-18T11:38:24.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:24.313+0000 7fe0093d0640 1 -- 172.21.15.12:0/3942870776 shutdown_connections 2024-02-18T11:38:24.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:24.313+0000 7fe0093d0640 1 --2- 172.21.15.12:0/3942870776 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fdff00a2750 0x7fdff00a4c10 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:24.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:24.313+0000 7fe0093d0640 1 --2- 172.21.15.12:0/3942870776 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe004101d10 0x7fe00419e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:24.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:24.313+0000 7fe0093d0640 1 --2- 172.21.15.12:0/3942870776 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe004101350 0x7fe00419bf40 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:24.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:24.313+0000 7fe0093d0640 1 --2- 172.21.15.12:0/3942870776 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe004100160 0x7fe00406a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:24.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:24.313+0000 7fe0093d0640 1 -- 172.21.15.12:0/3942870776 >> 172.21.15.12:0/3942870776 conn(0x7fe0040fb6f0 msgr2=0x7fe004104650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:24.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:24.313+0000 7fe0093d0640 1 -- 172.21.15.12:0/3942870776 shutdown_connections 2024-02-18T11:38:24.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:24.313+0000 7fe0093d0640 1 -- 172.21.15.12:0/3942870776 wait complete. 2024-02-18T11:38:24.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:24 smithi188 ceph-mon[36925]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-18T11:38:24.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:24 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/3942870776' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-18T11:38:24.855 INFO:journalctl@ceph.mds.f.smithi188.stdout:Feb 18 11:38:24 smithi188 systemd[1]: Starting Ceph mds.f for 0c8bb0de-ce51-11ee-95ba-87774f69a715... 2024-02-18T11:38:24.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:24 smithi012 ceph-mon[32460]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-18T11:38:24.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:24 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/3942870776' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-18T11:38:24.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:24 smithi158 ceph-mon[36626]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-18T11:38:24.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:24 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/3942870776' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-18T11:38:25.260 INFO:journalctl@ceph.mds.f.smithi188.stdout:Feb 18 11:38:24 smithi188 podman[67276]: 2024-02-18 11:38:24.972071425 +0000 UTC m=+0.113571134 container create 38349ecccb970532609eb26ed89e33c7e22d4f17cd87ee6a73fb798e5b913ed4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-f, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-02-18T11:38:25.260 INFO:journalctl@ceph.mds.f.smithi188.stdout:Feb 18 11:38:24 smithi188 podman[67276]: 2024-02-18 11:38:24.876516541 +0000 UTC m=+0.018016259 image pull 53c33ee3bf62dc411b650f86eedbbee7698efb1001b4c063271dcd8d65b41165 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1 2024-02-18T11:38:25.260 INFO:journalctl@ceph.mds.f.smithi188.stdout:Feb 18 11:38:25 smithi188 podman[67276]: 2024-02-18 11:38:25.191147769 +0000 UTC m=+0.332647483 container init 38349ecccb970532609eb26ed89e33c7e22d4f17cd87ee6a73fb798e5b913ed4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-f, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240217.015652-1c920af) 2024-02-18T11:38:25.260 INFO:journalctl@ceph.mds.f.smithi188.stdout:Feb 18 11:38:25 smithi188 podman[67276]: 2024-02-18 11:38:25.194047701 +0000 UTC m=+0.335547417 container start 38349ecccb970532609eb26ed89e33c7e22d4f17cd87ee6a73fb798e5b913ed4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-f, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-18T11:38:25.260 INFO:journalctl@ceph.mds.f.smithi188.stdout:Feb 18 11:38:25 smithi188 ceph-mds[67296]: -- 172.21.15.188:0/3371750411 <== mon.1 v2:172.21.15.188:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d64e2941a0 con 0x55d64e26b000 2024-02-18T11:38:25.260 INFO:journalctl@ceph.mds.f.smithi188.stdout:Feb 18 11:38:25 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-f[67292]: starting mds.f at 2024-02-18T11:38:25.260 INFO:journalctl@ceph.mds.f.smithi188.stdout:Feb 18 11:38:25 smithi188 bash[67276]: 38349ecccb970532609eb26ed89e33c7e22d4f17cd87ee6a73fb798e5b913ed4 2024-02-18T11:38:25.604 INFO:journalctl@ceph.mds.f.smithi188.stdout:Feb 18 11:38:25 smithi188 systemd[1]: Started Ceph mds.f for 0c8bb0de-ce51-11ee-95ba-87774f69a715. 2024-02-18T11:38:26.318 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-18T11:38:26.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:26.437+0000 7fd39900e640 1 -- 172.21.15.12:0/2923396761 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd3941012d0 msgr2=0x7fd394101750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:26.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:26.437+0000 7fd39900e640 1 --2- 172.21.15.12:0/2923396761 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd3941012d0 0x7fd394101750 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd37c009900 tx=0x7fd37c02bc30 comp rx=0 tx=0).stop 2024-02-18T11:38:26.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:26.437+0000 7fd39900e640 1 -- 172.21.15.12:0/2923396761 shutdown_connections 2024-02-18T11:38:26.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:26.437+0000 7fd39900e640 1 --2- 172.21.15.12:0/2923396761 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd394101c90 0x7fd394108520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:26.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:26.437+0000 7fd39900e640 1 --2- 172.21.15.12:0/2923396761 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd3941012d0 0x7fd394101750 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:26.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:26.437+0000 7fd39900e640 1 --2- 172.21.15.12:0/2923396761 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd3941000e0 0x7fd394100500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:26.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:26.437+0000 7fd39900e640 1 -- 172.21.15.12:0/2923396761 >> 172.21.15.12:0/2923396761 conn(0x7fd3940fb670 msgr2=0x7fd3940fda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:26.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:26.437+0000 7fd39900e640 1 -- 172.21.15.12:0/2923396761 shutdown_connections 2024-02-18T11:38:26.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:26.437+0000 7fd39900e640 1 -- 172.21.15.12:0/2923396761 wait complete. 2024-02-18T11:38:26.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:26.438+0000 7fd39900e640 1 Processor -- start 2024-02-18T11:38:26.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:26.438+0000 7fd39900e640 1 -- start start 2024-02-18T11:38:26.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:26.438+0000 7fd39900e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3941000e0 0x7fd39406a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:26.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:26.438+0000 7fd39900e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd3941012d0 0x7fd39419e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:26.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:26.438+0000 7fd39900e640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd394101c90 0x7fd3941a0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:26.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:26.438+0000 7fd39900e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3941a0f50 con 0x7fd3941012d0 2024-02-18T11:38:26.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:26.438+0000 7fd39900e640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3941a10c0 con 0x7fd394101c90 2024-02-18T11:38:26.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:26.438+0000 7fd39900e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3941a1280 con 0x7fd3941000e0 2024-02-18T11:38:26.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:26.438+0000 7fd38bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd3941012d0 0x7fd39419e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:26.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:26.438+0000 7fd392d9d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3941000e0 0x7fd39406a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:26.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:26.438+0000 7fd38bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd3941012d0 0x7fd39419e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54062/0 (socket says 172.21.15.12:54062) 2024-02-18T11:38:26.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:26.438+0000 7fd38bfff640 1 -- 172.21.15.12:0/3169123992 learned_addr learned my addr 172.21.15.12:0/3169123992 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:38:26.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:26.438+0000 7fd39359e640 1 --2- 172.21.15.12:0/3169123992 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd394101c90 0x7fd3941a0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:26.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:26.439+0000 7fd38bfff640 1 -- 172.21.15.12:0/3169123992 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3941000e0 msgr2=0x7fd39406a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:26.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:26.439+0000 7fd38bfff640 1 --2- 172.21.15.12:0/3169123992 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3941000e0 0x7fd39406a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:26.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:26.439+0000 7fd38bfff640 1 -- 172.21.15.12:0/3169123992 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd394101c90 msgr2=0x7fd3941a0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:26.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:26.439+0000 7fd38bfff640 1 --2- 172.21.15.12:0/3169123992 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd394101c90 0x7fd3941a0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:26.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:26.439+0000 7fd38bfff640 1 -- 172.21.15.12:0/3169123992 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd37c009560 con 0x7fd3941012d0 2024-02-18T11:38:26.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:26.439+0000 7fd38bfff640 1 --2- 172.21.15.12:0/3169123992 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd3941012d0 0x7fd39419e330 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd37c006010 tx=0x7fd37c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:26.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:26.439+0000 7fd390d99640 1 -- 172.21.15.12:0/3169123992 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37c0044e0 con 0x7fd3941012d0 2024-02-18T11:38:26.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:26.439+0000 7fd39900e640 1 -- 172.21.15.12:0/3169123992 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3941a1500 con 0x7fd3941012d0 2024-02-18T11:38:26.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:26.439+0000 7fd390d99640 1 -- 172.21.15.12:0/3169123992 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd37c0054e0 con 0x7fd3941012d0 2024-02-18T11:38:26.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:26.439+0000 7fd390d99640 1 -- 172.21.15.12:0/3169123992 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37c03e450 con 0x7fd3941012d0 2024-02-18T11:38:26.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:26.439+0000 7fd39900e640 1 -- 172.21.15.12:0/3169123992 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3941a1a60 con 0x7fd3941012d0 2024-02-18T11:38:26.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:26.440+0000 7fd39900e640 1 -- 172.21.15.12:0/3169123992 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd394107480 con 0x7fd3941012d0 2024-02-18T11:38:26.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:26.441+0000 7fd390d99640 1 -- 172.21.15.12:0/3169123992 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) v1 ==== 137245+0+0 (secure 0 0 0) 0x7fd37c0035f0 con 0x7fd3941012d0 2024-02-18T11:38:26.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:26.442+0000 7fd390d99640 1 --2- 172.21.15.12:0/3169123992 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fd3800a2750 0x7fd3800a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:26.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:26.442+0000 7fd392d9d640 1 --2- 172.21.15.12:0/3169123992 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fd3800a2750 0x7fd3800a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:26.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:26.442+0000 7fd390d99640 1 -- 172.21.15.12:0/3169123992 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd37c0f0520 con 0x7fd3941012d0 2024-02-18T11:38:26.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:26.442+0000 7fd392d9d640 1 --2- 172.21.15.12:0/3169123992 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fd3800a2750 0x7fd3800a4c10 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd394101130 tx=0x7fd38400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:26.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:26.446+0000 7fd390d99640 1 -- 172.21.15.12:0/3169123992 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd37c0b96b0 con 0x7fd3941012d0 2024-02-18T11:38:26.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:26 smithi158 ceph-mon[36626]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-18T11:38:26.495 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:26 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:26.495 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:26 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:26.495 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:26 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:26.495 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:26 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-18T11:38:26.495 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:26 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-18T11:38:26.495 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:26 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:38:26.495 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:26 smithi158 ceph-mon[36626]: Deploying daemon mds.c on smithi188 2024-02-18T11:38:26.495 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:26 smithi158 ceph-mon[36626]: mds.? [v2:172.21.15.188:6834/806709644,v1:172.21.15.188:6835/806709644] up:boot 2024-02-18T11:38:26.495 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:26 smithi158 ceph-mon[36626]: fsmap 6 up:standby 2024-02-18T11:38:26.495 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:26 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-18T11:38:26.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:26 smithi188 ceph-mon[36925]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-18T11:38:26.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:26 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:26.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:26 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:26.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:26 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:26.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:26 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-18T11:38:26.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:26 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-18T11:38:26.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:26 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:38:26.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:26 smithi188 ceph-mon[36925]: Deploying daemon mds.c on smithi188 2024-02-18T11:38:26.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:26 smithi188 ceph-mon[36925]: mds.? [v2:172.21.15.188:6834/806709644,v1:172.21.15.188:6835/806709644] up:boot 2024-02-18T11:38:26.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:26 smithi188 ceph-mon[36925]: fsmap 6 up:standby 2024-02-18T11:38:26.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:26 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-18T11:38:26.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:26.665+0000 7fd39900e640 1 -- 172.21.15.12:0/3169123992 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd39402ce10 con 0x7fd3941012d0 2024-02-18T11:38:26.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:26.666+0000 7fd390d99640 1 -- 172.21.15.12:0/3169123992 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5289 (secure 0 0 0) 0x7fd37c0bcd70 con 0x7fd3941012d0 2024-02-18T11:38:26.661 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:38:26.661 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1788335302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1788335302},{"type":"v1","addr":"172.21.15.12:6835","nonce":1788335302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/2144692455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":2144692455},{"type":"v1","addr":"172.21.15.12:6837","nonce":2144692455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1207202465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1207202465},{"type":"v1","addr":"172.21.15.188:6833","nonce":1207202465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6833/2636348323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6832","nonce":2636348323},{"type":"v1","addr":"172.21.15.158:6833","nonce":2636348323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24446,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6835/3673253642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":3673253642},{"type":"v1","addr":"172.21.15.158:6835","nonce":3673253642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/806709644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":806709644},{"type":"v1","addr":"172.21.15.188:6835","nonce":806709644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[]} 2024-02-18T11:38:26.661 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 7 2024-02-18T11:38:26.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:26.668+0000 7fd39900e640 1 -- 172.21.15.12:0/3169123992 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fd3800a2750 msgr2=0x7fd3800a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:26.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:26.668+0000 7fd39900e640 1 --2- 172.21.15.12:0/3169123992 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fd3800a2750 0x7fd3800a4c10 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd394101130 tx=0x7fd38400b390 comp rx=0 tx=0).stop 2024-02-18T11:38:26.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:26.668+0000 7fd39900e640 1 -- 172.21.15.12:0/3169123992 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd3941012d0 msgr2=0x7fd39419e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:26.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:26.668+0000 7fd39900e640 1 --2- 172.21.15.12:0/3169123992 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd3941012d0 0x7fd39419e330 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd37c006010 tx=0x7fd37c0042f0 comp rx=0 tx=0).stop 2024-02-18T11:38:26.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:26.668+0000 7fd39900e640 1 -- 172.21.15.12:0/3169123992 shutdown_connections 2024-02-18T11:38:26.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:26.668+0000 7fd39900e640 1 --2- 172.21.15.12:0/3169123992 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fd3800a2750 0x7fd3800a4c10 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:26.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:26.668+0000 7fd39900e640 1 --2- 172.21.15.12:0/3169123992 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd394101c90 0x7fd3941a0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:26.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:26.668+0000 7fd39900e640 1 --2- 172.21.15.12:0/3169123992 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd3941012d0 0x7fd39419e330 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:26.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:26.668+0000 7fd39900e640 1 --2- 172.21.15.12:0/3169123992 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3941000e0 0x7fd39406a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:26.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:26.668+0000 7fd39900e640 1 -- 172.21.15.12:0/3169123992 >> 172.21.15.12:0/3169123992 conn(0x7fd3940fb670 msgr2=0x7fd394104650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:26.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:26.668+0000 7fd39900e640 1 -- 172.21.15.12:0/3169123992 shutdown_connections 2024-02-18T11:38:26.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:26.668+0000 7fd39900e640 1 -- 172.21.15.12:0/3169123992 wait complete. 2024-02-18T11:38:26.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:26 smithi012 ceph-mon[32460]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-18T11:38:26.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:26 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:26.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:26 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:26.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:26 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:26.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:26 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-18T11:38:26.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:26 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-18T11:38:26.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:26 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:38:26.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:26 smithi012 ceph-mon[32460]: Deploying daemon mds.c on smithi188 2024-02-18T11:38:26.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:26 smithi012 ceph-mon[32460]: mds.? [v2:172.21.15.188:6834/806709644,v1:172.21.15.188:6835/806709644] up:boot 2024-02-18T11:38:26.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:26 smithi012 ceph-mon[32460]: fsmap 6 up:standby 2024-02-18T11:38:26.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:26 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-18T11:38:27.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:27 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/3169123992' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-18T11:38:27.605 INFO:journalctl@ceph.mds.c.smithi188.stdout:Feb 18 11:38:27 smithi188 systemd[1]: Starting Ceph mds.c for 0c8bb0de-ce51-11ee-95ba-87774f69a715... 2024-02-18T11:38:27.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:27 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/3169123992' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-18T11:38:27.744 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:27 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/3169123992' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-18T11:38:27.906 INFO:journalctl@ceph.mds.c.smithi188.stdout:Feb 18 11:38:27 smithi188 podman[67580]: 2024-02-18 11:38:27.547818787 +0000 UTC m=+0.017872321 image pull 53c33ee3bf62dc411b650f86eedbbee7698efb1001b4c063271dcd8d65b41165 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1 2024-02-18T11:38:27.906 INFO:journalctl@ceph.mds.c.smithi188.stdout:Feb 18 11:38:27 smithi188 podman[67580]: 2024-02-18 11:38:27.651870027 +0000 UTC m=+0.121923553 container create b51fc279ee8ac6bfc4b836b67a96b29832d13e549ebda5a6a3cad5a17269bb1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-c, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-18T11:38:27.907 INFO:journalctl@ceph.mds.c.smithi188.stdout:Feb 18 11:38:27 smithi188 podman[67580]: 2024-02-18 11:38:27.912828817 +0000 UTC m=+0.382882348 container init b51fc279ee8ac6bfc4b836b67a96b29832d13e549ebda5a6a3cad5a17269bb1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-c, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True) 2024-02-18T11:38:28.321 INFO:journalctl@ceph.mds.c.smithi188.stdout:Feb 18 11:38:27 smithi188 podman[67580]: 2024-02-18 11:38:27.915477735 +0000 UTC m=+0.385531253 container start b51fc279ee8ac6bfc4b836b67a96b29832d13e549ebda5a6a3cad5a17269bb1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-c, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240217.015652-1c920af, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-02-18T11:38:28.321 INFO:journalctl@ceph.mds.c.smithi188.stdout:Feb 18 11:38:27 smithi188 ceph-mds[67599]: -- 172.21.15.188:0/856381684 <== mon.1 v2:172.21.15.188:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561b049c41a0 con 0x561b0499ac00 2024-02-18T11:38:28.321 INFO:journalctl@ceph.mds.c.smithi188.stdout:Feb 18 11:38:27 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-c[67595]: starting mds.c at 2024-02-18T11:38:28.321 INFO:journalctl@ceph.mds.c.smithi188.stdout:Feb 18 11:38:28 smithi188 bash[67580]: b51fc279ee8ac6bfc4b836b67a96b29832d13e549ebda5a6a3cad5a17269bb1c 2024-02-18T11:38:28.321 INFO:journalctl@ceph.mds.c.smithi188.stdout:Feb 18 11:38:28 smithi188 systemd[1]: Started Ceph mds.c for 0c8bb0de-ce51-11ee-95ba-87774f69a715. 2024-02-18T11:38:28.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:28 smithi188 ceph-mon[36925]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-18T11:38:28.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:28 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:28.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:28 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:28.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:28 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:28.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:28 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-18T11:38:28.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:28 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-18T11:38:28.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:28 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:38:28.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:28 smithi012 ceph-mon[32460]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-18T11:38:28.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:28 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:28.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:28 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:28.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:28 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:28.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:28 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-18T11:38:28.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:28 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-18T11:38:28.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:28 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:38:28.675 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-18T11:38:28.699 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:28 smithi158 ceph-mon[36626]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-18T11:38:28.699 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:28 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:28.699 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:28 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:28.699 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:28 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:28.700 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:28 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-18T11:38:28.700 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:28 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-18T11:38:28.700 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:28 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:38:28.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:28.795+0000 7fb58598a640 1 -- 172.21.15.12:0/2720054304 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5801012f0 msgr2=0x7fb580101770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:28.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:28.795+0000 7fb58598a640 1 --2- 172.21.15.12:0/2720054304 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5801012f0 0x7fb580101770 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb574009900 tx=0x7fb57402bc30 comp rx=0 tx=0).stop 2024-02-18T11:38:28.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:28.795+0000 7fb58598a640 1 -- 172.21.15.12:0/2720054304 shutdown_connections 2024-02-18T11:38:28.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:28.795+0000 7fb58598a640 1 --2- 172.21.15.12:0/2720054304 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb580101cb0 0x7fb580108540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:28.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:28.795+0000 7fb58598a640 1 --2- 172.21.15.12:0/2720054304 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5801012f0 0x7fb580101770 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:28.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:28.795+0000 7fb58598a640 1 --2- 172.21.15.12:0/2720054304 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb580100100 0x7fb580100520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:28.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:28.795+0000 7fb58598a640 1 -- 172.21.15.12:0/2720054304 >> 172.21.15.12:0/2720054304 conn(0x7fb5800fb670 msgr2=0x7fb5800fdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:28.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:28.795+0000 7fb58598a640 1 -- 172.21.15.12:0/2720054304 shutdown_connections 2024-02-18T11:38:28.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:28.795+0000 7fb58598a640 1 -- 172.21.15.12:0/2720054304 wait complete. 2024-02-18T11:38:28.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:28.796+0000 7fb58598a640 1 Processor -- start 2024-02-18T11:38:28.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:28.796+0000 7fb58598a640 1 -- start start 2024-02-18T11:38:28.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:28.796+0000 7fb58598a640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb580100100 0x7fb58006a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:28.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:28.796+0000 7fb58598a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5801012f0 0x7fb58019bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:28.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:28.796+0000 7fb58598a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb580101cb0 0x7fb58019e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:28.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:28.796+0000 7fb58598a640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb58019ede0 con 0x7fb580101cb0 2024-02-18T11:38:28.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:28.796+0000 7fb58598a640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb58019ef50 con 0x7fb580100100 2024-02-18T11:38:28.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:28.796+0000 7fb58598a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb58019f110 con 0x7fb5801012f0 2024-02-18T11:38:28.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:28.796+0000 7fb57f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb580101cb0 0x7fb58019e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:28.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:28.796+0000 7fb57f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb580101cb0 0x7fb58019e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54072/0 (socket says 172.21.15.12:54072) 2024-02-18T11:38:28.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:28.796+0000 7fb57f7fe640 1 -- 172.21.15.12:0/2685849679 learned_addr learned my addr 172.21.15.12:0/2685849679 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:38:28.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:28.796+0000 7fb57e7fc640 1 --2- 172.21.15.12:0/2685849679 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5801012f0 0x7fb58019bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:28.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:28.796+0000 7fb57effd640 1 --2- 172.21.15.12:0/2685849679 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb580100100 0x7fb58006a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:28.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:28.796+0000 7fb57f7fe640 1 -- 172.21.15.12:0/2685849679 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5801012f0 msgr2=0x7fb58019bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:28.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:28.796+0000 7fb57f7fe640 1 --2- 172.21.15.12:0/2685849679 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5801012f0 0x7fb58019bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:28.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:28.796+0000 7fb57f7fe640 1 -- 172.21.15.12:0/2685849679 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb580100100 msgr2=0x7fb58006a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:28.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:28.796+0000 7fb57f7fe640 1 --2- 172.21.15.12:0/2685849679 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb580100100 0x7fb58006a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:28.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:28.796+0000 7fb57f7fe640 1 -- 172.21.15.12:0/2685849679 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb574009560 con 0x7fb580101cb0 2024-02-18T11:38:28.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:28.797+0000 7fb57f7fe640 1 --2- 172.21.15.12:0/2685849679 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb580101cb0 0x7fb58019e600 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb57800eb10 tx=0x7fb57800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:28.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:28.797+0000 7fb584988640 1 -- 172.21.15.12:0/2685849679 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb578004b20 con 0x7fb580101cb0 2024-02-18T11:38:28.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:28.797+0000 7fb584988640 1 -- 172.21.15.12:0/2685849679 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb578004c80 con 0x7fb580101cb0 2024-02-18T11:38:28.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:28.797+0000 7fb58598a640 1 -- 172.21.15.12:0/2685849679 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb58019f390 con 0x7fb580101cb0 2024-02-18T11:38:28.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:28.797+0000 7fb584988640 1 -- 172.21.15.12:0/2685849679 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb578010650 con 0x7fb580101cb0 2024-02-18T11:38:28.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:28.797+0000 7fb58598a640 1 -- 172.21.15.12:0/2685849679 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb58019f870 con 0x7fb580101cb0 2024-02-18T11:38:28.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:28.798+0000 7fb58598a640 1 -- 172.21.15.12:0/2685849679 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb580107420 con 0x7fb580101cb0 2024-02-18T11:38:28.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:28.799+0000 7fb584988640 1 -- 172.21.15.12:0/2685849679 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) v1 ==== 137245+0+0 (secure 0 0 0) 0x7fb578005020 con 0x7fb580101cb0 2024-02-18T11:38:28.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:28.800+0000 7fb584988640 1 --2- 172.21.15.12:0/2685849679 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fb5680a2750 0x7fb5680a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:28.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:28.800+0000 7fb57effd640 1 --2- 172.21.15.12:0/2685849679 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fb5680a2750 0x7fb5680a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:28.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:28.800+0000 7fb584988640 1 -- 172.21.15.12:0/2685849679 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb578014070 con 0x7fb580101cb0 2024-02-18T11:38:28.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:28.800+0000 7fb57effd640 1 --2- 172.21.15.12:0/2685849679 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fb5680a2750 0x7fb5680a4c10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb580101150 tx=0x7fb570005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:28.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:28.804+0000 7fb584988640 1 -- 172.21.15.12:0/2685849679 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb5780978d0 con 0x7fb580101cb0 2024-02-18T11:38:29.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:29.025+0000 7fb58598a640 1 -- 172.21.15.12:0/2685849679 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb58002cee0 con 0x7fb580101cb0 2024-02-18T11:38:29.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:29.025+0000 7fb584988640 1 -- 172.21.15.12:0/2685849679 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6083 (secure 0 0 0) 0x7fb57809af90 con 0x7fb580101cb0 2024-02-18T11:38:29.020 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:38:29.020 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1788335302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1788335302},{"type":"v1","addr":"172.21.15.12:6835","nonce":1788335302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/2144692455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":2144692455},{"type":"v1","addr":"172.21.15.12:6837","nonce":2144692455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1207202465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1207202465},{"type":"v1","addr":"172.21.15.188:6833","nonce":1207202465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6833/2636348323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6832","nonce":2636348323},{"type":"v1","addr":"172.21.15.158:6833","nonce":2636348323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24446,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6835/3673253642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":3673253642},{"type":"v1","addr":"172.21.15.158:6835","nonce":3673253642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/806709644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":806709644},{"type":"v1","addr":"172.21.15.188:6835","nonce":806709644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1036630143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1036630143},{"type":"v1","addr":"172.21.15.188:6837","nonce":1036630143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2024-02-18T11:38:29.020 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 8 2024-02-18T11:38:29.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:29.027+0000 7fb58598a640 1 -- 172.21.15.12:0/2685849679 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fb5680a2750 msgr2=0x7fb5680a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:29.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:29.027+0000 7fb58598a640 1 --2- 172.21.15.12:0/2685849679 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fb5680a2750 0x7fb5680a4c10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb580101150 tx=0x7fb570005f90 comp rx=0 tx=0).stop 2024-02-18T11:38:29.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:29.027+0000 7fb58598a640 1 -- 172.21.15.12:0/2685849679 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb580101cb0 msgr2=0x7fb58019e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:29.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:29.027+0000 7fb58598a640 1 --2- 172.21.15.12:0/2685849679 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb580101cb0 0x7fb58019e600 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb57800eb10 tx=0x7fb57800c2d0 comp rx=0 tx=0).stop 2024-02-18T11:38:29.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:29.027+0000 7fb58598a640 1 -- 172.21.15.12:0/2685849679 shutdown_connections 2024-02-18T11:38:29.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:29.027+0000 7fb58598a640 1 --2- 172.21.15.12:0/2685849679 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fb5680a2750 0x7fb5680a4c10 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:29.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:29.027+0000 7fb58598a640 1 --2- 172.21.15.12:0/2685849679 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5801012f0 0x7fb58019bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:29.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:29.027+0000 7fb58598a640 1 --2- 172.21.15.12:0/2685849679 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb580101cb0 0x7fb58019e600 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:29.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:29.027+0000 7fb58598a640 1 --2- 172.21.15.12:0/2685849679 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb580100100 0x7fb58006a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:29.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:29.027+0000 7fb58598a640 1 -- 172.21.15.12:0/2685849679 >> 172.21.15.12:0/2685849679 conn(0x7fb5800fb670 msgr2=0x7fb5801045f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:29.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:29.027+0000 7fb58598a640 1 -- 172.21.15.12:0/2685849679 shutdown_connections 2024-02-18T11:38:29.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:29.027+0000 7fb58598a640 1 -- 172.21.15.12:0/2685849679 wait complete. 2024-02-18T11:38:29.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:29 smithi188 ceph-mon[36925]: Deploying daemon mds.h on smithi158 2024-02-18T11:38:29.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:29 smithi188 ceph-mon[36925]: mds.? [v2:172.21.15.188:6836/1036630143,v1:172.21.15.188:6837/1036630143] up:boot 2024-02-18T11:38:29.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:29 smithi188 ceph-mon[36925]: fsmap 7 up:standby 2024-02-18T11:38:29.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:29 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-18T11:38:29.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:29 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/2685849679' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-18T11:38:29.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:29 smithi012 ceph-mon[32460]: Deploying daemon mds.h on smithi158 2024-02-18T11:38:29.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:29 smithi012 ceph-mon[32460]: mds.? [v2:172.21.15.188:6836/1036630143,v1:172.21.15.188:6837/1036630143] up:boot 2024-02-18T11:38:29.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:29 smithi012 ceph-mon[32460]: fsmap 7 up:standby 2024-02-18T11:38:29.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:29 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-18T11:38:29.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:29 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/2685849679' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-18T11:38:29.701 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:29 smithi158 ceph-mon[36626]: Deploying daemon mds.h on smithi158 2024-02-18T11:38:29.701 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:29 smithi158 ceph-mon[36626]: mds.? [v2:172.21.15.188:6836/1036630143,v1:172.21.15.188:6837/1036630143] up:boot 2024-02-18T11:38:29.702 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:29 smithi158 ceph-mon[36626]: fsmap 7 up:standby 2024-02-18T11:38:29.702 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:29 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-18T11:38:29.702 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:29 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/2685849679' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-18T11:38:30.327 INFO:journalctl@ceph.mds.h.smithi158.stdout:Feb 18 11:38:30 smithi158 systemd[1]: Starting Ceph mds.h for 0c8bb0de-ce51-11ee-95ba-87774f69a715... 2024-02-18T11:38:30.585 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:30 smithi158 ceph-mon[36626]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-18T11:38:30.586 INFO:journalctl@ceph.mds.h.smithi158.stdout:Feb 18 11:38:30 smithi158 podman[66500]: 2024-02-18 11:38:30.335158683 +0000 UTC m=+0.017935817 image pull 53c33ee3bf62dc411b650f86eedbbee7698efb1001b4c063271dcd8d65b41165 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1 2024-02-18T11:38:30.586 INFO:journalctl@ceph.mds.h.smithi158.stdout:Feb 18 11:38:30 smithi158 podman[66500]: 2024-02-18 11:38:30.452385769 +0000 UTC m=+0.135162925 container create 93295aab312378f4cd4e1fd17aab5d39b956922813f83a0f306a5cd5e69fff7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-h, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, org.label-schema.build-date=20240214, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True) 2024-02-18T11:38:30.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:30 smithi188 ceph-mon[36925]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-18T11:38:30.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:30 smithi012 ceph-mon[32460]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-18T11:38:30.994 INFO:journalctl@ceph.mds.h.smithi158.stdout:Feb 18 11:38:30 smithi158 podman[66500]: 2024-02-18 11:38:30.688114332 +0000 UTC m=+0.370891467 container init 93295aab312378f4cd4e1fd17aab5d39b956922813f83a0f306a5cd5e69fff7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux ) 2024-02-18T11:38:30.995 INFO:journalctl@ceph.mds.h.smithi158.stdout:Feb 18 11:38:30 smithi158 podman[66500]: 2024-02-18 11:38:30.690853633 +0000 UTC m=+0.373630756 container start 93295aab312378f4cd4e1fd17aab5d39b956922813f83a0f306a5cd5e69fff7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-h, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0) 2024-02-18T11:38:30.995 INFO:journalctl@ceph.mds.h.smithi158.stdout:Feb 18 11:38:30 smithi158 ceph-mds[66519]: -- 172.21.15.158:0/1183714868 <== mon.2 v2:172.21.15.158:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55dcee4c21a0 con 0x55dcee499000 2024-02-18T11:38:30.995 INFO:journalctl@ceph.mds.h.smithi158.stdout:Feb 18 11:38:30 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-h[66515]: starting mds.h at 2024-02-18T11:38:30.995 INFO:journalctl@ceph.mds.h.smithi158.stdout:Feb 18 11:38:30 smithi158 bash[66500]: 93295aab312378f4cd4e1fd17aab5d39b956922813f83a0f306a5cd5e69fff7a 2024-02-18T11:38:30.995 INFO:journalctl@ceph.mds.h.smithi158.stdout:Feb 18 11:38:30 smithi158 systemd[1]: Started Ceph mds.h for 0c8bb0de-ce51-11ee-95ba-87774f69a715. 2024-02-18T11:38:31.033 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-18T11:38:31.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:31.152+0000 7fbd21bb6640 1 -- 172.21.15.12:0/2618259491 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbd1c1011e0 msgr2=0x7fbd1c101600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:31.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:31.152+0000 7fbd21bb6640 1 --2- 172.21.15.12:0/2618259491 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbd1c1011e0 0x7fbd1c101600 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c009900 tx=0x7fbd0c02bc30 comp rx=0 tx=0).stop 2024-02-18T11:38:31.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:31.152+0000 7fbd21bb6640 1 -- 172.21.15.12:0/2618259491 shutdown_connections 2024-02-18T11:38:31.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:31.152+0000 7fbd21bb6640 1 --2- 172.21.15.12:0/2618259491 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd1c102d90 0x7fbd1c109620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:31.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:31.152+0000 7fbd21bb6640 1 --2- 172.21.15.12:0/2618259491 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbd1c1023d0 0x7fbd1c102850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:31.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:31.152+0000 7fbd21bb6640 1 --2- 172.21.15.12:0/2618259491 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbd1c1011e0 0x7fbd1c101600 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:31.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:31.152+0000 7fbd21bb6640 1 -- 172.21.15.12:0/2618259491 >> 172.21.15.12:0/2618259491 conn(0x7fbd1c0fc770 msgr2=0x7fbd1c0feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:31.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:31.152+0000 7fbd21bb6640 1 -- 172.21.15.12:0/2618259491 shutdown_connections 2024-02-18T11:38:31.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:31.152+0000 7fbd21bb6640 1 -- 172.21.15.12:0/2618259491 wait complete. 2024-02-18T11:38:31.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:31.152+0000 7fbd21bb6640 1 Processor -- start 2024-02-18T11:38:31.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:31.153+0000 7fbd21bb6640 1 -- start start 2024-02-18T11:38:31.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:31.153+0000 7fbd21bb6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbd1c1011e0 0x7fbd1c197270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:31.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:31.153+0000 7fbd21bb6640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd1c1023d0 0x7fbd1c1977b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:31.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:31.153+0000 7fbd21bb6640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbd1c102d90 0x7fbd1c19e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:31.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:31.153+0000 7fbd21bb6640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd1c19ed60 con 0x7fbd1c1011e0 2024-02-18T11:38:31.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:31.153+0000 7fbd21bb6640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd1c19eed0 con 0x7fbd1c102d90 2024-02-18T11:38:31.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:31.153+0000 7fbd21bb6640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd1c19f090 con 0x7fbd1c1023d0 2024-02-18T11:38:31.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:31.153+0000 7fbd1b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbd1c1011e0 0x7fbd1c197270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:31.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:31.153+0000 7fbd1b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbd1c1011e0 0x7fbd1c197270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54076/0 (socket says 172.21.15.12:54076) 2024-02-18T11:38:31.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:31.153+0000 7fbd1b7fe640 1 -- 172.21.15.12:0/1522102466 learned_addr learned my addr 172.21.15.12:0/1522102466 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:38:31.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:31.153+0000 7fbd1affd640 1 --2- 172.21.15.12:0/1522102466 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd1c1023d0 0x7fbd1c1977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:31.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:31.153+0000 7fbd1bfff640 1 --2- 172.21.15.12:0/1522102466 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbd1c102d90 0x7fbd1c19e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:31.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:31.153+0000 7fbd1b7fe640 1 -- 172.21.15.12:0/1522102466 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd1c1023d0 msgr2=0x7fbd1c1977b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:31.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:31.153+0000 7fbd1b7fe640 1 --2- 172.21.15.12:0/1522102466 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd1c1023d0 0x7fbd1c1977b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:31.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:31.153+0000 7fbd1b7fe640 1 -- 172.21.15.12:0/1522102466 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbd1c102d90 msgr2=0x7fbd1c19e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:31.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:31.153+0000 7fbd1b7fe640 1 --2- 172.21.15.12:0/1522102466 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbd1c102d90 0x7fbd1c19e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:31.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:31.153+0000 7fbd1b7fe640 1 -- 172.21.15.12:0/1522102466 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd0c009560 con 0x7fbd1c1011e0 2024-02-18T11:38:31.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:31.153+0000 7fbd1b7fe640 1 --2- 172.21.15.12:0/1522102466 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbd1c1011e0 0x7fbd1c197270 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c005ab0 tx=0x7fbd0c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:31.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:31.153+0000 7fbd18ff9640 1 -- 172.21.15.12:0/1522102466 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd0c004aa0 con 0x7fbd1c1011e0 2024-02-18T11:38:31.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:31.154+0000 7fbd21bb6640 1 -- 172.21.15.12:0/1522102466 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd1c19f310 con 0x7fbd1c1011e0 2024-02-18T11:38:31.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:31.154+0000 7fbd18ff9640 1 -- 172.21.15.12:0/1522102466 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbd0c0052d0 con 0x7fbd1c1011e0 2024-02-18T11:38:31.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:31.154+0000 7fbd18ff9640 1 -- 172.21.15.12:0/1522102466 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd0c003e80 con 0x7fbd1c1011e0 2024-02-18T11:38:31.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:31.154+0000 7fbd21bb6640 1 -- 172.21.15.12:0/1522102466 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd1c19f870 con 0x7fbd1c1011e0 2024-02-18T11:38:31.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:31.155+0000 7fbd21bb6640 1 -- 172.21.15.12:0/1522102466 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd1c108500 con 0x7fbd1c1011e0 2024-02-18T11:38:31.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:31.156+0000 7fbd18ff9640 1 -- 172.21.15.12:0/1522102466 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) v1 ==== 137245+0+0 (secure 0 0 0) 0x7fbd0c005440 con 0x7fbd1c1011e0 2024-02-18T11:38:31.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:31.156+0000 7fbd18ff9640 1 --2- 172.21.15.12:0/1522102466 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fbd040a2750 0x7fbd040a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:31.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:31.156+0000 7fbd1affd640 1 --2- 172.21.15.12:0/1522102466 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fbd040a2750 0x7fbd040a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:31.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:31.156+0000 7fbd18ff9640 1 -- 172.21.15.12:0/1522102466 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fbd0c0f04f0 con 0x7fbd1c1011e0 2024-02-18T11:38:31.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:31.157+0000 7fbd1affd640 1 --2- 172.21.15.12:0/1522102466 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fbd040a2750 0x7fbd040a4c10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c198780 tx=0x7fbd1000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:31.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:31.160+0000 7fbd18ff9640 1 -- 172.21.15.12:0/1522102466 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbd0c0b9720 con 0x7fbd1c1011e0 2024-02-18T11:38:31.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:31.381+0000 7fbd21bb6640 1 -- 172.21.15.12:0/1522102466 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbd1c107f70 con 0x7fbd1c1011e0 2024-02-18T11:38:31.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:31.382+0000 7fbd18ff9640 1 -- 172.21.15.12:0/1522102466 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6874 (secure 0 0 0) 0x7fbd0c0bcde0 con 0x7fbd1c1011e0 2024-02-18T11:38:31.376 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:38:31.377 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1788335302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1788335302},{"type":"v1","addr":"172.21.15.12:6835","nonce":1788335302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/2144692455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":2144692455},{"type":"v1","addr":"172.21.15.12:6837","nonce":2144692455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1207202465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1207202465},{"type":"v1","addr":"172.21.15.188:6833","nonce":1207202465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6833/2636348323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6832","nonce":2636348323},{"type":"v1","addr":"172.21.15.158:6833","nonce":2636348323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24446,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6835/3673253642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":3673253642},{"type":"v1","addr":"172.21.15.158:6835","nonce":3673253642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/806709644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":806709644},{"type":"v1","addr":"172.21.15.188:6835","nonce":806709644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24476,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6837/813251825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6836","nonce":813251825},{"type":"v1","addr":"172.21.15.158:6837","nonce":813251825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1036630143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1036630143},{"type":"v1","addr":"172.21.15.188:6837","nonce":1036630143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2024-02-18T11:38:31.377 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 9 2024-02-18T11:38:31.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:31.384+0000 7fbd21bb6640 1 -- 172.21.15.12:0/1522102466 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fbd040a2750 msgr2=0x7fbd040a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:31.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:31.384+0000 7fbd21bb6640 1 --2- 172.21.15.12:0/1522102466 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fbd040a2750 0x7fbd040a4c10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c198780 tx=0x7fbd1000b410 comp rx=0 tx=0).stop 2024-02-18T11:38:31.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:31.384+0000 7fbd21bb6640 1 -- 172.21.15.12:0/1522102466 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbd1c1011e0 msgr2=0x7fbd1c197270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:31.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:31.384+0000 7fbd21bb6640 1 --2- 172.21.15.12:0/1522102466 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbd1c1011e0 0x7fbd1c197270 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c005ab0 tx=0x7fbd0c004270 comp rx=0 tx=0).stop 2024-02-18T11:38:31.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:31.384+0000 7fbd21bb6640 1 -- 172.21.15.12:0/1522102466 shutdown_connections 2024-02-18T11:38:31.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:31.384+0000 7fbd21bb6640 1 --2- 172.21.15.12:0/1522102466 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fbd040a2750 0x7fbd040a4c10 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:31.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:31.384+0000 7fbd21bb6640 1 --2- 172.21.15.12:0/1522102466 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd1c1023d0 0x7fbd1c1977b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:31.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:31.384+0000 7fbd21bb6640 1 --2- 172.21.15.12:0/1522102466 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbd1c102d90 0x7fbd1c19e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:31.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:31.384+0000 7fbd21bb6640 1 --2- 172.21.15.12:0/1522102466 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbd1c1011e0 0x7fbd1c197270 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:31.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:31.384+0000 7fbd21bb6640 1 -- 172.21.15.12:0/1522102466 >> 172.21.15.12:0/1522102466 conn(0x7fbd1c0fc770 msgr2=0x7fbd1c1056d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:31.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:31.384+0000 7fbd21bb6640 1 -- 172.21.15.12:0/1522102466 shutdown_connections 2024-02-18T11:38:31.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:31.384+0000 7fbd21bb6640 1 -- 172.21.15.12:0/1522102466 wait complete. 2024-02-18T11:38:32.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:31 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:32.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:31 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:32.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:31 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:32.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:31 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-18T11:38:32.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:31 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-18T11:38:32.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:31 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:38:32.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:31 smithi188 ceph-mon[36925]: Deploying daemon mds.i on smithi188 2024-02-18T11:38:32.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:31 smithi188 ceph-mon[36925]: mds.? [v2:172.21.15.158:6836/813251825,v1:172.21.15.158:6837/813251825] up:boot 2024-02-18T11:38:32.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:31 smithi188 ceph-mon[36925]: fsmap 8 up:standby 2024-02-18T11:38:32.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:31 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-18T11:38:32.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:31 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/1522102466' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-18T11:38:32.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:31 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:32.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:31 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:32.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:31 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:32.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:31 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-18T11:38:32.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:31 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-18T11:38:32.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:31 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:38:32.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:31 smithi012 ceph-mon[32460]: Deploying daemon mds.i on smithi188 2024-02-18T11:38:32.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:31 smithi012 ceph-mon[32460]: mds.? [v2:172.21.15.158:6836/813251825,v1:172.21.15.158:6837/813251825] up:boot 2024-02-18T11:38:32.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:31 smithi012 ceph-mon[32460]: fsmap 8 up:standby 2024-02-18T11:38:32.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:31 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-18T11:38:32.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:31 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/1522102466' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-18T11:38:32.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:31 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:32.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:31 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:32.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:31 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:32.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:31 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-18T11:38:32.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:31 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-18T11:38:32.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:31 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:38:32.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:31 smithi158 ceph-mon[36626]: Deploying daemon mds.i on smithi188 2024-02-18T11:38:32.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:31 smithi158 ceph-mon[36626]: mds.? [v2:172.21.15.158:6836/813251825,v1:172.21.15.158:6837/813251825] up:boot 2024-02-18T11:38:32.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:31 smithi158 ceph-mon[36626]: fsmap 8 up:standby 2024-02-18T11:38:32.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:31 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-18T11:38:32.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:31 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/1522102466' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-18T11:38:33.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:32 smithi188 ceph-mon[36925]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-18T11:38:33.105 INFO:journalctl@ceph.mds.i.smithi188.stdout:Feb 18 11:38:32 smithi188 systemd[1]: Starting Ceph mds.i for 0c8bb0de-ce51-11ee-95ba-87774f69a715... 2024-02-18T11:38:33.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:32 smithi012 ceph-mon[32460]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-18T11:38:33.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:32 smithi158 ceph-mon[36626]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-18T11:38:33.390 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-18T11:38:33.417 INFO:journalctl@ceph.mds.i.smithi188.stdout:Feb 18 11:38:33 smithi188 podman[67887]: 2024-02-18 11:38:33.064817126 +0000 UTC m=+0.017598081 image pull 53c33ee3bf62dc411b650f86eedbbee7698efb1001b4c063271dcd8d65b41165 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1 2024-02-18T11:38:33.418 INFO:journalctl@ceph.mds.i.smithi188.stdout:Feb 18 11:38:33 smithi188 podman[67887]: 2024-02-18 11:38:33.266171595 +0000 UTC m=+0.218952587 container create 7a8b1bd187ae09241dfe967ccff872543710cee7044ef24d2f84292ce2c8f654 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-i, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214) 2024-02-18T11:38:33.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:33.511+0000 7f5ffbe64640 1 -- 172.21.15.12:0/3558772752 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ff4100dd0 msgr2=0x7f5ff4101250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:33.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:33.511+0000 7f5ffbe64640 1 --2- 172.21.15.12:0/3558772752 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ff4100dd0 0x7f5ff4101250 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5fec009be0 tx=0x7f5fec02bef0 comp rx=0 tx=0).stop 2024-02-18T11:38:33.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:33.511+0000 7f5ffbe64640 1 -- 172.21.15.12:0/3558772752 shutdown_connections 2024-02-18T11:38:33.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:33.511+0000 7f5ffbe64640 1 --2- 172.21.15.12:0/3558772752 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ff4100dd0 0x7f5ff4101250 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:33.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:33.511+0000 7f5ffbe64640 1 --2- 172.21.15.12:0/3558772752 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5ff4100430 0x7f5ff4100890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:33.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:33.511+0000 7f5ffbe64640 1 --2- 172.21.15.12:0/3558772752 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ff40ffad0 0x7f5ff40ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:33.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:33.511+0000 7f5ffbe64640 1 -- 172.21.15.12:0/3558772752 >> 172.21.15.12:0/3558772752 conn(0x7f5ff40fb6f0 msgr2=0x7f5ff40fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:33.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:33.511+0000 7f5ffbe64640 1 -- 172.21.15.12:0/3558772752 shutdown_connections 2024-02-18T11:38:33.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:33.511+0000 7f5ffbe64640 1 -- 172.21.15.12:0/3558772752 wait complete. 2024-02-18T11:38:33.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:33.512+0000 7f5ffbe64640 1 Processor -- start 2024-02-18T11:38:33.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:33.512+0000 7f5ffbe64640 1 -- start start 2024-02-18T11:38:33.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:33.512+0000 7f5ffbe64640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ff40ffad0 0x7f5ff41993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:33.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:33.512+0000 7f5ffbe64640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5ff4100430 0x7f5ff4199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:33.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:33.512+0000 7f5ffbe64640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ff4100dd0 0x7f5ff41a0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:33.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:33.512+0000 7f5ffbe64640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff41a0ed0 con 0x7f5ff40ffad0 2024-02-18T11:38:33.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:33.512+0000 7f5ffbe64640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff41a1040 con 0x7f5ff4100430 2024-02-18T11:38:33.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:33.512+0000 7f5ffbe64640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff41a1200 con 0x7f5ff4100dd0 2024-02-18T11:38:33.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:33.512+0000 7f5ff9c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ff40ffad0 0x7f5ff41993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:33.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:33.512+0000 7f5ff9c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ff40ffad0 0x7f5ff41993e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42420/0 (socket says 172.21.15.12:42420) 2024-02-18T11:38:33.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:33.512+0000 7f5ff9c00640 1 -- 172.21.15.12:0/293060689 learned_addr learned my addr 172.21.15.12:0/293060689 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:38:33.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:33.512+0000 7f5ffa401640 1 --2- 172.21.15.12:0/293060689 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ff4100dd0 0x7f5ff41a0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:33.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:33.512+0000 7f5ff93ff640 1 --2- 172.21.15.12:0/293060689 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5ff4100430 0x7f5ff4199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:33.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:33.512+0000 7f5ff9c00640 1 -- 172.21.15.12:0/293060689 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ff4100dd0 msgr2=0x7f5ff41a0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:33.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:33.512+0000 7f5ff9c00640 1 --2- 172.21.15.12:0/293060689 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ff4100dd0 0x7f5ff41a0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:33.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:33.512+0000 7f5ff9c00640 1 -- 172.21.15.12:0/293060689 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5ff4100430 msgr2=0x7f5ff4199920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:33.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:33.512+0000 7f5ff9c00640 1 --2- 172.21.15.12:0/293060689 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5ff4100430 0x7f5ff4199920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:33.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:33.512+0000 7f5ff9c00640 1 -- 172.21.15.12:0/293060689 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fec009840 con 0x7f5ff40ffad0 2024-02-18T11:38:33.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:33.512+0000 7f5ff93ff640 1 --2- 172.21.15.12:0/293060689 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5ff4100430 0x7f5ff4199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:38:33.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:33.513+0000 7f5ff9c00640 1 --2- 172.21.15.12:0/293060689 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ff40ffad0 0x7f5ff41993e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5ff000e9c0 tx=0x7f5ff000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:33.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:33.513+0000 7f5feaffd640 1 -- 172.21.15.12:0/293060689 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff000ccc0 con 0x7f5ff40ffad0 2024-02-18T11:38:33.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:33.513+0000 7f5feaffd640 1 -- 172.21.15.12:0/293060689 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5ff000ce20 con 0x7f5ff40ffad0 2024-02-18T11:38:33.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:33.513+0000 7f5feaffd640 1 -- 172.21.15.12:0/293060689 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff0010430 con 0x7f5ff40ffad0 2024-02-18T11:38:33.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:33.513+0000 7f5ffbe64640 1 -- 172.21.15.12:0/293060689 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ff41a14e0 con 0x7f5ff40ffad0 2024-02-18T11:38:33.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:33.513+0000 7f5ffbe64640 1 -- 172.21.15.12:0/293060689 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ff41a19f0 con 0x7f5ff40ffad0 2024-02-18T11:38:33.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:33.514+0000 7f5ffbe64640 1 -- 172.21.15.12:0/293060689 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ff410a820 con 0x7f5ff40ffad0 2024-02-18T11:38:33.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:33.515+0000 7f5feaffd640 1 -- 172.21.15.12:0/293060689 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) v1 ==== 137245+0+0 (secure 0 0 0) 0x7f5ff0004f80 con 0x7f5ff40ffad0 2024-02-18T11:38:33.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:33.516+0000 7f5feaffd640 1 --2- 172.21.15.12:0/293060689 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f5fe00a2750 0x7f5fe00a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:33.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:33.516+0000 7f5ff93ff640 1 --2- 172.21.15.12:0/293060689 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f5fe00a2750 0x7f5fe00a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:33.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:33.516+0000 7f5feaffd640 1 -- 172.21.15.12:0/293060689 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f5ff0014070 con 0x7f5ff40ffad0 2024-02-18T11:38:33.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:33.516+0000 7f5ff93ff640 1 --2- 172.21.15.12:0/293060689 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f5fe00a2750 0x7f5fe00a4c10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5ff419a8f0 tx=0x7f5fe4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:33.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:33.520+0000 7f5feaffd640 1 -- 172.21.15.12:0/293060689 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5ff0097670 con 0x7f5ff40ffad0 2024-02-18T11:38:33.678 INFO:journalctl@ceph.mds.i.smithi188.stdout:Feb 18 11:38:33 smithi188 podman[67887]: 2024-02-18 11:38:33.566187005 +0000 UTC m=+0.518967972 container init 7a8b1bd187ae09241dfe967ccff872543710cee7044ef24d2f84292ce2c8f654 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-i, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214) 2024-02-18T11:38:33.678 INFO:journalctl@ceph.mds.i.smithi188.stdout:Feb 18 11:38:33 smithi188 podman[67887]: 2024-02-18 11:38:33.56983266 +0000 UTC m=+0.522613604 container start 7a8b1bd187ae09241dfe967ccff872543710cee7044ef24d2f84292ce2c8f654 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-i, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0) 2024-02-18T11:38:33.679 INFO:journalctl@ceph.mds.i.smithi188.stdout:Feb 18 11:38:33 smithi188 ceph-mds[67907]: -- 172.21.15.188:0/338813825 <== mon.1 v2:172.21.15.188:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d50a9aa1a0 con 0x55d50a981000 2024-02-18T11:38:33.679 INFO:journalctl@ceph.mds.i.smithi188.stdout:Feb 18 11:38:33 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-i[67902]: starting mds.i at 2024-02-18T11:38:33.679 INFO:journalctl@ceph.mds.i.smithi188.stdout:Feb 18 11:38:33 smithi188 bash[67887]: 7a8b1bd187ae09241dfe967ccff872543710cee7044ef24d2f84292ce2c8f654 2024-02-18T11:38:33.679 INFO:journalctl@ceph.mds.i.smithi188.stdout:Feb 18 11:38:33 smithi188 systemd[1]: Started Ceph mds.i for 0c8bb0de-ce51-11ee-95ba-87774f69a715. 2024-02-18T11:38:33.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:33.748+0000 7f5ffbe64640 1 -- 172.21.15.12:0/293060689 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5ff410ee30 con 0x7f5ff40ffad0 2024-02-18T11:38:33.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:33.748+0000 7f5feaffd640 1 -- 172.21.15.12:0/293060689 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6874 (secure 0 0 0) 0x7f5ff009ad30 con 0x7f5ff40ffad0 2024-02-18T11:38:33.743 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:38:33.743 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1788335302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1788335302},{"type":"v1","addr":"172.21.15.12:6835","nonce":1788335302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/2144692455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":2144692455},{"type":"v1","addr":"172.21.15.12:6837","nonce":2144692455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1207202465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1207202465},{"type":"v1","addr":"172.21.15.188:6833","nonce":1207202465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6833/2636348323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6832","nonce":2636348323},{"type":"v1","addr":"172.21.15.158:6833","nonce":2636348323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24446,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6835/3673253642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":3673253642},{"type":"v1","addr":"172.21.15.158:6835","nonce":3673253642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/806709644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":806709644},{"type":"v1","addr":"172.21.15.188:6835","nonce":806709644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24476,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6837/813251825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6836","nonce":813251825},{"type":"v1","addr":"172.21.15.158:6837","nonce":813251825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1036630143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1036630143},{"type":"v1","addr":"172.21.15.188:6837","nonce":1036630143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2024-02-18T11:38:33.743 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 9 2024-02-18T11:38:33.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:33.751+0000 7f5ffbe64640 1 -- 172.21.15.12:0/293060689 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f5fe00a2750 msgr2=0x7f5fe00a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:33.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:33.751+0000 7f5ffbe64640 1 --2- 172.21.15.12:0/293060689 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f5fe00a2750 0x7f5fe00a4c10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5ff419a8f0 tx=0x7f5fe4005f90 comp rx=0 tx=0).stop 2024-02-18T11:38:33.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:33.751+0000 7f5ffbe64640 1 -- 172.21.15.12:0/293060689 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ff40ffad0 msgr2=0x7f5ff41993e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:33.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:33.751+0000 7f5ffbe64640 1 --2- 172.21.15.12:0/293060689 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ff40ffad0 0x7f5ff41993e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5ff000e9c0 tx=0x7f5ff000ee80 comp rx=0 tx=0).stop 2024-02-18T11:38:33.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:33.751+0000 7f5ffbe64640 1 -- 172.21.15.12:0/293060689 shutdown_connections 2024-02-18T11:38:33.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:33.751+0000 7f5ffbe64640 1 --2- 172.21.15.12:0/293060689 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f5fe00a2750 0x7f5fe00a4c10 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:33.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:33.751+0000 7f5ffbe64640 1 --2- 172.21.15.12:0/293060689 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ff4100dd0 0x7f5ff41a0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:33.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:33.751+0000 7f5ffbe64640 1 --2- 172.21.15.12:0/293060689 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5ff4100430 0x7f5ff4199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:33.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:33.751+0000 7f5ffbe64640 1 --2- 172.21.15.12:0/293060689 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ff40ffad0 0x7f5ff41993e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:33.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:33.751+0000 7f5ffbe64640 1 -- 172.21.15.12:0/293060689 >> 172.21.15.12:0/293060689 conn(0x7f5ff40fb6f0 msgr2=0x7f5ff40fd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:33.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:33.751+0000 7f5ffbe64640 1 -- 172.21.15.12:0/293060689 shutdown_connections 2024-02-18T11:38:33.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:33.751+0000 7f5ffbe64640 1 -- 172.21.15.12:0/293060689 wait complete. 2024-02-18T11:38:34.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:34 smithi188 ceph-mon[36925]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-18T11:38:34.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:34 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:34.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:34 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:34.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:34 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:34.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:34 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-18T11:38:34.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:34 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-18T11:38:34.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:34 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:38:34.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:34 smithi188 ceph-mon[36925]: Deploying daemon mds.g on smithi012 2024-02-18T11:38:34.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:34 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/293060689' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-18T11:38:34.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:34 smithi188 ceph-mon[36925]: mds.? [v2:172.21.15.188:6838/1342671190,v1:172.21.15.188:6839/1342671190] up:boot 2024-02-18T11:38:34.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:34 smithi188 ceph-mon[36925]: fsmap 9 up:standby 2024-02-18T11:38:34.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:34 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-18T11:38:34.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:34 smithi158 ceph-mon[36626]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-18T11:38:34.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:34 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:34.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:34 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:34.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:34 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:34.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:34 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-18T11:38:34.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:34 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-18T11:38:34.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:34 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:38:34.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:34 smithi158 ceph-mon[36626]: Deploying daemon mds.g on smithi012 2024-02-18T11:38:34.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:34 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/293060689' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-18T11:38:34.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:34 smithi158 ceph-mon[36626]: mds.? [v2:172.21.15.188:6838/1342671190,v1:172.21.15.188:6839/1342671190] up:boot 2024-02-18T11:38:34.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:34 smithi158 ceph-mon[36626]: fsmap 9 up:standby 2024-02-18T11:38:34.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:34 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-18T11:38:35.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:34 smithi012 ceph-mon[32460]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-18T11:38:35.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:34 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:35.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:34 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:35.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:34 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:35.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:34 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-18T11:38:35.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:34 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-18T11:38:35.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:34 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:38:35.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:34 smithi012 ceph-mon[32460]: Deploying daemon mds.g on smithi012 2024-02-18T11:38:35.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:34 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/293060689' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-18T11:38:35.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:34 smithi012 ceph-mon[32460]: mds.? [v2:172.21.15.188:6838/1342671190,v1:172.21.15.188:6839/1342671190] up:boot 2024-02-18T11:38:35.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:34 smithi012 ceph-mon[32460]: fsmap 9 up:standby 2024-02-18T11:38:35.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:34 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-18T11:38:35.757 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-18T11:38:35.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:35.876+0000 7f4ab0b55640 1 -- 172.21.15.12:0/3925814961 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4aac1000e0 msgr2=0x7f4aac100500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:35.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:35.876+0000 7f4ab0b55640 1 --2- 172.21.15.12:0/3925814961 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4aac1000e0 0x7f4aac100500 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c009900 tx=0x7f4a9c02bc30 comp rx=0 tx=0).stop 2024-02-18T11:38:35.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:35.876+0000 7f4ab0b55640 1 -- 172.21.15.12:0/3925814961 shutdown_connections 2024-02-18T11:38:35.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:35.876+0000 7f4ab0b55640 1 --2- 172.21.15.12:0/3925814961 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4aac101c90 0x7f4aac108520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:35.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:35.876+0000 7f4ab0b55640 1 --2- 172.21.15.12:0/3925814961 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4aac1012d0 0x7f4aac101750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:35.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:35.876+0000 7f4ab0b55640 1 --2- 172.21.15.12:0/3925814961 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4aac1000e0 0x7f4aac100500 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:35.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:35.876+0000 7f4ab0b55640 1 -- 172.21.15.12:0/3925814961 >> 172.21.15.12:0/3925814961 conn(0x7f4aac0fb650 msgr2=0x7f4aac0fda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:35.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:35.876+0000 7f4ab0b55640 1 -- 172.21.15.12:0/3925814961 shutdown_connections 2024-02-18T11:38:35.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:35.876+0000 7f4ab0b55640 1 -- 172.21.15.12:0/3925814961 wait complete. 2024-02-18T11:38:35.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:35.876+0000 7f4ab0b55640 1 Processor -- start 2024-02-18T11:38:35.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:35.876+0000 7f4ab0b55640 1 -- start start 2024-02-18T11:38:35.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:35.877+0000 7f4ab0b55640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4aac1000e0 0x7f4aac1972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:35.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:35.877+0000 7f4ab0b55640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4aac1012d0 0x7f4aac197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:35.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:35.877+0000 7f4ab0b55640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4aac101c90 0x7f4aac19e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:35.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:35.877+0000 7f4ab0b55640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aac19edb0 con 0x7f4aac1012d0 2024-02-18T11:38:35.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:35.877+0000 7f4ab0b55640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aac19ef20 con 0x7f4aac101c90 2024-02-18T11:38:35.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:35.877+0000 7f4ab0b55640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aac19f0e0 con 0x7f4aac1000e0 2024-02-18T11:38:35.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:35.877+0000 7f4aa3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4aac1012d0 0x7f4aac197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:35.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:35.877+0000 7f4aa3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4aac1012d0 0x7f4aac197800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42436/0 (socket says 172.21.15.12:42436) 2024-02-18T11:38:35.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:35.877+0000 7f4aa3fff640 1 -- 172.21.15.12:0/467444568 learned_addr learned my addr 172.21.15.12:0/467444568 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:38:35.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:35.877+0000 7f4aaa59c640 1 --2- 172.21.15.12:0/467444568 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4aac1000e0 0x7f4aac1972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:35.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:35.877+0000 7f4aaad9d640 1 --2- 172.21.15.12:0/467444568 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4aac101c90 0x7f4aac19e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:35.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:35.877+0000 7f4aa3fff640 1 -- 172.21.15.12:0/467444568 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4aac1000e0 msgr2=0x7f4aac1972c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:35.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:35.877+0000 7f4aa3fff640 1 --2- 172.21.15.12:0/467444568 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4aac1000e0 0x7f4aac1972c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:35.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:35.877+0000 7f4aa3fff640 1 -- 172.21.15.12:0/467444568 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4aac101c90 msgr2=0x7f4aac19e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:35.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:35.877+0000 7f4aa3fff640 1 --2- 172.21.15.12:0/467444568 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4aac101c90 0x7f4aac19e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:35.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:35.877+0000 7f4aa3fff640 1 -- 172.21.15.12:0/467444568 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a9c009560 con 0x7f4aac1012d0 2024-02-18T11:38:35.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:35.877+0000 7f4aa3fff640 1 --2- 172.21.15.12:0/467444568 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4aac1012d0 0x7f4aac197800 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4a9400c890 tx=0x7f4a9400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:35.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:35.877+0000 7f4aa37fe640 1 -- 172.21.15.12:0/467444568 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a94004220 con 0x7f4aac1012d0 2024-02-18T11:38:35.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:35.877+0000 7f4aa37fe640 1 -- 172.21.15.12:0/467444568 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4a94007e90 con 0x7f4aac1012d0 2024-02-18T11:38:35.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:35.877+0000 7f4ab0b55640 1 -- 172.21.15.12:0/467444568 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4aac19f3c0 con 0x7f4aac1012d0 2024-02-18T11:38:35.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:35.877+0000 7f4aa37fe640 1 -- 172.21.15.12:0/467444568 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a94004a00 con 0x7f4aac1012d0 2024-02-18T11:38:35.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:35.877+0000 7f4ab0b55640 1 -- 172.21.15.12:0/467444568 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4aac19f8d0 con 0x7f4aac1012d0 2024-02-18T11:38:35.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:35.878+0000 7f4ab0b55640 1 -- 172.21.15.12:0/467444568 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4aac10baf0 con 0x7f4aac1012d0 2024-02-18T11:38:35.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:35.880+0000 7f4aa37fe640 1 -- 172.21.15.12:0/467444568 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) v1 ==== 137245+0+0 (secure 0 0 0) 0x7f4a940079b0 con 0x7f4aac1012d0 2024-02-18T11:38:35.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:35.880+0000 7f4aa37fe640 1 --2- 172.21.15.12:0/467444568 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f4a980a2750 0x7f4a980a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:35.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:35.880+0000 7f4aaa59c640 1 --2- 172.21.15.12:0/467444568 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f4a980a2750 0x7f4a980a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:35.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:35.880+0000 7f4aa37fe640 1 -- 172.21.15.12:0/467444568 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4a940ce160 con 0x7f4aac1012d0 2024-02-18T11:38:35.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:35.881+0000 7f4aaa59c640 1 --2- 172.21.15.12:0/467444568 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f4a980a2750 0x7f4a980a4c10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c005ab0 tx=0x7f4a9c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:35.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:35.888+0000 7f4aa37fe640 1 -- 172.21.15.12:0/467444568 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4a94097390 con 0x7f4aac1012d0 2024-02-18T11:38:36.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:36.115+0000 7f4ab0b55640 1 -- 172.21.15.12:0/467444568 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4aac10bc80 con 0x7f4aac1012d0 2024-02-18T11:38:36.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:36.116+0000 7f4aa37fe640 1 -- 172.21.15.12:0/467444568 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7670 (secure 0 0 0) 0x7f4a9409aa50 con 0x7f4aac1012d0 2024-02-18T11:38:36.110 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:38:36.110 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1788335302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1788335302},{"type":"v1","addr":"172.21.15.12:6835","nonce":1788335302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/2144692455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":2144692455},{"type":"v1","addr":"172.21.15.12:6837","nonce":2144692455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1207202465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1207202465},{"type":"v1","addr":"172.21.15.188:6833","nonce":1207202465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6833/2636348323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6832","nonce":2636348323},{"type":"v1","addr":"172.21.15.158:6833","nonce":2636348323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24446,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6835/3673253642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":3673253642},{"type":"v1","addr":"172.21.15.158:6835","nonce":3673253642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/806709644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":806709644},{"type":"v1","addr":"172.21.15.188:6835","nonce":806709644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24476,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6837/813251825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6836","nonce":813251825},{"type":"v1","addr":"172.21.15.158:6837","nonce":813251825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1036630143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1036630143},{"type":"v1","addr":"172.21.15.188:6837","nonce":1036630143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24508,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6839/1342671190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":1342671190},{"type":"v1","addr":"172.21.15.188:6839","nonce":1342671190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-02-18T11:38:36.110 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 10 2024-02-18T11:38:36.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:36.118+0000 7f4ab0b55640 1 -- 172.21.15.12:0/467444568 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f4a980a2750 msgr2=0x7f4a980a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:36.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:36.118+0000 7f4ab0b55640 1 --2- 172.21.15.12:0/467444568 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f4a980a2750 0x7f4a980a4c10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c005ab0 tx=0x7f4a9c005fd0 comp rx=0 tx=0).stop 2024-02-18T11:38:36.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:36.118+0000 7f4ab0b55640 1 -- 172.21.15.12:0/467444568 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4aac1012d0 msgr2=0x7f4aac197800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:36.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:36.118+0000 7f4ab0b55640 1 --2- 172.21.15.12:0/467444568 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4aac1012d0 0x7f4aac197800 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4a9400c890 tx=0x7f4a9400cd50 comp rx=0 tx=0).stop 2024-02-18T11:38:36.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:36.118+0000 7f4ab0b55640 1 -- 172.21.15.12:0/467444568 shutdown_connections 2024-02-18T11:38:36.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:36.118+0000 7f4ab0b55640 1 --2- 172.21.15.12:0/467444568 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f4a980a2750 0x7f4a980a4c10 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:36.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:36.118+0000 7f4ab0b55640 1 --2- 172.21.15.12:0/467444568 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4aac101c90 0x7f4aac19e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:36.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:36.118+0000 7f4ab0b55640 1 --2- 172.21.15.12:0/467444568 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4aac1012d0 0x7f4aac197800 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:36.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:36.118+0000 7f4ab0b55640 1 --2- 172.21.15.12:0/467444568 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4aac1000e0 0x7f4aac1972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:36.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:36.118+0000 7f4ab0b55640 1 -- 172.21.15.12:0/467444568 >> 172.21.15.12:0/467444568 conn(0x7f4aac0fb650 msgr2=0x7f4aac0fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:36.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:36.118+0000 7f4ab0b55640 1 -- 172.21.15.12:0/467444568 shutdown_connections 2024-02-18T11:38:36.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:36.118+0000 7f4ab0b55640 1 -- 172.21.15.12:0/467444568 wait complete. 2024-02-18T11:38:36.124 INFO:journalctl@ceph.mds.g.smithi012.stdout:Feb 18 11:38:35 smithi012 systemd[1]: Starting Ceph mds.g for 0c8bb0de-ce51-11ee-95ba-87774f69a715... 2024-02-18T11:38:36.422 INFO:journalctl@ceph.mds.g.smithi012.stdout:Feb 18 11:38:36 smithi012 podman[72178]: 2024-02-18 11:38:36.118056499 +0000 UTC m=+0.017853653 image pull 53c33ee3bf62dc411b650f86eedbbee7698efb1001b4c063271dcd8d65b41165 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1 2024-02-18T11:38:36.422 INFO:journalctl@ceph.mds.g.smithi012.stdout:Feb 18 11:38:36 smithi012 podman[72178]: 2024-02-18 11:38:36.226119431 +0000 UTC m=+0.125916576 container create 2d26a2be1b71391324e8748deec8596b86578bf837ffc8236e894069a4af03b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-g, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-02-18T11:38:36.422 INFO:journalctl@ceph.mds.g.smithi012.stdout:Feb 18 11:38:36 smithi012 podman[72178]: 2024-02-18 11:38:36.428662435 +0000 UTC m=+0.328459581 container init 2d26a2be1b71391324e8748deec8596b86578bf837ffc8236e894069a4af03b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-g, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, io.buildah.version=1.33.5) 2024-02-18T11:38:36.673 INFO:journalctl@ceph.mds.g.smithi012.stdout:Feb 18 11:38:36 smithi012 podman[72178]: 2024-02-18 11:38:36.431536197 +0000 UTC m=+0.331333341 container start 2d26a2be1b71391324e8748deec8596b86578bf837ffc8236e894069a4af03b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-g, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-18T11:38:36.673 INFO:journalctl@ceph.mds.g.smithi012.stdout:Feb 18 11:38:36 smithi012 ceph-mds[72199]: -- 172.21.15.12:0/194279986 <== mon.0 v2:172.21.15.12:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5644cd44a1a0 con 0x5644cd420c00 2024-02-18T11:38:36.673 INFO:journalctl@ceph.mds.g.smithi012.stdout:Feb 18 11:38:36 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-g[72195]: starting mds.g at 2024-02-18T11:38:36.673 INFO:journalctl@ceph.mds.g.smithi012.stdout:Feb 18 11:38:36 smithi012 bash[72178]: 2d26a2be1b71391324e8748deec8596b86578bf837ffc8236e894069a4af03b2 2024-02-18T11:38:36.673 INFO:journalctl@ceph.mds.g.smithi012.stdout:Feb 18 11:38:36 smithi012 systemd[1]: Started Ceph mds.g for 0c8bb0de-ce51-11ee-95ba-87774f69a715. 2024-02-18T11:38:36.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:36 smithi158 ceph-mon[36626]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-18T11:38:36.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:36 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/467444568' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-18T11:38:36.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:36 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:36.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:36 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:36.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:36 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:36.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:36 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-18T11:38:36.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:36 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-18T11:38:36.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:36 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:38:37.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:36 smithi188 ceph-mon[36925]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-18T11:38:37.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:36 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/467444568' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-18T11:38:37.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:36 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:37.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:36 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:37.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:36 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:37.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:36 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-18T11:38:37.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:36 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-18T11:38:37.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:36 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:38:37.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:36 smithi012 ceph-mon[32460]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-18T11:38:37.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:36 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/467444568' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-18T11:38:37.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:36 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:37.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:36 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:37.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:36 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:37.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:36 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-18T11:38:37.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:36 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-18T11:38:37.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:36 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:38:37.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:37 smithi012 ceph-mon[32460]: Deploying daemon mds.a on smithi012 2024-02-18T11:38:37.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:37 smithi012 ceph-mon[32460]: mds.? [v2:172.21.15.12:6838/3569056036,v1:172.21.15.12:6839/3569056036] up:boot 2024-02-18T11:38:37.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:37 smithi012 ceph-mon[32460]: fsmap 10 up:standby 2024-02-18T11:38:37.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:37 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-02-18T11:38:37.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:37 smithi158 ceph-mon[36626]: Deploying daemon mds.a on smithi012 2024-02-18T11:38:37.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:37 smithi158 ceph-mon[36626]: mds.? [v2:172.21.15.12:6838/3569056036,v1:172.21.15.12:6839/3569056036] up:boot 2024-02-18T11:38:37.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:37 smithi158 ceph-mon[36626]: fsmap 10 up:standby 2024-02-18T11:38:37.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:37 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-02-18T11:38:38.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:37 smithi188 ceph-mon[36925]: Deploying daemon mds.a on smithi012 2024-02-18T11:38:38.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:37 smithi188 ceph-mon[36925]: mds.? [v2:172.21.15.12:6838/3569056036,v1:172.21.15.12:6839/3569056036] up:boot 2024-02-18T11:38:38.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:37 smithi188 ceph-mon[36925]: fsmap 10 up:standby 2024-02-18T11:38:38.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:37 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-02-18T11:38:38.125 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-18T11:38:38.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:38.238+0000 7fd2768d4640 1 -- 172.21.15.12:0/2153281481 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2701000e0 msgr2=0x7fd270100500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:38.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:38.238+0000 7fd2768d4640 1 --2- 172.21.15.12:0/2153281481 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2701000e0 0x7fd270100500 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd260009900 tx=0x7fd26002bc30 comp rx=0 tx=0).stop 2024-02-18T11:38:38.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:38.238+0000 7fd2768d4640 1 -- 172.21.15.12:0/2153281481 shutdown_connections 2024-02-18T11:38:38.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:38.238+0000 7fd2768d4640 1 --2- 172.21.15.12:0/2153281481 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2701012d0 0x7fd270101750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:38.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:38.238+0000 7fd2768d4640 1 --2- 172.21.15.12:0/2153281481 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd270101c90 0x7fd270108520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:38.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:38.238+0000 7fd2768d4640 1 --2- 172.21.15.12:0/2153281481 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2701000e0 0x7fd270100500 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:38.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:38.238+0000 7fd2768d4640 1 -- 172.21.15.12:0/2153281481 >> 172.21.15.12:0/2153281481 conn(0x7fd2700fb650 msgr2=0x7fd2700fda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:38.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:38.238+0000 7fd2768d4640 1 -- 172.21.15.12:0/2153281481 shutdown_connections 2024-02-18T11:38:38.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:38.238+0000 7fd2768d4640 1 -- 172.21.15.12:0/2153281481 wait complete. 2024-02-18T11:38:38.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:38.239+0000 7fd2768d4640 1 Processor -- start 2024-02-18T11:38:38.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:38.239+0000 7fd2768d4640 1 -- start start 2024-02-18T11:38:38.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:38.239+0000 7fd2768d4640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd2701000e0 0x7fd2701972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:38.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:38.239+0000 7fd2768d4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2701012d0 0x7fd270197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:38.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:38.239+0000 7fd2768d4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd270101c90 0x7fd27019e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:38.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:38.239+0000 7fd2768d4640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd27019edb0 con 0x7fd270101c90 2024-02-18T11:38:38.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:38.239+0000 7fd2768d4640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd27019ef20 con 0x7fd2701000e0 2024-02-18T11:38:38.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:38.239+0000 7fd2768d4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd27019f0e0 con 0x7fd2701012d0 2024-02-18T11:38:38.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:38.239+0000 7fd274e71640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd270101c90 0x7fd27019e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:38.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:38.239+0000 7fd274e71640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd270101c90 0x7fd27019e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42454/0 (socket says 172.21.15.12:42454) 2024-02-18T11:38:38.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:38.240+0000 7fd274e71640 1 -- 172.21.15.12:0/3541477157 learned_addr learned my addr 172.21.15.12:0/3541477157 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:38:38.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:38.240+0000 7fd26ffff640 1 --2- 172.21.15.12:0/3541477157 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd2701000e0 0x7fd2701972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:38.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:38.240+0000 7fd26f7fe640 1 --2- 172.21.15.12:0/3541477157 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2701012d0 0x7fd270197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:38.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:38.240+0000 7fd274e71640 1 -- 172.21.15.12:0/3541477157 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2701012d0 msgr2=0x7fd270197800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:38.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:38.240+0000 7fd274e71640 1 --2- 172.21.15.12:0/3541477157 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2701012d0 0x7fd270197800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:38.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:38.240+0000 7fd274e71640 1 -- 172.21.15.12:0/3541477157 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd2701000e0 msgr2=0x7fd2701972c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:38.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:38.240+0000 7fd274e71640 1 --2- 172.21.15.12:0/3541477157 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd2701000e0 0x7fd2701972c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:38.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:38.240+0000 7fd274e71640 1 -- 172.21.15.12:0/3541477157 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd260009560 con 0x7fd270101c90 2024-02-18T11:38:38.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:38.240+0000 7fd26f7fe640 1 --2- 172.21.15.12:0/3541477157 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2701012d0 0x7fd270197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:38:38.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:38.240+0000 7fd274e71640 1 --2- 172.21.15.12:0/3541477157 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd270101c90 0x7fd27019e870 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd26800eaa0 tx=0x7fd26800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:38.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:38.240+0000 7fd26d7fa640 1 -- 172.21.15.12:0/3541477157 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd26800cdb0 con 0x7fd270101c90 2024-02-18T11:38:38.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:38.240+0000 7fd26d7fa640 1 -- 172.21.15.12:0/3541477157 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd268005020 con 0x7fd270101c90 2024-02-18T11:38:38.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:38.240+0000 7fd26d7fa640 1 -- 172.21.15.12:0/3541477157 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2680116b0 con 0x7fd270101c90 2024-02-18T11:38:38.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:38.240+0000 7fd2768d4640 1 -- 172.21.15.12:0/3541477157 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd27019f3c0 con 0x7fd270101c90 2024-02-18T11:38:38.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:38.240+0000 7fd2768d4640 1 -- 172.21.15.12:0/3541477157 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd27019f880 con 0x7fd270101c90 2024-02-18T11:38:38.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:38.241+0000 7fd2768d4640 1 -- 172.21.15.12:0/3541477157 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd270107400 con 0x7fd270101c90 2024-02-18T11:38:38.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:38.242+0000 7fd26d7fa640 1 -- 172.21.15.12:0/3541477157 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) v1 ==== 137245+0+0 (secure 0 0 0) 0x7fd268005520 con 0x7fd270101c90 2024-02-18T11:38:38.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:38.243+0000 7fd26d7fa640 1 --2- 172.21.15.12:0/3541477157 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fd2580a2750 0x7fd2580a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:38.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:38.243+0000 7fd26ffff640 1 --2- 172.21.15.12:0/3541477157 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fd2580a2750 0x7fd2580a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:38.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:38.243+0000 7fd26d7fa640 1 -- 172.21.15.12:0/3541477157 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd2680ce770 con 0x7fd270101c90 2024-02-18T11:38:38.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:38.244+0000 7fd26ffff640 1 --2- 172.21.15.12:0/3541477157 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fd2580a2750 0x7fd2580a4c10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd26002c240 tx=0x7fd26002c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:38.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:38.247+0000 7fd26d7fa640 1 -- 172.21.15.12:0/3541477157 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd268097900 con 0x7fd270101c90 2024-02-18T11:38:38.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:38.476+0000 7fd2768d4640 1 -- 172.21.15.12:0/3541477157 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd27010cc40 con 0x7fd270101c90 2024-02-18T11:38:38.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:38.477+0000 7fd26d7fa640 1 -- 172.21.15.12:0/3541477157 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8462 (secure 0 0 0) 0x7fd26809afc0 con 0x7fd270101c90 2024-02-18T11:38:38.471 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:38:38.471 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1788335302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1788335302},{"type":"v1","addr":"172.21.15.12:6835","nonce":1788335302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/2144692455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":2144692455},{"type":"v1","addr":"172.21.15.12:6837","nonce":2144692455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3569056036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3569056036},{"type":"v1","addr":"172.21.15.12:6839","nonce":3569056036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1207202465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1207202465},{"type":"v1","addr":"172.21.15.188:6833","nonce":1207202465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6833/2636348323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6832","nonce":2636348323},{"type":"v1","addr":"172.21.15.158:6833","nonce":2636348323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24446,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6835/3673253642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":3673253642},{"type":"v1","addr":"172.21.15.158:6835","nonce":3673253642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/806709644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":806709644},{"type":"v1","addr":"172.21.15.188:6835","nonce":806709644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24476,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6837/813251825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6836","nonce":813251825},{"type":"v1","addr":"172.21.15.158:6837","nonce":813251825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1036630143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1036630143},{"type":"v1","addr":"172.21.15.188:6837","nonce":1036630143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24508,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6839/1342671190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":1342671190},{"type":"v1","addr":"172.21.15.188:6839","nonce":1342671190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-02-18T11:38:38.472 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 11 2024-02-18T11:38:38.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:38.480+0000 7fd2768d4640 1 -- 172.21.15.12:0/3541477157 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fd2580a2750 msgr2=0x7fd2580a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:38.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:38.480+0000 7fd2768d4640 1 --2- 172.21.15.12:0/3541477157 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fd2580a2750 0x7fd2580a4c10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd26002c240 tx=0x7fd26002c7e0 comp rx=0 tx=0).stop 2024-02-18T11:38:38.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:38.480+0000 7fd2768d4640 1 -- 172.21.15.12:0/3541477157 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd270101c90 msgr2=0x7fd27019e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:38.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:38.480+0000 7fd2768d4640 1 --2- 172.21.15.12:0/3541477157 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd270101c90 0x7fd27019e870 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd26800eaa0 tx=0x7fd26800ef60 comp rx=0 tx=0).stop 2024-02-18T11:38:38.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:38.480+0000 7fd2768d4640 1 -- 172.21.15.12:0/3541477157 shutdown_connections 2024-02-18T11:38:38.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:38.480+0000 7fd2768d4640 1 --2- 172.21.15.12:0/3541477157 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fd2580a2750 0x7fd2580a4c10 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:38.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:38.480+0000 7fd2768d4640 1 --2- 172.21.15.12:0/3541477157 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2701012d0 0x7fd270197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:38.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:38.480+0000 7fd2768d4640 1 --2- 172.21.15.12:0/3541477157 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd270101c90 0x7fd27019e870 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:38.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:38.480+0000 7fd2768d4640 1 --2- 172.21.15.12:0/3541477157 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd2701000e0 0x7fd2701972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:38.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:38.480+0000 7fd2768d4640 1 -- 172.21.15.12:0/3541477157 >> 172.21.15.12:0/3541477157 conn(0x7fd2700fb650 msgr2=0x7fd2701045d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:38.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:38.480+0000 7fd2768d4640 1 -- 172.21.15.12:0/3541477157 shutdown_connections 2024-02-18T11:38:38.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:38.480+0000 7fd2768d4640 1 -- 172.21.15.12:0/3541477157 wait complete. 2024-02-18T11:38:38.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:38 smithi012 ceph-mon[32460]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-18T11:38:38.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:38 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/3541477157' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-18T11:38:38.923 INFO:journalctl@ceph.mds.a.smithi012.stdout:Feb 18 11:38:38 smithi012 systemd[1]: Starting Ceph mds.a for 0c8bb0de-ce51-11ee-95ba-87774f69a715... 2024-02-18T11:38:38.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:38 smithi158 ceph-mon[36626]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-18T11:38:38.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:38 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/3541477157' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-18T11:38:39.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:38 smithi188 ceph-mon[36925]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-18T11:38:39.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:38 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/3541477157' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-18T11:38:39.298 INFO:journalctl@ceph.mds.a.smithi012.stdout:Feb 18 11:38:39 smithi012 podman[72538]: 2024-02-18 11:38:38.938647644 +0000 UTC m=+0.017747826 image pull 53c33ee3bf62dc411b650f86eedbbee7698efb1001b4c063271dcd8d65b41165 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1 2024-02-18T11:38:39.298 INFO:journalctl@ceph.mds.a.smithi012.stdout:Feb 18 11:38:39 smithi012 podman[72538]: 2024-02-18 11:38:39.051968431 +0000 UTC m=+0.131068605 container create 651883f871d93bee10b8a16dfb7ec4103b5a802fb93b9069e0d7377dae87ad27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-a, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-18T11:38:39.673 INFO:journalctl@ceph.mds.a.smithi012.stdout:Feb 18 11:38:39 smithi012 podman[72538]: 2024-02-18 11:38:39.417490544 +0000 UTC m=+0.496590721 container init 651883f871d93bee10b8a16dfb7ec4103b5a802fb93b9069e0d7377dae87ad27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-a, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-02-18T11:38:39.673 INFO:journalctl@ceph.mds.a.smithi012.stdout:Feb 18 11:38:39 smithi012 podman[72538]: 2024-02-18 11:38:39.420102465 +0000 UTC m=+0.499202644 container start 651883f871d93bee10b8a16dfb7ec4103b5a802fb93b9069e0d7377dae87ad27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-a, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-18T11:38:39.673 INFO:journalctl@ceph.mds.a.smithi012.stdout:Feb 18 11:38:39 smithi012 ceph-mds[72557]: -- 172.21.15.12:0/1251707923 <== mon.0 v2:172.21.15.12:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55cab1b1e1a0 con 0x55cab1af4c00 2024-02-18T11:38:39.673 INFO:journalctl@ceph.mds.a.smithi012.stdout:Feb 18 11:38:39 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-a[72553]: starting mds.a at 2024-02-18T11:38:39.673 INFO:journalctl@ceph.mds.a.smithi012.stdout:Feb 18 11:38:39 smithi012 bash[72538]: 651883f871d93bee10b8a16dfb7ec4103b5a802fb93b9069e0d7377dae87ad27 2024-02-18T11:38:39.674 INFO:journalctl@ceph.mds.a.smithi012.stdout:Feb 18 11:38:39 smithi012 systemd[1]: Started Ceph mds.a for 0c8bb0de-ce51-11ee-95ba-87774f69a715. 2024-02-18T11:38:40.489 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-18T11:38:40.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:40.615+0000 7f2ac0040640 1 -- 172.21.15.12:0/3295112056 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ab8101c90 msgr2=0x7f2ab8108520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:40.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:40.615+0000 7f2ac0040640 1 --2- 172.21.15.12:0/3295112056 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ab8101c90 0x7f2ab8108520 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2aa8009ca0 tx=0x7f2aa802c0f0 comp rx=0 tx=0).stop 2024-02-18T11:38:40.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:40.615+0000 7f2ac0040640 1 -- 172.21.15.12:0/3295112056 shutdown_connections 2024-02-18T11:38:40.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:40.615+0000 7f2ac0040640 1 --2- 172.21.15.12:0/3295112056 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ab81012d0 0x7f2ab8101750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:40.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:40.615+0000 7f2ac0040640 1 --2- 172.21.15.12:0/3295112056 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ab8101c90 0x7f2ab8108520 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:40.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:40.615+0000 7f2ac0040640 1 --2- 172.21.15.12:0/3295112056 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2ab81000e0 0x7f2ab8100500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:40.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:40.615+0000 7f2ac0040640 1 -- 172.21.15.12:0/3295112056 >> 172.21.15.12:0/3295112056 conn(0x7f2ab80fb650 msgr2=0x7f2ab80fda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:40.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:40.615+0000 7f2ac0040640 1 -- 172.21.15.12:0/3295112056 shutdown_connections 2024-02-18T11:38:40.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:40.615+0000 7f2ac0040640 1 -- 172.21.15.12:0/3295112056 wait complete. 2024-02-18T11:38:40.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:40.616+0000 7f2ac0040640 1 Processor -- start 2024-02-18T11:38:40.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:40.616+0000 7f2ac0040640 1 -- start start 2024-02-18T11:38:40.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:40.616+0000 7f2ac0040640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ab81000e0 0x7f2ab810a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:40.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:40.616+0000 7f2ac0040640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ab81012d0 0x7f2ab8108fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:40.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:40.616+0000 7f2ac0040640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2ab8101c90 0x7f2ab8109520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:40.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:40.616+0000 7f2ac0040640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab8109c80 con 0x7f2ab81012d0 2024-02-18T11:38:40.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:40.616+0000 7f2ac0040640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab8109df0 con 0x7f2ab8101c90 2024-02-18T11:38:40.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:40.616+0000 7f2ac0040640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab81a1300 con 0x7f2ab81000e0 2024-02-18T11:38:40.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:40.616+0000 7f2abd5db640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ab81012d0 0x7f2ab8108fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:40.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:40.616+0000 7f2abd5db640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ab81012d0 0x7f2ab8108fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42502/0 (socket says 172.21.15.12:42502) 2024-02-18T11:38:40.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:40.616+0000 7f2abd5db640 1 -- 172.21.15.12:0/664178858 learned_addr learned my addr 172.21.15.12:0/664178858 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:38:40.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:40.616+0000 7f2abe5dd640 1 --2- 172.21.15.12:0/664178858 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2ab8101c90 0x7f2ab8109520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:40.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:40.616+0000 7f2abd5db640 1 -- 172.21.15.12:0/664178858 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ab81000e0 msgr2=0x7f2ab810a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:40.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:40.616+0000 7f2abd5db640 1 --2- 172.21.15.12:0/664178858 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ab81000e0 0x7f2ab810a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:40.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:40.616+0000 7f2abd5db640 1 -- 172.21.15.12:0/664178858 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2ab8101c90 msgr2=0x7f2ab8109520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:40.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:40.616+0000 7f2abd5db640 1 --2- 172.21.15.12:0/664178858 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2ab8101c90 0x7f2ab8109520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:40.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:40.616+0000 7f2abd5db640 1 -- 172.21.15.12:0/664178858 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aa8009900 con 0x7f2ab81012d0 2024-02-18T11:38:40.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:40.617+0000 7f2abd5db640 1 --2- 172.21.15.12:0/664178858 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ab81012d0 0x7f2ab8108fe0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2aac00b4d0 tx=0x7f2aac00b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:40.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:40.617+0000 7f2ab6ffd640 1 -- 172.21.15.12:0/664178858 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aac010870 con 0x7f2ab81012d0 2024-02-18T11:38:40.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:40.617+0000 7f2ac0040640 1 -- 172.21.15.12:0/664178858 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ab81a15b0 con 0x7f2ab81012d0 2024-02-18T11:38:40.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:40.617+0000 7f2ab6ffd640 1 -- 172.21.15.12:0/664178858 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2aac00f3c0 con 0x7f2ab81012d0 2024-02-18T11:38:40.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:40.617+0000 7f2ab6ffd640 1 -- 172.21.15.12:0/664178858 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aac0175c0 con 0x7f2ab81012d0 2024-02-18T11:38:40.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:40.617+0000 7f2ac0040640 1 -- 172.21.15.12:0/664178858 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ab81a1b10 con 0x7f2ab81012d0 2024-02-18T11:38:40.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:40.618+0000 7f2ac0040640 1 -- 172.21.15.12:0/664178858 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ab81073a0 con 0x7f2ab81012d0 2024-02-18T11:38:40.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:40.620+0000 7f2ab6ffd640 1 -- 172.21.15.12:0/664178858 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) v1 ==== 137245+0+0 (secure 0 0 0) 0x7f2aac00f8c0 con 0x7f2ab81012d0 2024-02-18T11:38:40.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:40.620+0000 7f2ab6ffd640 1 --2- 172.21.15.12:0/664178858 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f2aa00a2750 0x7f2aa00a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:40.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:40.620+0000 7f2abdddc640 1 --2- 172.21.15.12:0/664178858 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f2aa00a2750 0x7f2aa00a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:40.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:40.620+0000 7f2ab6ffd640 1 -- 172.21.15.12:0/664178858 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f2aac0cdff0 con 0x7f2ab81012d0 2024-02-18T11:38:40.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:40.621+0000 7f2abdddc640 1 --2- 172.21.15.12:0/664178858 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f2aa00a2750 0x7f2aa00a4c10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2ab0005fd0 tx=0x7f2ab000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:40.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:40.624+0000 7f2ab6ffd640 1 -- 172.21.15.12:0/664178858 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2aac0971c0 con 0x7f2ab81012d0 2024-02-18T11:38:40.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:40.843+0000 7f2ac0040640 1 -- 172.21.15.12:0/664178858 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2ab810cc30 con 0x7f2ab81012d0 2024-02-18T11:38:40.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:40.844+0000 7f2ab6ffd640 1 -- 172.21.15.12:0/664178858 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+9254 (secure 0 0 0) 0x7f2aac0092b0 con 0x7f2ab81012d0 2024-02-18T11:38:40.838 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:38:40.838 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1788335302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1788335302},{"type":"v1","addr":"172.21.15.12:6835","nonce":1788335302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/2144692455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":2144692455},{"type":"v1","addr":"172.21.15.12:6837","nonce":2144692455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3569056036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3569056036},{"type":"v1","addr":"172.21.15.12:6839","nonce":3569056036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14622,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/1973234486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":1973234486},{"type":"v1","addr":"172.21.15.12:6841","nonce":1973234486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1207202465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1207202465},{"type":"v1","addr":"172.21.15.188:6833","nonce":1207202465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6833/2636348323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6832","nonce":2636348323},{"type":"v1","addr":"172.21.15.158:6833","nonce":2636348323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24446,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6835/3673253642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":3673253642},{"type":"v1","addr":"172.21.15.158:6835","nonce":3673253642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/806709644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":806709644},{"type":"v1","addr":"172.21.15.188:6835","nonce":806709644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24476,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6837/813251825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6836","nonce":813251825},{"type":"v1","addr":"172.21.15.158:6837","nonce":813251825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1036630143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1036630143},{"type":"v1","addr":"172.21.15.188:6837","nonce":1036630143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24508,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6839/1342671190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":1342671190},{"type":"v1","addr":"172.21.15.188:6839","nonce":1342671190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-02-18T11:38:40.839 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 12 2024-02-18T11:38:40.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:40.846+0000 7f2ac0040640 1 -- 172.21.15.12:0/664178858 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f2aa00a2750 msgr2=0x7f2aa00a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:40.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:40.846+0000 7f2ac0040640 1 --2- 172.21.15.12:0/664178858 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f2aa00a2750 0x7f2aa00a4c10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2ab0005fd0 tx=0x7f2ab000b410 comp rx=0 tx=0).stop 2024-02-18T11:38:40.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:40.846+0000 7f2ac0040640 1 -- 172.21.15.12:0/664178858 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ab81012d0 msgr2=0x7f2ab8108fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:40.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:40.846+0000 7f2ac0040640 1 --2- 172.21.15.12:0/664178858 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ab81012d0 0x7f2ab8108fe0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2aac00b4d0 tx=0x7f2aac00b990 comp rx=0 tx=0).stop 2024-02-18T11:38:40.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:40.846+0000 7f2ac0040640 1 -- 172.21.15.12:0/664178858 shutdown_connections 2024-02-18T11:38:40.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:40.846+0000 7f2ac0040640 1 --2- 172.21.15.12:0/664178858 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f2aa00a2750 0x7f2aa00a4c10 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:40.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:40.846+0000 7f2ac0040640 1 --2- 172.21.15.12:0/664178858 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2ab8101c90 0x7f2ab8109520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:40.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:40.846+0000 7f2ac0040640 1 --2- 172.21.15.12:0/664178858 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ab81012d0 0x7f2ab8108fe0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:40.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:40.846+0000 7f2ac0040640 1 --2- 172.21.15.12:0/664178858 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ab81000e0 0x7f2ab810a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:40.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:40.846+0000 7f2ac0040640 1 -- 172.21.15.12:0/664178858 >> 172.21.15.12:0/664178858 conn(0x7f2ab80fb650 msgr2=0x7f2ab8104570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:40.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:40.846+0000 7f2ac0040640 1 -- 172.21.15.12:0/664178858 shutdown_connections 2024-02-18T11:38:40.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:40.846+0000 7f2ac0040640 1 -- 172.21.15.12:0/664178858 wait complete. 2024-02-18T11:38:40.850 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:40 smithi012 ceph-mon[32460]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-18T11:38:40.850 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:40 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:40.850 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:40 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:40.850 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:40 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:40.850 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:40 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-18T11:38:40.850 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:40 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-18T11:38:40.851 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:40 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:38:40.851 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:40 smithi012 ceph-mon[32460]: Deploying daemon mds.b on smithi158 2024-02-18T11:38:40.851 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:40 smithi012 ceph-mon[32460]: mds.? [v2:172.21.15.12:6840/1973234486,v1:172.21.15.12:6841/1973234486] up:boot 2024-02-18T11:38:40.851 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:40 smithi012 ceph-mon[32460]: fsmap 11 up:standby 2024-02-18T11:38:40.851 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:40 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-02-18T11:38:40.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:40 smithi188 ceph-mon[36925]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-18T11:38:40.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:40 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:40.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:40 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:40.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:40 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:40.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:40 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-18T11:38:40.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:40 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-18T11:38:40.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:40 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:38:40.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:40 smithi188 ceph-mon[36925]: Deploying daemon mds.b on smithi158 2024-02-18T11:38:40.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:40 smithi188 ceph-mon[36925]: mds.? [v2:172.21.15.12:6840/1973234486,v1:172.21.15.12:6841/1973234486] up:boot 2024-02-18T11:38:40.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:40 smithi188 ceph-mon[36925]: fsmap 11 up:standby 2024-02-18T11:38:40.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:40 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-02-18T11:38:40.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:40 smithi158 ceph-mon[36626]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-18T11:38:40.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:40 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:40.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:40 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:40.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:40 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:40.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:40 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-18T11:38:40.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:40 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-18T11:38:40.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:40 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:38:40.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:40 smithi158 ceph-mon[36626]: Deploying daemon mds.b on smithi158 2024-02-18T11:38:40.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:40 smithi158 ceph-mon[36626]: mds.? [v2:172.21.15.12:6840/1973234486,v1:172.21.15.12:6841/1973234486] up:boot 2024-02-18T11:38:40.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:40 smithi158 ceph-mon[36626]: fsmap 11 up:standby 2024-02-18T11:38:40.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:40 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-02-18T11:38:41.795 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:41 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/664178858' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-18T11:38:41.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:41 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/664178858' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-18T11:38:41.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:41 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/664178858' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-18T11:38:42.541 INFO:journalctl@ceph.mds.b.smithi158.stdout:Feb 18 11:38:42 smithi158 systemd[1]: Started Ceph mds.b for 0c8bb0de-ce51-11ee-95ba-87774f69a715. 2024-02-18T11:38:42.852 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-18T11:38:42.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:42 smithi188 ceph-mon[36925]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-18T11:38:42.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:42 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:42.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:42 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:42.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:42 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:42.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:42 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:42.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:42 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:42.877 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:42 smithi012 ceph-mon[32460]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-18T11:38:42.877 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:42 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:42.877 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:42 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:42.877 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:42 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:42.877 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:42 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:42.877 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:42 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:42.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:42.964+0000 7f5cf2c5b640 1 -- 172.21.15.12:0/498427226 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5cec101200 msgr2=0x7f5cec101620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:42.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:42.964+0000 7f5cf2c5b640 1 --2- 172.21.15.12:0/498427226 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5cec101200 0x7f5cec101620 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5ce0009900 tx=0x7f5ce002bc30 comp rx=0 tx=0).stop 2024-02-18T11:38:42.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:42.964+0000 7f5cf2c5b640 1 -- 172.21.15.12:0/498427226 shutdown_connections 2024-02-18T11:38:42.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:42.964+0000 7f5cf2c5b640 1 --2- 172.21.15.12:0/498427226 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5cec102db0 0x7f5cec109640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:42.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:42.964+0000 7f5cf2c5b640 1 --2- 172.21.15.12:0/498427226 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5cec1023f0 0x7f5cec102870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:42.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:42.964+0000 7f5cf2c5b640 1 --2- 172.21.15.12:0/498427226 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5cec101200 0x7f5cec101620 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:42.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:42.964+0000 7f5cf2c5b640 1 -- 172.21.15.12:0/498427226 >> 172.21.15.12:0/498427226 conn(0x7f5cec0fc770 msgr2=0x7f5cec0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:42.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:42.964+0000 7f5cf2c5b640 1 -- 172.21.15.12:0/498427226 shutdown_connections 2024-02-18T11:38:42.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:42.964+0000 7f5cf2c5b640 1 -- 172.21.15.12:0/498427226 wait complete. 2024-02-18T11:38:42.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:42.964+0000 7f5cf2c5b640 1 Processor -- start 2024-02-18T11:38:42.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:42.964+0000 7f5cf2c5b640 1 -- start start 2024-02-18T11:38:42.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:42.964+0000 7f5cf2c5b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5cec101200 0x7f5cec074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:42.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:42.965+0000 7f5cf2c5b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5cec1023f0 0x7f5cec0735b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:42.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:42.965+0000 7f5cf2c5b640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5cec102db0 0x7f5cec073af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:42.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:42.965+0000 7f5cf2c5b640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cec074250 con 0x7f5cec101200 2024-02-18T11:38:42.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:42.965+0000 7f5cf2c5b640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cec0743c0 con 0x7f5cec102db0 2024-02-18T11:38:42.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:42.965+0000 7f5cf2c5b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cec1a1310 con 0x7f5cec1023f0 2024-02-18T11:38:42.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:42.965+0000 7f5cf09f7640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5cec101200 0x7f5cec074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:42.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:42.965+0000 7f5cf09f7640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5cec101200 0x7f5cec074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46604/0 (socket says 172.21.15.12:46604) 2024-02-18T11:38:42.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:42.965+0000 7f5cf09f7640 1 -- 172.21.15.12:0/491148043 learned_addr learned my addr 172.21.15.12:0/491148043 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:38:42.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:42.965+0000 7f5cebfff640 1 --2- 172.21.15.12:0/491148043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5cec1023f0 0x7f5cec0735b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:42.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:42.965+0000 7f5cf11f8640 1 --2- 172.21.15.12:0/491148043 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5cec102db0 0x7f5cec073af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:42.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:42.965+0000 7f5cf09f7640 1 -- 172.21.15.12:0/491148043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5cec1023f0 msgr2=0x7f5cec0735b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:42.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:42.965+0000 7f5cf09f7640 1 --2- 172.21.15.12:0/491148043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5cec1023f0 0x7f5cec0735b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:42.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:42.965+0000 7f5cf09f7640 1 -- 172.21.15.12:0/491148043 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5cec102db0 msgr2=0x7f5cec073af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:42.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:42.965+0000 7f5cf09f7640 1 --2- 172.21.15.12:0/491148043 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5cec102db0 0x7f5cec073af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:42.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:42.965+0000 7f5cf09f7640 1 -- 172.21.15.12:0/491148043 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ce0009560 con 0x7f5cec101200 2024-02-18T11:38:42.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:42.965+0000 7f5cf09f7640 1 --2- 172.21.15.12:0/491148043 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5cec101200 0x7f5cec074f70 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5ce0000c00 tx=0x7f5ce0005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:42.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:42.965+0000 7f5ce9ffb640 1 -- 172.21.15.12:0/491148043 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce00043d0 con 0x7f5cec101200 2024-02-18T11:38:42.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:42.965+0000 7f5ce9ffb640 1 -- 172.21.15.12:0/491148043 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5ce00052d0 con 0x7f5cec101200 2024-02-18T11:38:42.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:42.965+0000 7f5cf2c5b640 1 -- 172.21.15.12:0/491148043 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cec1a1560 con 0x7f5cec101200 2024-02-18T11:38:42.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:42.965+0000 7f5ce9ffb640 1 -- 172.21.15.12:0/491148043 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce0003d70 con 0x7f5cec101200 2024-02-18T11:38:42.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:42.966+0000 7f5cf2c5b640 1 -- 172.21.15.12:0/491148043 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cec1a1a70 con 0x7f5cec101200 2024-02-18T11:38:42.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:42.966+0000 7f5cf2c5b640 1 -- 172.21.15.12:0/491148043 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cec108570 con 0x7f5cec101200 2024-02-18T11:38:42.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:42.968+0000 7f5ce9ffb640 1 -- 172.21.15.12:0/491148043 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) v1 ==== 137245+0+0 (secure 0 0 0) 0x7f5ce0005440 con 0x7f5cec101200 2024-02-18T11:38:42.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:42.968+0000 7f5ce9ffb640 1 --2- 172.21.15.12:0/491148043 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f5cdc0a2750 0x7f5cdc0a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:42.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:42.968+0000 7f5ce9ffb640 1 -- 172.21.15.12:0/491148043 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f5ce00f0770 con 0x7f5cec101200 2024-02-18T11:38:42.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:42.968+0000 7f5cebfff640 1 --2- 172.21.15.12:0/491148043 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f5cdc0a2750 0x7f5cdc0a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:42.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:42.969+0000 7f5cebfff640 1 --2- 172.21.15.12:0/491148043 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f5cdc0a2750 0x7f5cdc0a4c10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5cec074ce0 tx=0x7f5cd8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:42.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:42.976+0000 7f5ce9ffb640 1 -- 172.21.15.12:0/491148043 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5ce00b99a0 con 0x7f5cec101200 2024-02-18T11:38:42.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:42 smithi158 ceph-mon[36626]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-18T11:38:42.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:42 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:42.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:42 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:42.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:42 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:42.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:42 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:42.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:42 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:43.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.211+0000 7f5cf2c5b640 1 -- 172.21.15.12:0/491148043 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5cec10cc10 con 0x7f5cec101200 2024-02-18T11:38:43.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.212+0000 7f5ce9ffb640 1 -- 172.21.15.12:0/491148043 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+10049 (secure 0 0 0) 0x7f5ce00bd060 con 0x7f5cec101200 2024-02-18T11:38:43.207 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:38:43.207 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1788335302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1788335302},{"type":"v1","addr":"172.21.15.12:6835","nonce":1788335302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/2144692455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":2144692455},{"type":"v1","addr":"172.21.15.12:6837","nonce":2144692455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3569056036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3569056036},{"type":"v1","addr":"172.21.15.12:6839","nonce":3569056036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14622,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/1973234486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":1973234486},{"type":"v1","addr":"172.21.15.12:6841","nonce":1973234486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1207202465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1207202465},{"type":"v1","addr":"172.21.15.188:6833","nonce":1207202465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6833/2636348323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6832","nonce":2636348323},{"type":"v1","addr":"172.21.15.158:6833","nonce":2636348323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24446,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6835/3673253642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":3673253642},{"type":"v1","addr":"172.21.15.158:6835","nonce":3673253642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/806709644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":806709644},{"type":"v1","addr":"172.21.15.188:6835","nonce":806709644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24476,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6837/813251825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6836","nonce":813251825},{"type":"v1","addr":"172.21.15.158:6837","nonce":813251825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1036630143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1036630143},{"type":"v1","addr":"172.21.15.188:6837","nonce":1036630143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24508,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6839/1342671190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":1342671190},{"type":"v1","addr":"172.21.15.188:6839","nonce":1342671190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6839/1127200849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6838","nonce":1127200849},{"type":"v1","addr":"172.21.15.158:6839","nonce":1127200849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[]} 2024-02-18T11:38:43.207 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 13 2024-02-18T11:38:43.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.215+0000 7f5cf2c5b640 1 -- 172.21.15.12:0/491148043 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f5cdc0a2750 msgr2=0x7f5cdc0a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:43.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.215+0000 7f5cf2c5b640 1 --2- 172.21.15.12:0/491148043 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f5cdc0a2750 0x7f5cdc0a4c10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5cec074ce0 tx=0x7f5cd8005f90 comp rx=0 tx=0).stop 2024-02-18T11:38:43.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.215+0000 7f5cf2c5b640 1 -- 172.21.15.12:0/491148043 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5cec101200 msgr2=0x7f5cec074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:43.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.215+0000 7f5cf2c5b640 1 --2- 172.21.15.12:0/491148043 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5cec101200 0x7f5cec074f70 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5ce0000c00 tx=0x7f5ce0005e50 comp rx=0 tx=0).stop 2024-02-18T11:38:43.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.215+0000 7f5cf2c5b640 1 -- 172.21.15.12:0/491148043 shutdown_connections 2024-02-18T11:38:43.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.215+0000 7f5cf2c5b640 1 --2- 172.21.15.12:0/491148043 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f5cdc0a2750 0x7f5cdc0a4c10 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:43.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.215+0000 7f5cf2c5b640 1 --2- 172.21.15.12:0/491148043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5cec1023f0 0x7f5cec0735b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:43.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.215+0000 7f5cf2c5b640 1 --2- 172.21.15.12:0/491148043 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5cec102db0 0x7f5cec073af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:43.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.215+0000 7f5cf2c5b640 1 --2- 172.21.15.12:0/491148043 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5cec101200 0x7f5cec074f70 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:43.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.215+0000 7f5cf2c5b640 1 -- 172.21.15.12:0/491148043 >> 172.21.15.12:0/491148043 conn(0x7f5cec0fc770 msgr2=0x7f5cec105740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:43.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.215+0000 7f5cf2c5b640 1 -- 172.21.15.12:0/491148043 shutdown_connections 2024-02-18T11:38:43.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.215+0000 7f5cf2c5b640 1 -- 172.21.15.12:0/491148043 wait complete. 2024-02-18T11:38:43.222 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-18T11:38:43.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.369+0000 7f2ce0902640 1 -- 172.21.15.12:0/4072308611 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2cdc1012f0 msgr2=0x7f2cdc101770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:43.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.369+0000 7f2ce0902640 1 --2- 172.21.15.12:0/4072308611 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2cdc1012f0 0x7f2cdc101770 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc009900 tx=0x7f2ccc02bc30 comp rx=0 tx=0).stop 2024-02-18T11:38:43.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.369+0000 7f2ce0902640 1 -- 172.21.15.12:0/4072308611 shutdown_connections 2024-02-18T11:38:43.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.369+0000 7f2ce0902640 1 --2- 172.21.15.12:0/4072308611 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2cdc101cb0 0x7f2cdc108540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:43.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.369+0000 7f2ce0902640 1 --2- 172.21.15.12:0/4072308611 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2cdc1012f0 0x7f2cdc101770 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:43.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.369+0000 7f2ce0902640 1 --2- 172.21.15.12:0/4072308611 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2cdc100100 0x7f2cdc100520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:43.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.369+0000 7f2ce0902640 1 -- 172.21.15.12:0/4072308611 >> 172.21.15.12:0/4072308611 conn(0x7f2cdc0fb670 msgr2=0x7f2cdc0fdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:43.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.370+0000 7f2ce0902640 1 -- 172.21.15.12:0/4072308611 shutdown_connections 2024-02-18T11:38:43.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.370+0000 7f2ce0902640 1 -- 172.21.15.12:0/4072308611 wait complete. 2024-02-18T11:38:43.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.370+0000 7f2ce0902640 1 Processor -- start 2024-02-18T11:38:43.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.370+0000 7f2ce0902640 1 -- start start 2024-02-18T11:38:43.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.370+0000 7f2ce0902640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2cdc100100 0x7f2cdc104090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:43.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.370+0000 7f2ce0902640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2cdc1012f0 0x7f2cdc1026d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:43.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.370+0000 7f2ce0902640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2cdc101cb0 0x7f2cdc102c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:43.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.370+0000 7f2ce0902640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cdc103370 con 0x7f2cdc101cb0 2024-02-18T11:38:43.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.370+0000 7f2ce0902640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cdc1034e0 con 0x7f2cdc100100 2024-02-18T11:38:43.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.370+0000 7f2ce0902640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cdc1a1380 con 0x7f2cdc1012f0 2024-02-18T11:38:43.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.371+0000 7f2cdad9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2cdc101cb0 0x7f2cdc102c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:43.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.371+0000 7f2cdad9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2cdc101cb0 0x7f2cdc102c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46612/0 (socket says 172.21.15.12:46612) 2024-02-18T11:38:43.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.371+0000 7f2cdad9d640 1 -- 172.21.15.12:0/149984442 learned_addr learned my addr 172.21.15.12:0/149984442 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:38:43.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.371+0000 7f2cd9d9b640 1 --2- 172.21.15.12:0/149984442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2cdc1012f0 0x7f2cdc1026d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:43.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.371+0000 7f2cda59c640 1 --2- 172.21.15.12:0/149984442 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2cdc100100 0x7f2cdc104090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:43.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.371+0000 7f2cdad9d640 1 -- 172.21.15.12:0/149984442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2cdc1012f0 msgr2=0x7f2cdc1026d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:43.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.371+0000 7f2cdad9d640 1 --2- 172.21.15.12:0/149984442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2cdc1012f0 0x7f2cdc1026d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:43.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.371+0000 7f2cdad9d640 1 -- 172.21.15.12:0/149984442 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2cdc100100 msgr2=0x7f2cdc104090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:43.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.371+0000 7f2cdad9d640 1 --2- 172.21.15.12:0/149984442 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2cdc100100 0x7f2cdc104090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:43.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.371+0000 7f2cdad9d640 1 -- 172.21.15.12:0/149984442 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ccc009560 con 0x7f2cdc101cb0 2024-02-18T11:38:43.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.371+0000 7f2cda59c640 1 --2- 172.21.15.12:0/149984442 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2cdc100100 0x7f2cdc104090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:38:43.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.371+0000 7f2cdad9d640 1 --2- 172.21.15.12:0/149984442 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2cdc101cb0 0x7f2cdc102c10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2cd400b9a0 tx=0x7f2cd400be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:43.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.371+0000 7f2ccb7fe640 1 -- 172.21.15.12:0/149984442 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd400cba0 con 0x7f2cdc101cb0 2024-02-18T11:38:43.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.371+0000 7f2ccb7fe640 1 -- 172.21.15.12:0/149984442 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2cd400cd00 con 0x7f2cdc101cb0 2024-02-18T11:38:43.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.371+0000 7f2ce0902640 1 -- 172.21.15.12:0/149984442 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cdc1a1630 con 0x7f2cdc101cb0 2024-02-18T11:38:43.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.371+0000 7f2ccb7fe640 1 -- 172.21.15.12:0/149984442 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd40076f0 con 0x7f2cdc101cb0 2024-02-18T11:38:43.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.371+0000 7f2ce0902640 1 -- 172.21.15.12:0/149984442 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cdc1a1b40 con 0x7f2cdc101cb0 2024-02-18T11:38:43.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.372+0000 7f2ce0902640 1 -- 172.21.15.12:0/149984442 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cdc10bbc0 con 0x7f2cdc101cb0 2024-02-18T11:38:43.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.373+0000 7f2ccb7fe640 1 -- 172.21.15.12:0/149984442 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) v1 ==== 137245+0+0 (secure 0 0 0) 0x7f2cd4014a20 con 0x7f2cdc101cb0 2024-02-18T11:38:43.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.374+0000 7f2ccb7fe640 1 --2- 172.21.15.12:0/149984442 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f2cc40a2750 0x7f2cc40a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:43.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.374+0000 7f2cda59c640 1 --2- 172.21.15.12:0/149984442 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f2cc40a2750 0x7f2cc40a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:43.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.374+0000 7f2ccb7fe640 1 -- 172.21.15.12:0/149984442 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f2cd40cd950 con 0x7f2cdc101cb0 2024-02-18T11:38:43.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.374+0000 7f2cda59c640 1 --2- 172.21.15.12:0/149984442 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f2cc40a2750 0x7f2cc40a4c10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2cd00058c0 tx=0x7f2cd0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:43.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.378+0000 7f2ccb7fe640 1 -- 172.21.15.12:0/149984442 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2cd404fd20 con 0x7f2cdc101cb0 2024-02-18T11:38:43.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.561+0000 7f2ce0902640 1 -- 172.21.15.12:0/149984442 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2cdc10bd50 con 0x7f2cdc101cb0 2024-02-18T11:38:43.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.563+0000 7f2ccb7fe640 1 -- 172.21.15.12:0/149984442 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+18466 (secure 0 0 0) 0x7f2cd4096c30 con 0x7f2cdc101cb0 2024-02-18T11:38:43.557 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:38:43.557 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":83,"fsid":"0c8bb0de-ce51-11ee-95ba-87774f69a715","created":"2024-02-18T11:30:49.110059+0000","modified":"2024-02-18T11:37:58.739798+0000","last_up_change":"2024-02-18T11:37:57.512337+0000","last_in_change":"2024-02-18T11:37:36.567969+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-18T11:34:15.281138+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"35f15ea6-deb8-48a1-a7dd-ab649ec13533","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":536522426},{"type":"v1","addr":"172.21.15.12:6803","nonce":536522426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":536522426},{"type":"v1","addr":"172.21.15.12:6805","nonce":536522426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":536522426},{"type":"v1","addr":"172.21.15.12:6809","nonce":536522426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":536522426},{"type":"v1","addr":"172.21.15.12:6807","nonce":536522426}]},"public_addr":"172.21.15.12:6803/536522426","cluster_addr":"172.21.15.12:6805/536522426","heartbeat_back_addr":"172.21.15.12:6809/536522426","heartbeat_front_addr":"172.21.15.12:6807/536522426","state":["exists","up"]},{"osd":1,"uuid":"2af937ef-d827-44e6-b805-70cffbfe1126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":4249111488},{"type":"v1","addr":"172.21.15.158:6801","nonce":4249111488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":4249111488},{"type":"v1","addr":"172.21.15.158:6803","nonce":4249111488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":4249111488},{"type":"v1","addr":"172.21.15.158:6807","nonce":4249111488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":4249111488},{"type":"v1","addr":"172.21.15.158:6805","nonce":4249111488}]},"public_addr":"172.21.15.158:6801/4249111488","cluster_addr":"172.21.15.158:6803/4249111488","heartbeat_back_addr":"172.21.15.158:6807/4249111488","heartbeat_front_addr":"172.21.15.158:6805/4249111488","state":["exists","up"]},{"osd":2,"uuid":"17b804dd-31df-45ae-a637-4a5f65bbaf26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":627976150},{"type":"v1","addr":"172.21.15.188:6801","nonce":627976150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":627976150},{"type":"v1","addr":"172.21.15.188:6803","nonce":627976150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":627976150},{"type":"v1","addr":"172.21.15.188:6807","nonce":627976150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":627976150},{"type":"v1","addr":"172.21.15.188:6805","nonce":627976150}]},"public_addr":"172.21.15.188:6801/627976150","cluster_addr":"172.21.15.188:6803/627976150","heartbeat_back_addr":"172.21.15.188:6807/627976150","heartbeat_front_addr":"172.21.15.188:6805/627976150","state":["exists","up"]},{"osd":3,"uuid":"3f9cc423-cdfa-4acf-aaa2-b58b32ccb9cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":3758512503},{"type":"v1","addr":"172.21.15.12:6811","nonce":3758512503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":3758512503},{"type":"v1","addr":"172.21.15.12:6813","nonce":3758512503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3758512503},{"type":"v1","addr":"172.21.15.12:6817","nonce":3758512503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":3758512503},{"type":"v1","addr":"172.21.15.12:6815","nonce":3758512503}]},"public_addr":"172.21.15.12:6811/3758512503","cluster_addr":"172.21.15.12:6813/3758512503","heartbeat_back_addr":"172.21.15.12:6817/3758512503","heartbeat_front_addr":"172.21.15.12:6815/3758512503","state":["exists","up"]},{"osd":4,"uuid":"a92e20ec-af1c-4ef4-a0cb-caba5fcc71aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":3892820997},{"type":"v1","addr":"172.21.15.158:6809","nonce":3892820997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":3892820997},{"type":"v1","addr":"172.21.15.158:6811","nonce":3892820997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":3892820997},{"type":"v1","addr":"172.21.15.158:6815","nonce":3892820997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":3892820997},{"type":"v1","addr":"172.21.15.158:6813","nonce":3892820997}]},"public_addr":"172.21.15.158:6809/3892820997","cluster_addr":"172.21.15.158:6811/3892820997","heartbeat_back_addr":"172.21.15.158:6815/3892820997","heartbeat_front_addr":"172.21.15.158:6813/3892820997","state":["exists","up"]},{"osd":5,"uuid":"d9c2e0c8-85f5-4ae2-bf3e-125c1be3a2db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3939495896},{"type":"v1","addr":"172.21.15.188:6809","nonce":3939495896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3939495896},{"type":"v1","addr":"172.21.15.188:6811","nonce":3939495896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3939495896},{"type":"v1","addr":"172.21.15.188:6815","nonce":3939495896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3939495896},{"type":"v1","addr":"172.21.15.188:6813","nonce":3939495896}]},"public_addr":"172.21.15.188:6809/3939495896","cluster_addr":"172.21.15.188:6811/3939495896","heartbeat_back_addr":"172.21.15.188:6815/3939495896","heartbeat_front_addr":"172.21.15.188:6813/3939495896","state":["exists","up"]},{"osd":6,"uuid":"4a523512-de6b-49f0-be81-00f2f00c1c10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":1206337767},{"type":"v1","addr":"172.21.15.12:6819","nonce":1206337767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":1206337767},{"type":"v1","addr":"172.21.15.12:6821","nonce":1206337767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":1206337767},{"type":"v1","addr":"172.21.15.12:6825","nonce":1206337767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":1206337767},{"type":"v1","addr":"172.21.15.12:6823","nonce":1206337767}]},"public_addr":"172.21.15.12:6819/1206337767","cluster_addr":"172.21.15.12:6821/1206337767","heartbeat_back_addr":"172.21.15.12:6825/1206337767","heartbeat_front_addr":"172.21.15.12:6823/1206337767","state":["exists","up"]},{"osd":7,"uuid":"363271a6-df93-44c9-b0e5-3d29768aa96c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":3283400032},{"type":"v1","addr":"172.21.15.158:6817","nonce":3283400032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":3283400032},{"type":"v1","addr":"172.21.15.158:6819","nonce":3283400032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":3283400032},{"type":"v1","addr":"172.21.15.158:6823","nonce":3283400032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":3283400032},{"type":"v1","addr":"172.21.15.158:6821","nonce":3283400032}]},"public_addr":"172.21.15.158:6817/3283400032","cluster_addr":"172.21.15.158:6819/3283400032","heartbeat_back_addr":"172.21.15.158:6823/3283400032","heartbeat_front_addr":"172.21.15.158:6821/3283400032","state":["exists","up"]},{"osd":8,"uuid":"9a089528-3e8e-45d3-9279-ddb54088b96a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2110170682},{"type":"v1","addr":"172.21.15.188:6817","nonce":2110170682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2110170682},{"type":"v1","addr":"172.21.15.188:6819","nonce":2110170682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":2110170682},{"type":"v1","addr":"172.21.15.188:6823","nonce":2110170682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2110170682},{"type":"v1","addr":"172.21.15.188:6821","nonce":2110170682}]},"public_addr":"172.21.15.188:6817/2110170682","cluster_addr":"172.21.15.188:6819/2110170682","heartbeat_back_addr":"172.21.15.188:6823/2110170682","heartbeat_front_addr":"172.21.15.188:6821/2110170682","state":["exists","up"]},{"osd":9,"uuid":"b12e2727-d649-4464-90e1-fafbc1754832","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1825928986},{"type":"v1","addr":"172.21.15.12:6827","nonce":1825928986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1825928986},{"type":"v1","addr":"172.21.15.12:6829","nonce":1825928986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":1825928986},{"type":"v1","addr":"172.21.15.12:6833","nonce":1825928986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1825928986},{"type":"v1","addr":"172.21.15.12:6831","nonce":1825928986}]},"public_addr":"172.21.15.12:6827/1825928986","cluster_addr":"172.21.15.12:6829/1825928986","heartbeat_back_addr":"172.21.15.12:6833/1825928986","heartbeat_front_addr":"172.21.15.12:6831/1825928986","state":["exists","up"]},{"osd":10,"uuid":"518b4cbf-9d90-4e9a-8ac9-b5cb9b3dee4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":3239052167},{"type":"v1","addr":"172.21.15.158:6825","nonce":3239052167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":3239052167},{"type":"v1","addr":"172.21.15.158:6827","nonce":3239052167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":3239052167},{"type":"v1","addr":"172.21.15.158:6831","nonce":3239052167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":3239052167},{"type":"v1","addr":"172.21.15.158:6829","nonce":3239052167}]},"public_addr":"172.21.15.158:6825/3239052167","cluster_addr":"172.21.15.158:6827/3239052167","heartbeat_back_addr":"172.21.15.158:6831/3239052167","heartbeat_front_addr":"172.21.15.158:6829/3239052167","state":["exists","up"]},{"osd":11,"uuid":"98aa3a0d-4b2f-46c8-9341-375d13794c3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1135773350},{"type":"v1","addr":"172.21.15.188:6825","nonce":1135773350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1135773350},{"type":"v1","addr":"172.21.15.188:6827","nonce":1135773350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1135773350},{"type":"v1","addr":"172.21.15.188:6831","nonce":1135773350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1135773350},{"type":"v1","addr":"172.21.15.188:6829","nonce":1135773350}]},"public_addr":"172.21.15.188:6825/1135773350","cluster_addr":"172.21.15.188:6827/1135773350","heartbeat_back_addr":"172.21.15.188:6831/1135773350","heartbeat_front_addr":"172.21.15.188:6829/1135773350","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:33:24.280879+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:33:47.859461+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:34:11.725283+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:34:35.707629+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:35:00.024197+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:35:24.499803+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:35:48.953870+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:36:13.642456+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:36:38.936457+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:37:04.021466+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:37:29.073178+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:37:54.863919+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:6801/4135274411":"2024-02-19T11:31:50.936928+0000","172.21.15.12:6800/4135274411":"2024-02-19T11:31:50.936928+0000","172.21.15.12:0/118428584":"2024-02-19T11:31:50.936928+0000","172.21.15.12:0/4276810915":"2024-02-19T11:31:50.936928+0000","172.21.15.12:0/3777545262":"2024-02-19T11:31:24.000830+0000","172.21.15.12:0/114707743":"2024-02-19T11:31:24.000830+0000","172.21.15.12:6800/3925204723":"2024-02-19T11:31:24.000830+0000","172.21.15.12:6801/3925204723":"2024-02-19T11:31:24.000830+0000","172.21.15.12:0/74881618":"2024-02-19T11:31:24.000830+0000","172.21.15.12:0/1614805375":"2024-02-19T11:31:50.936928+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-18T11:38:43.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.565+0000 7f2ce0902640 1 -- 172.21.15.12:0/149984442 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f2cc40a2750 msgr2=0x7f2cc40a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:43.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.565+0000 7f2ce0902640 1 --2- 172.21.15.12:0/149984442 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f2cc40a2750 0x7f2cc40a4c10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2cd00058c0 tx=0x7f2cd0005850 comp rx=0 tx=0).stop 2024-02-18T11:38:43.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.565+0000 7f2ce0902640 1 -- 172.21.15.12:0/149984442 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2cdc101cb0 msgr2=0x7f2cdc102c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:43.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.565+0000 7f2ce0902640 1 --2- 172.21.15.12:0/149984442 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2cdc101cb0 0x7f2cdc102c10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2cd400b9a0 tx=0x7f2cd400be60 comp rx=0 tx=0).stop 2024-02-18T11:38:43.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.565+0000 7f2ce0902640 1 -- 172.21.15.12:0/149984442 shutdown_connections 2024-02-18T11:38:43.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.565+0000 7f2ce0902640 1 --2- 172.21.15.12:0/149984442 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f2cc40a2750 0x7f2cc40a4c10 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:43.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.565+0000 7f2ce0902640 1 --2- 172.21.15.12:0/149984442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2cdc1012f0 0x7f2cdc1026d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:43.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.565+0000 7f2ce0902640 1 --2- 172.21.15.12:0/149984442 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2cdc101cb0 0x7f2cdc102c10 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:43.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.565+0000 7f2ce0902640 1 --2- 172.21.15.12:0/149984442 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2cdc100100 0x7f2cdc104090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:43.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.565+0000 7f2ce0902640 1 -- 172.21.15.12:0/149984442 >> 172.21.15.12:0/149984442 conn(0x7f2cdc0fb670 msgr2=0x7f2cdc0fd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:43.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.565+0000 7f2ce0902640 1 -- 172.21.15.12:0/149984442 shutdown_connections 2024-02-18T11:38:43.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.565+0000 7f2ce0902640 1 -- 172.21.15.12:0/149984442 wait complete. 2024-02-18T11:38:43.573 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-18T11:34:15.281138+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-18T11:38:43.573 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-18T11:38:43.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.686+0000 7fb461bc2640 1 -- 172.21.15.12:0/3383170149 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb45c102550 msgr2=0x7fb45c1029d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:43.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.686+0000 7fb461bc2640 1 --2- 172.21.15.12:0/3383170149 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb45c102550 0x7fb45c1029d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb44c009900 tx=0x7fb44c02bc30 comp rx=0 tx=0).stop 2024-02-18T11:38:43.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.686+0000 7fb461bc2640 1 -- 172.21.15.12:0/3383170149 shutdown_connections 2024-02-18T11:38:43.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.686+0000 7fb461bc2640 1 --2- 172.21.15.12:0/3383170149 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb45c102f10 0x7fb45c1097a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:43.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.686+0000 7fb461bc2640 1 --2- 172.21.15.12:0/3383170149 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb45c102550 0x7fb45c1029d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:43.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.686+0000 7fb461bc2640 1 --2- 172.21.15.12:0/3383170149 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb45c0ff4e0 0x7fb45c0ff900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:43.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.686+0000 7fb461bc2640 1 -- 172.21.15.12:0/3383170149 >> 172.21.15.12:0/3383170149 conn(0x7fb45c0fadf0 msgr2=0x7fb45c0fd230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:43.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.686+0000 7fb461bc2640 1 -- 172.21.15.12:0/3383170149 shutdown_connections 2024-02-18T11:38:43.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.686+0000 7fb461bc2640 1 -- 172.21.15.12:0/3383170149 wait complete. 2024-02-18T11:38:43.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.687+0000 7fb461bc2640 1 Processor -- start 2024-02-18T11:38:43.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.687+0000 7fb461bc2640 1 -- start start 2024-02-18T11:38:43.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.687+0000 7fb461bc2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb45c0ff4e0 0x7fb45c1972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:43.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.687+0000 7fb461bc2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb45c102550 0x7fb45c197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:43.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.687+0000 7fb461bc2640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb45c102f10 0x7fb45c19e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:43.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.687+0000 7fb461bc2640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb45c19edc0 con 0x7fb45c102550 2024-02-18T11:38:43.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.687+0000 7fb461bc2640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb45c19ef30 con 0x7fb45c102f10 2024-02-18T11:38:43.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.687+0000 7fb461bc2640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb45c19f0f0 con 0x7fb45c0ff4e0 2024-02-18T11:38:43.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.687+0000 7fb45affd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb45c102550 0x7fb45c197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:43.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.687+0000 7fb45affd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb45c102550 0x7fb45c197810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46624/0 (socket says 172.21.15.12:46624) 2024-02-18T11:38:43.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.687+0000 7fb45affd640 1 -- 172.21.15.12:0/1537402917 learned_addr learned my addr 172.21.15.12:0/1537402917 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:38:43.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.687+0000 7fb45b7fe640 1 --2- 172.21.15.12:0/1537402917 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb45c0ff4e0 0x7fb45c1972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:43.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.687+0000 7fb45bfff640 1 --2- 172.21.15.12:0/1537402917 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb45c102f10 0x7fb45c19e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:43.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.687+0000 7fb45affd640 1 -- 172.21.15.12:0/1537402917 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb45c0ff4e0 msgr2=0x7fb45c1972d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:43.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.687+0000 7fb45affd640 1 --2- 172.21.15.12:0/1537402917 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb45c0ff4e0 0x7fb45c1972d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:43.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.687+0000 7fb45affd640 1 -- 172.21.15.12:0/1537402917 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb45c102f10 msgr2=0x7fb45c19e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:43.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.687+0000 7fb45affd640 1 --2- 172.21.15.12:0/1537402917 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb45c102f10 0x7fb45c19e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:43.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.687+0000 7fb45affd640 1 -- 172.21.15.12:0/1537402917 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb44c009560 con 0x7fb45c102550 2024-02-18T11:38:43.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.688+0000 7fb45affd640 1 --2- 172.21.15.12:0/1537402917 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb45c102550 0x7fb45c197810 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb44c006010 tx=0x7fb44c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:43.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.688+0000 7fb458ff9640 1 -- 172.21.15.12:0/1537402917 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb44c0044c0 con 0x7fb45c102550 2024-02-18T11:38:43.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.688+0000 7fb458ff9640 1 -- 172.21.15.12:0/1537402917 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb44c0052d0 con 0x7fb45c102550 2024-02-18T11:38:43.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.688+0000 7fb461bc2640 1 -- 172.21.15.12:0/1537402917 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb45c19f370 con 0x7fb45c102550 2024-02-18T11:38:43.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.688+0000 7fb458ff9640 1 -- 172.21.15.12:0/1537402917 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb44c003e10 con 0x7fb45c102550 2024-02-18T11:38:43.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.688+0000 7fb461bc2640 1 -- 172.21.15.12:0/1537402917 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb45c19f880 con 0x7fb45c102550 2024-02-18T11:38:43.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.689+0000 7fb461bc2640 1 -- 172.21.15.12:0/1537402917 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb45c108680 con 0x7fb45c102550 2024-02-18T11:38:43.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.690+0000 7fb458ff9640 1 -- 172.21.15.12:0/1537402917 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) v1 ==== 137245+0+0 (secure 0 0 0) 0x7fb44c005440 con 0x7fb45c102550 2024-02-18T11:38:43.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.690+0000 7fb458ff9640 1 --2- 172.21.15.12:0/1537402917 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fb4440a2750 0x7fb4440a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:43.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.690+0000 7fb45b7fe640 1 --2- 172.21.15.12:0/1537402917 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fb4440a2750 0x7fb4440a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:43.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.690+0000 7fb458ff9640 1 -- 172.21.15.12:0/1537402917 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb44c0f03b0 con 0x7fb45c102550 2024-02-18T11:38:43.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.691+0000 7fb45b7fe640 1 --2- 172.21.15.12:0/1537402917 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fb4440a2750 0x7fb4440a4c10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb45c069530 tx=0x7fb450005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:43.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.695+0000 7fb458ff9640 1 -- 172.21.15.12:0/1537402917 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb44c0b95e0 con 0x7fb45c102550 2024-02-18T11:38:43.744 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:43 smithi158 ceph-mon[36626]: mds.? [v2:172.21.15.158:6838/1127200849,v1:172.21.15.158:6839/1127200849] up:boot 2024-02-18T11:38:43.744 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:43 smithi158 ceph-mon[36626]: fsmap 12 up:standby 2024-02-18T11:38:43.745 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:43 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-18T11:38:43.745 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:43 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:38:43.745 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:43 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/491148043' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-18T11:38:43.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:43 smithi188 ceph-mon[36925]: mds.? [v2:172.21.15.158:6838/1127200849,v1:172.21.15.158:6839/1127200849] up:boot 2024-02-18T11:38:43.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:43 smithi188 ceph-mon[36925]: fsmap 12 up:standby 2024-02-18T11:38:43.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:43 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-18T11:38:43.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:43 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:38:43.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:43 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/491148043' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-18T11:38:43.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.877+0000 7fb461bc2640 1 -- 172.21.15.12:0/1537402917 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb45c02ce80 con 0x7fb45c102550 2024-02-18T11:38:43.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.877+0000 7fb458ff9640 1 -- 172.21.15.12:0/1537402917 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb44c0bcca0 con 0x7fb45c102550 2024-02-18T11:38:43.872 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-02-18T11:38:43.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.879+0000 7fb461bc2640 1 -- 172.21.15.12:0/1537402917 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fb4440a2750 msgr2=0x7fb4440a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:43.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.879+0000 7fb461bc2640 1 --2- 172.21.15.12:0/1537402917 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fb4440a2750 0x7fb4440a4c10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb45c069530 tx=0x7fb450005f90 comp rx=0 tx=0).stop 2024-02-18T11:38:43.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.879+0000 7fb461bc2640 1 -- 172.21.15.12:0/1537402917 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb45c102550 msgr2=0x7fb45c197810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:43.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.879+0000 7fb461bc2640 1 --2- 172.21.15.12:0/1537402917 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb45c102550 0x7fb45c197810 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb44c006010 tx=0x7fb44c0040c0 comp rx=0 tx=0).stop 2024-02-18T11:38:43.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.879+0000 7fb461bc2640 1 -- 172.21.15.12:0/1537402917 shutdown_connections 2024-02-18T11:38:43.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.879+0000 7fb461bc2640 1 --2- 172.21.15.12:0/1537402917 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fb4440a2750 0x7fb4440a4c10 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:43.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.879+0000 7fb461bc2640 1 --2- 172.21.15.12:0/1537402917 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb45c102f10 0x7fb45c19e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:43.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.879+0000 7fb461bc2640 1 --2- 172.21.15.12:0/1537402917 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb45c102550 0x7fb45c197810 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:43.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.879+0000 7fb461bc2640 1 --2- 172.21.15.12:0/1537402917 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb45c0ff4e0 0x7fb45c1972d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:43.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.879+0000 7fb461bc2640 1 -- 172.21.15.12:0/1537402917 >> 172.21.15.12:0/1537402917 conn(0x7fb45c0fadf0 msgr2=0x7fb45c105850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:43.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.880+0000 7fb461bc2640 1 -- 172.21.15.12:0/1537402917 shutdown_connections 2024-02-18T11:38:43.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.880+0000 7fb461bc2640 1 -- 172.21.15.12:0/1537402917 wait complete. 2024-02-18T11:38:43.883 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:43 smithi012 ceph-mon[32460]: mds.? [v2:172.21.15.158:6838/1127200849,v1:172.21.15.158:6839/1127200849] up:boot 2024-02-18T11:38:43.884 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:43 smithi012 ceph-mon[32460]: fsmap 12 up:standby 2024-02-18T11:38:43.884 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:43 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-18T11:38:43.884 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:43 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:38:43.884 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:43 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/491148043' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-18T11:38:43.885 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-02-18T11:38:43.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.993+0000 7fd1341e9640 1 -- 172.21.15.12:0/3159558944 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd12c101c90 msgr2=0x7fd12c108520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:43.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.993+0000 7fd1341e9640 1 --2- 172.21.15.12:0/3159558944 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd12c101c90 0x7fd12c108520 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd11c009ca0 tx=0x7fd11c02c0f0 comp rx=0 tx=0).stop 2024-02-18T11:38:43.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.993+0000 7fd1341e9640 1 -- 172.21.15.12:0/3159558944 shutdown_connections 2024-02-18T11:38:43.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.993+0000 7fd1341e9640 1 --2- 172.21.15.12:0/3159558944 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd12c101c90 0x7fd12c108520 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:43.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.993+0000 7fd1341e9640 1 --2- 172.21.15.12:0/3159558944 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd12c1012d0 0x7fd12c101750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:43.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.993+0000 7fd1341e9640 1 --2- 172.21.15.12:0/3159558944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd12c1000e0 0x7fd12c100500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:43.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.993+0000 7fd1341e9640 1 -- 172.21.15.12:0/3159558944 >> 172.21.15.12:0/3159558944 conn(0x7fd12c0fb650 msgr2=0x7fd12c0fda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:43.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.993+0000 7fd1341e9640 1 -- 172.21.15.12:0/3159558944 shutdown_connections 2024-02-18T11:38:43.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.993+0000 7fd1341e9640 1 -- 172.21.15.12:0/3159558944 wait complete. 2024-02-18T11:38:43.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.993+0000 7fd1341e9640 1 Processor -- start 2024-02-18T11:38:43.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.993+0000 7fd1341e9640 1 -- start start 2024-02-18T11:38:43.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.994+0000 7fd1341e9640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd12c1000e0 0x7fd12c197220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:43.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.994+0000 7fd1341e9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd12c1012d0 0x7fd12c197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:43.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.994+0000 7fd1341e9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd12c101c90 0x7fd12c19e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:43.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.994+0000 7fd1341e9640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd12c19ed10 con 0x7fd12c101c90 2024-02-18T11:38:43.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.994+0000 7fd1341e9640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd12c19ee80 con 0x7fd12c1000e0 2024-02-18T11:38:43.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.994+0000 7fd1341e9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd12c19f040 con 0x7fd12c1012d0 2024-02-18T11:38:43.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.994+0000 7fd132786640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd12c101c90 0x7fd12c19e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:43.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.994+0000 7fd132786640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd12c101c90 0x7fd12c19e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46632/0 (socket says 172.21.15.12:46632) 2024-02-18T11:38:43.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.994+0000 7fd132786640 1 -- 172.21.15.12:0/2524047199 learned_addr learned my addr 172.21.15.12:0/2524047199 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:38:43.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.994+0000 7fd131784640 1 --2- 172.21.15.12:0/2524047199 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd12c1012d0 0x7fd12c197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:43.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.994+0000 7fd131f85640 1 --2- 172.21.15.12:0/2524047199 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd12c1000e0 0x7fd12c197220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:43.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.994+0000 7fd132786640 1 -- 172.21.15.12:0/2524047199 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd12c1012d0 msgr2=0x7fd12c197760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:43.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.994+0000 7fd132786640 1 --2- 172.21.15.12:0/2524047199 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd12c1012d0 0x7fd12c197760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:43.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.994+0000 7fd132786640 1 -- 172.21.15.12:0/2524047199 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd12c1000e0 msgr2=0x7fd12c197220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:43.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.994+0000 7fd132786640 1 --2- 172.21.15.12:0/2524047199 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd12c1000e0 0x7fd12c197220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:43.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.994+0000 7fd132786640 1 -- 172.21.15.12:0/2524047199 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd11c009900 con 0x7fd12c101c90 2024-02-18T11:38:43.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.994+0000 7fd132786640 1 --2- 172.21.15.12:0/2524047199 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd12c101c90 0x7fd12c19e7d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd11c037040 tx=0x7fd11c034a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:43.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.994+0000 7fd12affd640 1 -- 172.21.15.12:0/2524047199 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd11c0329b0 con 0x7fd12c101c90 2024-02-18T11:38:43.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.994+0000 7fd12affd640 1 -- 172.21.15.12:0/2524047199 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd11c003e50 con 0x7fd12c101c90 2024-02-18T11:38:43.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.994+0000 7fd1341e9640 1 -- 172.21.15.12:0/2524047199 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd12c19f2c0 con 0x7fd12c101c90 2024-02-18T11:38:43.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.994+0000 7fd12affd640 1 -- 172.21.15.12:0/2524047199 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd11c040450 con 0x7fd12c101c90 2024-02-18T11:38:43.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.994+0000 7fd1341e9640 1 -- 172.21.15.12:0/2524047199 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd12c19f7c0 con 0x7fd12c101c90 2024-02-18T11:38:43.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.995+0000 7fd1341e9640 1 -- 172.21.15.12:0/2524047199 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd12c1073a0 con 0x7fd12c101c90 2024-02-18T11:38:43.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.996+0000 7fd12affd640 1 -- 172.21.15.12:0/2524047199 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) v1 ==== 137245+0+0 (secure 0 0 0) 0x7fd11c00a040 con 0x7fd12c101c90 2024-02-18T11:38:43.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.997+0000 7fd12affd640 1 --2- 172.21.15.12:0/2524047199 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fd1140a27d0 0x7fd1140a4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:43.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.997+0000 7fd131f85640 1 --2- 172.21.15.12:0/2524047199 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fd1140a27d0 0x7fd1140a4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:43.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.997+0000 7fd12affd640 1 -- 172.21.15.12:0/2524047199 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd11c0f1240 con 0x7fd12c101c90 2024-02-18T11:38:43.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:43.998+0000 7fd131f85640 1 --2- 172.21.15.12:0/2524047199 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fd1140a27d0 0x7fd1140a4c90 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd124005fd0 tx=0x7fd12400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:43.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:44.001+0000 7fd12affd640 1 -- 172.21.15.12:0/2524047199 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd11c0ba4f0 con 0x7fd12c101c90 2024-02-18T11:38:44.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:44.212+0000 7fd1341e9640 1 -- 172.21.15.12:0/2524047199 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fd12c02ce80 con 0x7fd12c101c90 2024-02-18T11:38:44.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:44.213+0000 7fd12affd640 1 -- 172.21.15.12:0/2524047199 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v83) v1 ==== 84+0+65 (secure 0 0 0) 0x7fd11c0bdbb0 con 0x7fd12c101c90 2024-02-18T11:38:44.207 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:38:44.207 INFO:teuthology.orchestra.run.smithi012.stdout:[ 2024-02-18T11:38:44.207 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:38:44.207 INFO:teuthology.orchestra.run.smithi012.stdout: "poolnum": 1, 2024-02-18T11:38:44.208 INFO:teuthology.orchestra.run.smithi012.stdout: "poolname": ".mgr" 2024-02-18T11:38:44.208 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:38:44.208 INFO:teuthology.orchestra.run.smithi012.stdout:] 2024-02-18T11:38:44.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:44.215+0000 7fd1341e9640 1 -- 172.21.15.12:0/2524047199 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fd1140a27d0 msgr2=0x7fd1140a4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:44.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:44.215+0000 7fd1341e9640 1 --2- 172.21.15.12:0/2524047199 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fd1140a27d0 0x7fd1140a4c90 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd124005fd0 tx=0x7fd12400b410 comp rx=0 tx=0).stop 2024-02-18T11:38:44.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:44.215+0000 7fd1341e9640 1 -- 172.21.15.12:0/2524047199 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd12c101c90 msgr2=0x7fd12c19e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:44.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:44.215+0000 7fd1341e9640 1 --2- 172.21.15.12:0/2524047199 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd12c101c90 0x7fd12c19e7d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd11c037040 tx=0x7fd11c034a90 comp rx=0 tx=0).stop 2024-02-18T11:38:44.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:44.215+0000 7fd1341e9640 1 -- 172.21.15.12:0/2524047199 shutdown_connections 2024-02-18T11:38:44.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:44.215+0000 7fd1341e9640 1 --2- 172.21.15.12:0/2524047199 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fd1140a27d0 0x7fd1140a4c90 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:44.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:44.215+0000 7fd1341e9640 1 --2- 172.21.15.12:0/2524047199 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd12c1012d0 0x7fd12c197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:44.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:44.215+0000 7fd1341e9640 1 --2- 172.21.15.12:0/2524047199 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd12c101c90 0x7fd12c19e7d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:44.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:44.215+0000 7fd1341e9640 1 --2- 172.21.15.12:0/2524047199 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd12c1000e0 0x7fd12c197220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:44.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:44.215+0000 7fd1341e9640 1 -- 172.21.15.12:0/2524047199 >> 172.21.15.12:0/2524047199 conn(0x7fd12c0fb650 msgr2=0x7fd12c104570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:44.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:44.215+0000 7fd1341e9640 1 -- 172.21.15.12:0/2524047199 shutdown_connections 2024-02-18T11:38:44.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:44.215+0000 7fd1341e9640 1 -- 172.21.15.12:0/2524047199 wait complete. 2024-02-18T11:38:44.221 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-02-18T11:38:44.221 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-02-18T11:38:44.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:44.352+0000 7fbae16d4640 1 -- 172.21.15.12:0/91528530 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbadc1023c0 msgr2=0x7fbadc102840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:44.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:44.352+0000 7fbae16d4640 1 --2- 172.21.15.12:0/91528530 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbadc1023c0 0x7fbadc102840 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbad0009900 tx=0x7fbad002bc30 comp rx=0 tx=0).stop 2024-02-18T11:38:44.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:44.352+0000 7fbae16d4640 1 -- 172.21.15.12:0/91528530 shutdown_connections 2024-02-18T11:38:44.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:44.352+0000 7fbae16d4640 1 --2- 172.21.15.12:0/91528530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbadc102d80 0x7fbadc109610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:44.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:44.352+0000 7fbae16d4640 1 --2- 172.21.15.12:0/91528530 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbadc1023c0 0x7fbadc102840 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:44.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:44.352+0000 7fbae16d4640 1 --2- 172.21.15.12:0/91528530 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbadc1011d0 0x7fbadc1015f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:44.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:44.352+0000 7fbae16d4640 1 -- 172.21.15.12:0/91528530 >> 172.21.15.12:0/91528530 conn(0x7fbadc0fc760 msgr2=0x7fbadc0feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:44.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:44.352+0000 7fbae16d4640 1 -- 172.21.15.12:0/91528530 shutdown_connections 2024-02-18T11:38:44.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:44.352+0000 7fbae16d4640 1 -- 172.21.15.12:0/91528530 wait complete. 2024-02-18T11:38:44.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:44.353+0000 7fbae16d4640 1 Processor -- start 2024-02-18T11:38:44.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:44.353+0000 7fbae16d4640 1 -- start start 2024-02-18T11:38:44.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:44.353+0000 7fbae16d4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbadc1011d0 0x7fbadc1994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:44.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:44.353+0000 7fbae16d4640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbadc1023c0 0x7fbadc1999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:44.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:44.353+0000 7fbae16d4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbadc102d80 0x7fbadc1a0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:44.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:44.353+0000 7fbae16d4640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbadc1a0fa0 con 0x7fbadc1011d0 2024-02-18T11:38:44.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:44.353+0000 7fbae16d4640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbadc1a1110 con 0x7fbadc1023c0 2024-02-18T11:38:44.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:44.353+0000 7fbae16d4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbadc1a12d0 con 0x7fbadc102d80 2024-02-18T11:38:44.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:44.353+0000 7fbadaffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbadc1011d0 0x7fbadc1994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:44.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:44.353+0000 7fbadaffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbadc1011d0 0x7fbadc1994b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46648/0 (socket says 172.21.15.12:46648) 2024-02-18T11:38:44.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:44.353+0000 7fbadaffd640 1 -- 172.21.15.12:0/3961322531 learned_addr learned my addr 172.21.15.12:0/3961322531 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:38:44.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:44.353+0000 7fbadb7fe640 1 --2- 172.21.15.12:0/3961322531 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbadc102d80 0x7fbadc1a0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:44.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:44.353+0000 7fbada7fc640 1 --2- 172.21.15.12:0/3961322531 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbadc1023c0 0x7fbadc1999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:44.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:44.353+0000 7fbadaffd640 1 -- 172.21.15.12:0/3961322531 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbadc102d80 msgr2=0x7fbadc1a0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:44.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:44.353+0000 7fbadaffd640 1 --2- 172.21.15.12:0/3961322531 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbadc102d80 0x7fbadc1a0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:44.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:44.353+0000 7fbadaffd640 1 -- 172.21.15.12:0/3961322531 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbadc1023c0 msgr2=0x7fbadc1999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:44.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:44.353+0000 7fbadaffd640 1 --2- 172.21.15.12:0/3961322531 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbadc1023c0 0x7fbadc1999f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:44.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:44.353+0000 7fbadaffd640 1 -- 172.21.15.12:0/3961322531 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbad0009560 con 0x7fbadc1011d0 2024-02-18T11:38:44.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:44.354+0000 7fbadaffd640 1 --2- 172.21.15.12:0/3961322531 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbadc1011d0 0x7fbadc1994b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbacc00d770 tx=0x7fbacc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:44.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:44.354+0000 7fbac3fff640 1 -- 172.21.15.12:0/3961322531 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbacc009c30 con 0x7fbadc1011d0 2024-02-18T11:38:44.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:44.354+0000 7fbae16d4640 1 -- 172.21.15.12:0/3961322531 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbadc1a15b0 con 0x7fbadc1011d0 2024-02-18T11:38:44.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:44.354+0000 7fbac3fff640 1 -- 172.21.15.12:0/3961322531 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbacc00de80 con 0x7fbadc1011d0 2024-02-18T11:38:44.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:44.354+0000 7fbac3fff640 1 -- 172.21.15.12:0/3961322531 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbacc010460 con 0x7fbadc1011d0 2024-02-18T11:38:44.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:44.354+0000 7fbae16d4640 1 -- 172.21.15.12:0/3961322531 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbadc1a1ae0 con 0x7fbadc1011d0 2024-02-18T11:38:44.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:44.355+0000 7fbae16d4640 1 -- 172.21.15.12:0/3961322531 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbadc10cbe0 con 0x7fbadc1011d0 2024-02-18T11:38:44.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:44.356+0000 7fbac3fff640 1 -- 172.21.15.12:0/3961322531 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) v1 ==== 137245+0+0 (secure 0 0 0) 0x7fbacc003f60 con 0x7fbadc1011d0 2024-02-18T11:38:44.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:44.357+0000 7fbac3fff640 1 --2- 172.21.15.12:0/3961322531 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fbac40a2750 0x7fbac40a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:44.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:44.357+0000 7fbada7fc640 1 --2- 172.21.15.12:0/3961322531 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fbac40a2750 0x7fbac40a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:44.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:44.357+0000 7fbac3fff640 1 -- 172.21.15.12:0/3961322531 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fbacc0cdd60 con 0x7fbadc1011d0 2024-02-18T11:38:44.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:44.357+0000 7fbada7fc640 1 --2- 172.21.15.12:0/3961322531 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fbac40a2750 0x7fbac40a4c10 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbad0009230 tx=0x7fbad0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:44.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:44.361+0000 7fbac3fff640 1 -- 172.21.15.12:0/3961322531 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbacc097040 con 0x7fbadc1011d0 2024-02-18T11:38:44.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:44.553+0000 7fbae16d4640 1 -- 172.21.15.12:0/3961322531 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fbadc10cd90 con 0x7fbadc1011d0 2024-02-18T11:38:45.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:44 smithi188 ceph-mon[36925]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-18T11:38:45.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:44 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/149984442' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-18T11:38:45.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:44 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/1537402917' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-18T11:38:45.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:44 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:45.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:44 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:45.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:44 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:45.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:44 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/2524047199' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-02-18T11:38:45.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:44 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:45.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:44 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:45.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:44 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:45.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:44 smithi012 ceph-mon[32460]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-18T11:38:45.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:44 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/149984442' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-18T11:38:45.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:44 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/1537402917' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-18T11:38:45.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:44 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:45.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:44 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:45.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:44 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:45.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:44 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/2524047199' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-02-18T11:38:45.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:44 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:45.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:44 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:45.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:44 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:45.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:44 smithi158 ceph-mon[36626]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-18T11:38:45.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:44 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/149984442' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-18T11:38:45.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:44 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/1537402917' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-18T11:38:45.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:44 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:45.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:44 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:45.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:44 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:45.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:44 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/2524047199' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-02-18T11:38:45.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:44 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:45.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:44 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:45.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:44 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:45.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:45.511+0000 7fbac3fff640 1 -- 172.21.15.12:0/3961322531 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v84) v1 ==== 138+0+0 (secure 0 0 0) 0x7fbacc0200b0 con 0x7fbadc1011d0 2024-02-18T11:38:45.505 INFO:teuthology.orchestra.run.smithi012.stderr:pool 'cephfs_metadata' created 2024-02-18T11:38:45.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:45.513+0000 7fbae16d4640 1 -- 172.21.15.12:0/3961322531 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fbac40a2750 msgr2=0x7fbac40a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:45.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:45.513+0000 7fbae16d4640 1 --2- 172.21.15.12:0/3961322531 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fbac40a2750 0x7fbac40a4c10 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbad0009230 tx=0x7fbad0005fd0 comp rx=0 tx=0).stop 2024-02-18T11:38:45.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:45.513+0000 7fbae16d4640 1 -- 172.21.15.12:0/3961322531 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbadc1011d0 msgr2=0x7fbadc1994b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:45.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:45.513+0000 7fbae16d4640 1 --2- 172.21.15.12:0/3961322531 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbadc1011d0 0x7fbadc1994b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbacc00d770 tx=0x7fbacc00dc30 comp rx=0 tx=0).stop 2024-02-18T11:38:45.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:45.513+0000 7fbae16d4640 1 -- 172.21.15.12:0/3961322531 shutdown_connections 2024-02-18T11:38:45.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:45.513+0000 7fbae16d4640 1 --2- 172.21.15.12:0/3961322531 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fbac40a2750 0x7fbac40a4c10 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:45.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:45.513+0000 7fbae16d4640 1 --2- 172.21.15.12:0/3961322531 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbadc102d80 0x7fbadc1a0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:45.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:45.513+0000 7fbae16d4640 1 --2- 172.21.15.12:0/3961322531 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbadc1023c0 0x7fbadc1999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:45.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:45.513+0000 7fbae16d4640 1 --2- 172.21.15.12:0/3961322531 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbadc1011d0 0x7fbadc1994b0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:45.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:45.513+0000 7fbae16d4640 1 -- 172.21.15.12:0/3961322531 >> 172.21.15.12:0/3961322531 conn(0x7fbadc0fc760 msgr2=0x7fbadc0733f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:45.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:45.514+0000 7fbae16d4640 1 -- 172.21.15.12:0/3961322531 shutdown_connections 2024-02-18T11:38:45.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:45.514+0000 7fbae16d4640 1 -- 172.21.15.12:0/3961322531 wait complete. 2024-02-18T11:38:45.522 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-02-18T11:38:45.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:45.638+0000 7f206d08d640 1 -- 172.21.15.12:0/975843943 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20681027e0 msgr2=0x7f2068102c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:45.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:45.638+0000 7f206d08d640 1 --2- 172.21.15.12:0/975843943 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20681027e0 0x7f2068102c00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2058009900 tx=0x7f205802bc30 comp rx=0 tx=0).stop 2024-02-18T11:38:45.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:45.638+0000 7f206d08d640 1 -- 172.21.15.12:0/975843943 shutdown_connections 2024-02-18T11:38:45.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:45.638+0000 7f206d08d640 1 --2- 172.21.15.12:0/975843943 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20680ff250 0x7f20680ff6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:45.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:45.638+0000 7f206d08d640 1 --2- 172.21.15.12:0/975843943 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f20680ffc10 0x7f20681097c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:45.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:45.638+0000 7f206d08d640 1 --2- 172.21.15.12:0/975843943 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20681027e0 0x7f2068102c00 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:45.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:45.638+0000 7f206d08d640 1 -- 172.21.15.12:0/975843943 >> 172.21.15.12:0/975843943 conn(0x7f20680fadd0 msgr2=0x7f20680fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:45.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:45.638+0000 7f206d08d640 1 -- 172.21.15.12:0/975843943 shutdown_connections 2024-02-18T11:38:45.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:45.638+0000 7f206d08d640 1 -- 172.21.15.12:0/975843943 wait complete. 2024-02-18T11:38:45.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:45.639+0000 7f206d08d640 1 Processor -- start 2024-02-18T11:38:45.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:45.639+0000 7f206d08d640 1 -- start start 2024-02-18T11:38:45.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:45.639+0000 7f206d08d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20680ff250 0x7f20681972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:45.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:45.639+0000 7f206d08d640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f20680ffc10 0x7f20681977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:45.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:45.639+0000 7f206d08d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20681027e0 0x7f206819e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:45.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:45.639+0000 7f206d08d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f206819eda0 con 0x7f20680ff250 2024-02-18T11:38:45.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:45.639+0000 7f206d08d640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f206819ef10 con 0x7f20680ffc10 2024-02-18T11:38:45.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:45.639+0000 7f206d08d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f206819f0d0 con 0x7f20681027e0 2024-02-18T11:38:45.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:45.639+0000 7f2066d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20680ff250 0x7f20681972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:45.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:45.639+0000 7f2066d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20680ff250 0x7f20681972b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46668/0 (socket says 172.21.15.12:46668) 2024-02-18T11:38:45.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:45.639+0000 7f2066d9d640 1 -- 172.21.15.12:0/1570957829 learned_addr learned my addr 172.21.15.12:0/1570957829 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:38:45.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:45.639+0000 7f206759e640 1 --2- 172.21.15.12:0/1570957829 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20681027e0 0x7f206819e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:45.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:45.639+0000 7f205e59c640 1 --2- 172.21.15.12:0/1570957829 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f20680ffc10 0x7f20681977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:45.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:45.639+0000 7f2066d9d640 1 -- 172.21.15.12:0/1570957829 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20681027e0 msgr2=0x7f206819e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:45.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:45.639+0000 7f2066d9d640 1 --2- 172.21.15.12:0/1570957829 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20681027e0 0x7f206819e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:45.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:45.639+0000 7f2066d9d640 1 -- 172.21.15.12:0/1570957829 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f20680ffc10 msgr2=0x7f20681977f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:45.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:45.639+0000 7f2066d9d640 1 --2- 172.21.15.12:0/1570957829 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f20680ffc10 0x7f20681977f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:45.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:45.639+0000 7f2066d9d640 1 -- 172.21.15.12:0/1570957829 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2058009560 con 0x7f20680ff250 2024-02-18T11:38:45.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:45.639+0000 7f205e59c640 1 --2- 172.21.15.12:0/1570957829 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f20680ffc10 0x7f20681977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:38:45.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:45.640+0000 7f2066d9d640 1 --2- 172.21.15.12:0/1570957829 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20680ff250 0x7f20681972b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2058000c00 tx=0x7f2058004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:45.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:45.640+0000 7f2064d99640 1 -- 172.21.15.12:0/1570957829 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2058004df0 con 0x7f20680ff250 2024-02-18T11:38:45.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:45.640+0000 7f2064d99640 1 -- 172.21.15.12:0/1570957829 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2058005430 con 0x7f20680ff250 2024-02-18T11:38:45.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:45.640+0000 7f206d08d640 1 -- 172.21.15.12:0/1570957829 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f206819f350 con 0x7f20680ff250 2024-02-18T11:38:45.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:45.640+0000 7f2064d99640 1 -- 172.21.15.12:0/1570957829 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f205803e450 con 0x7f20680ff250 2024-02-18T11:38:45.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:45.640+0000 7f206d08d640 1 -- 172.21.15.12:0/1570957829 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f206819f860 con 0x7f20680ff250 2024-02-18T11:38:45.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:45.641+0000 7f206d08d640 1 -- 172.21.15.12:0/1570957829 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2068108720 con 0x7f20680ff250 2024-02-18T11:38:45.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:45.642+0000 7f2064d99640 1 -- 172.21.15.12:0/1570957829 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) v1 ==== 137245+0+0 (secure 0 0 0) 0x7f2058003680 con 0x7f20680ff250 2024-02-18T11:38:45.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:45.643+0000 7f2064d99640 1 --2- 172.21.15.12:0/1570957829 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f20540a2750 0x7f20540a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:45.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:45.643+0000 7f205e59c640 1 --2- 172.21.15.12:0/1570957829 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f20540a2750 0x7f20540a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:45.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:45.643+0000 7f2064d99640 1 -- 172.21.15.12:0/1570957829 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7497+0+0 (secure 0 0 0) 0x7f20580f0d40 con 0x7f20680ff250 2024-02-18T11:38:45.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:45.643+0000 7f205e59c640 1 --2- 172.21.15.12:0/1570957829 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f20540a2750 0x7f20540a4c10 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2050005d00 tx=0x7f2050005ed0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:45.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:45.647+0000 7f2064d99640 1 -- 172.21.15.12:0/1570957829 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f20580b9e00 con 0x7f20680ff250 2024-02-18T11:38:45.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:45.831+0000 7f206d08d640 1 -- 172.21.15.12:0/1570957829 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f206802ce80 con 0x7f20680ff250 2024-02-18T11:38:45.948 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:45 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/3961322531' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-02-18T11:38:45.948 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:45 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:38:45.948 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:45 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:38:45.948 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:45 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:45.948 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:45 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:38:45.948 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:45 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/3961322531' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-02-18T11:38:45.948 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:45 smithi188 ceph-mon[36925]: osdmap e84: 12 total, 12 up, 12 in 2024-02-18T11:38:45.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:45 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/3961322531' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-02-18T11:38:45.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:45 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:38:45.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:45 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:38:45.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:45 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:45.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:45 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:38:45.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:45 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/3961322531' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-02-18T11:38:45.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:45 smithi158 ceph-mon[36626]: osdmap e84: 12 total, 12 up, 12 in 2024-02-18T11:38:46.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:45 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/3961322531' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-02-18T11:38:46.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:45 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:38:46.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:45 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:38:46.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:45 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:46.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:45 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:38:46.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:45 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/3961322531' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-02-18T11:38:46.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:45 smithi012 ceph-mon[32460]: osdmap e84: 12 total, 12 up, 12 in 2024-02-18T11:38:46.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:46.513+0000 7f2064d99640 1 -- 172.21.15.12:0/1570957829 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v85) v1 ==== 170+0+0 (secure 0 0 0) 0x7f20580bd4c0 con 0x7f20680ff250 2024-02-18T11:38:46.507 INFO:teuthology.orchestra.run.smithi012.stderr:pool 'cephfs_data' created 2024-02-18T11:38:46.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:46.516+0000 7f206d08d640 1 -- 172.21.15.12:0/1570957829 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f20540a2750 msgr2=0x7f20540a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:46.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:46.516+0000 7f206d08d640 1 --2- 172.21.15.12:0/1570957829 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f20540a2750 0x7f20540a4c10 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2050005d00 tx=0x7f2050005ed0 comp rx=0 tx=0).stop 2024-02-18T11:38:46.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:46.516+0000 7f206d08d640 1 -- 172.21.15.12:0/1570957829 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20680ff250 msgr2=0x7f20681972b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:46.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:46.516+0000 7f206d08d640 1 --2- 172.21.15.12:0/1570957829 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20680ff250 0x7f20681972b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2058000c00 tx=0x7f2058004270 comp rx=0 tx=0).stop 2024-02-18T11:38:46.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:46.518+0000 7f206d08d640 1 -- 172.21.15.12:0/1570957829 shutdown_connections 2024-02-18T11:38:46.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:46.518+0000 7f206d08d640 1 --2- 172.21.15.12:0/1570957829 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f20540a2750 0x7f20540a4c10 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:46.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:46.518+0000 7f206d08d640 1 --2- 172.21.15.12:0/1570957829 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20681027e0 0x7f206819e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:46.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:46.518+0000 7f206d08d640 1 --2- 172.21.15.12:0/1570957829 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f20680ffc10 0x7f20681977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:46.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:46.518+0000 7f206d08d640 1 --2- 172.21.15.12:0/1570957829 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20680ff250 0x7f20681972b0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:46.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:46.518+0000 7f206d08d640 1 -- 172.21.15.12:0/1570957829 >> 172.21.15.12:0/1570957829 conn(0x7f20680fadd0 msgr2=0x7f20681058f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:46.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:46.520+0000 7f206d08d640 1 -- 172.21.15.12:0/1570957829 shutdown_connections 2024-02-18T11:38:46.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:46.520+0000 7f206d08d640 1 -- 172.21.15.12:0/1570957829 wait complete. 2024-02-18T11:38:46.535 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-02-18T11:38:46.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:46.654+0000 7f51283e2640 1 -- 172.21.15.12:0/2920000240 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5120100cd0 msgr2=0x7f5120108570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:46.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:46.654+0000 7f51283e2640 1 --2- 172.21.15.12:0/2920000240 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5120100cd0 0x7f5120108570 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5118009ca0 tx=0x7f511802c0f0 comp rx=0 tx=0).stop 2024-02-18T11:38:46.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:46.655+0000 7f51283e2640 1 -- 172.21.15.12:0/2920000240 shutdown_connections 2024-02-18T11:38:46.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:46.655+0000 7f51283e2640 1 --2- 172.21.15.12:0/2920000240 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5120100310 0x7f5120100790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:46.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:46.655+0000 7f51283e2640 1 --2- 172.21.15.12:0/2920000240 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5120100cd0 0x7f5120108570 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:46.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:46.655+0000 7f51283e2640 1 --2- 172.21.15.12:0/2920000240 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f51201016c0 0x7f5120101ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:46.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:46.655+0000 7f51283e2640 1 -- 172.21.15.12:0/2920000240 >> 172.21.15.12:0/2920000240 conn(0x7f51200fadc0 msgr2=0x7f51200fd220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:46.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:46.655+0000 7f51283e2640 1 -- 172.21.15.12:0/2920000240 shutdown_connections 2024-02-18T11:38:46.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:46.655+0000 7f51283e2640 1 -- 172.21.15.12:0/2920000240 wait complete. 2024-02-18T11:38:46.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:46.655+0000 7f51283e2640 1 Processor -- start 2024-02-18T11:38:46.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:46.655+0000 7f51283e2640 1 -- start start 2024-02-18T11:38:46.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:46.656+0000 7f51283e2640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5120100310 0x7f5120197220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:46.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:46.656+0000 7f51283e2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5120100cd0 0x7f5120197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:46.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:46.656+0000 7f51283e2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f51201016c0 0x7f512019e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:46.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:46.656+0000 7f51283e2640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f512019ed10 con 0x7f51201016c0 2024-02-18T11:38:46.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:46.656+0000 7f51283e2640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f512019ee80 con 0x7f5120100310 2024-02-18T11:38:46.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:46.656+0000 7f51283e2640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f512019f040 con 0x7f5120100cd0 2024-02-18T11:38:46.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:46.656+0000 7f512697f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f51201016c0 0x7f512019e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:46.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:46.656+0000 7f512697f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f51201016c0 0x7f512019e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46690/0 (socket says 172.21.15.12:46690) 2024-02-18T11:38:46.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:46.656+0000 7f512697f640 1 -- 172.21.15.12:0/4054359771 learned_addr learned my addr 172.21.15.12:0/4054359771 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:38:46.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:46.656+0000 7f512617e640 1 --2- 172.21.15.12:0/4054359771 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5120100310 0x7f5120197220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:46.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:46.656+0000 7f512597d640 1 --2- 172.21.15.12:0/4054359771 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5120100cd0 0x7f5120197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:46.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:46.656+0000 7f512697f640 1 -- 172.21.15.12:0/4054359771 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5120100cd0 msgr2=0x7f5120197760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:46.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:46.656+0000 7f512697f640 1 --2- 172.21.15.12:0/4054359771 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5120100cd0 0x7f5120197760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:46.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:46.656+0000 7f512697f640 1 -- 172.21.15.12:0/4054359771 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5120100310 msgr2=0x7f5120197220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:46.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:46.656+0000 7f512697f640 1 --2- 172.21.15.12:0/4054359771 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5120100310 0x7f5120197220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:46.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:46.656+0000 7f512697f640 1 -- 172.21.15.12:0/4054359771 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5118009900 con 0x7f51201016c0 2024-02-18T11:38:46.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:46.656+0000 7f512697f640 1 --2- 172.21.15.12:0/4054359771 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f51201016c0 0x7f512019e7d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5118037040 tx=0x7f5118034a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:46.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:46.656+0000 7f51177fe640 1 -- 172.21.15.12:0/4054359771 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51180329b0 con 0x7f51201016c0 2024-02-18T11:38:46.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:46.657+0000 7f51283e2640 1 -- 172.21.15.12:0/4054359771 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f512019f2c0 con 0x7f51201016c0 2024-02-18T11:38:46.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:46.657+0000 7f51177fe640 1 -- 172.21.15.12:0/4054359771 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5118003e50 con 0x7f51201016c0 2024-02-18T11:38:46.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:46.657+0000 7f51177fe640 1 -- 172.21.15.12:0/4054359771 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5118040450 con 0x7f51201016c0 2024-02-18T11:38:46.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:46.657+0000 7f51283e2640 1 -- 172.21.15.12:0/4054359771 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f512019f7c0 con 0x7f51201016c0 2024-02-18T11:38:46.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:46.658+0000 7f51283e2640 1 -- 172.21.15.12:0/4054359771 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f512010baf0 con 0x7f51201016c0 2024-02-18T11:38:46.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:46.659+0000 7f51177fe640 1 -- 172.21.15.12:0/4054359771 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) v1 ==== 137245+0+0 (secure 0 0 0) 0x7f511800a040 con 0x7f51201016c0 2024-02-18T11:38:46.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:46.660+0000 7f51177fe640 1 --2- 172.21.15.12:0/4054359771 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f510c0a2750 0x7f510c0a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:46.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:46.660+0000 7f512617e640 1 --2- 172.21.15.12:0/4054359771 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f510c0a2750 0x7f510c0a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:46.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:46.660+0000 7f51177fe640 1 -- 172.21.15.12:0/4054359771 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7879+0+0 (secure 0 0 0) 0x7f51180f15b0 con 0x7f51201016c0 2024-02-18T11:38:46.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:46.660+0000 7f512617e640 1 --2- 172.21.15.12:0/4054359771 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f510c0a2750 0x7f510c0a4c10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f51200692a0 tx=0x7f511000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:46.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:46.663+0000 7f51177fe640 1 -- 172.21.15.12:0/4054359771 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f51180ba5f0 con 0x7f51201016c0 2024-02-18T11:38:46.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:46.886+0000 7f51283e2640 1 -- 172.21.15.12:0/4054359771 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f512010bc80 con 0x7f51201016c0 2024-02-18T11:38:46.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:46 smithi158 ceph-mon[36626]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-18T11:38:46.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:46 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/1570957829' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-02-18T11:38:46.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:46 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:46.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:46 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/1570957829' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-02-18T11:38:46.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:46 smithi158 ceph-mon[36626]: osdmap e85: 12 total, 12 up, 12 in 2024-02-18T11:38:46.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:46 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:46.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:46 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:46.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:46 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:46.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:46 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:46.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:46 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:46.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:46 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:46.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:46 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:38:46.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:46 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:38:46.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:46 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:47.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:46 smithi188 ceph-mon[36925]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-18T11:38:47.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:46 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/1570957829' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-02-18T11:38:47.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:46 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:47.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:46 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/1570957829' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-02-18T11:38:47.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:46 smithi188 ceph-mon[36925]: osdmap e85: 12 total, 12 up, 12 in 2024-02-18T11:38:47.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:46 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:47.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:46 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:47.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:46 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:47.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:46 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:47.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:46 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:47.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:46 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:47.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:46 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:38:47.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:46 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:38:47.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:46 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:47.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:46 smithi012 ceph-mon[32460]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-18T11:38:47.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:46 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/1570957829' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-02-18T11:38:47.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:46 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:47.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:46 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/1570957829' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-02-18T11:38:47.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:46 smithi012 ceph-mon[32460]: osdmap e85: 12 total, 12 up, 12 in 2024-02-18T11:38:47.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:46 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:47.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:46 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:47.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:46 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:47.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:46 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:47.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:46 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:47.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:46 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:47.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:46 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:38:47.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:46 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:38:47.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:46 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:38:47.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:47.833+0000 7f51177fe640 1 -- 172.21.15.12:0/4054359771 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f51180390c0 con 0x7f51201016c0 2024-02-18T11:38:47.828 INFO:teuthology.orchestra.run.smithi012.stderr:new fs with metadata pool 2 and data pool 3 2024-02-18T11:38:47.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:47.838+0000 7f51283e2640 1 -- 172.21.15.12:0/4054359771 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f510c0a2750 msgr2=0x7f510c0a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:47.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:47.838+0000 7f51283e2640 1 --2- 172.21.15.12:0/4054359771 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f510c0a2750 0x7f510c0a4c10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f51200692a0 tx=0x7f511000b390 comp rx=0 tx=0).stop 2024-02-18T11:38:47.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:47.838+0000 7f51283e2640 1 -- 172.21.15.12:0/4054359771 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f51201016c0 msgr2=0x7f512019e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:47.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:47.838+0000 7f51283e2640 1 --2- 172.21.15.12:0/4054359771 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f51201016c0 0x7f512019e7d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5118037040 tx=0x7f5118034a90 comp rx=0 tx=0).stop 2024-02-18T11:38:47.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:47.838+0000 7f51283e2640 1 -- 172.21.15.12:0/4054359771 shutdown_connections 2024-02-18T11:38:47.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:47.838+0000 7f51283e2640 1 --2- 172.21.15.12:0/4054359771 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f510c0a2750 0x7f510c0a4c10 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:47.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:47.839+0000 7f51283e2640 1 --2- 172.21.15.12:0/4054359771 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5120100cd0 0x7f5120197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:47.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:47.839+0000 7f51283e2640 1 --2- 172.21.15.12:0/4054359771 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f51201016c0 0x7f512019e7d0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:47.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:47.839+0000 7f51283e2640 1 --2- 172.21.15.12:0/4054359771 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5120100310 0x7f5120197220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:47.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:47.839+0000 7f51283e2640 1 -- 172.21.15.12:0/4054359771 >> 172.21.15.12:0/4054359771 conn(0x7f51200fadc0 msgr2=0x7f51200ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:47.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:47.839+0000 7f51283e2640 1 -- 172.21.15.12:0/4054359771 shutdown_connections 2024-02-18T11:38:47.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:47.839+0000 7f51283e2640 1 -- 172.21.15.12:0/4054359771 wait complete. 2024-02-18T11:38:47.853 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-18T11:38:47.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:47.966+0000 7f7c7c948640 1 -- 172.21.15.12:0/1009513665 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c78107000 msgr2=0x7f7c781093f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:47.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:47.966+0000 7f7c7c948640 1 --2- 172.21.15.12:0/1009513665 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c78107000 0x7f7c781093f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7c70009ca0 tx=0x7f7c7002c0f0 comp rx=0 tx=0).stop 2024-02-18T11:38:47.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:47.966+0000 7f7c7c948640 1 -- 172.21.15.12:0/1009513665 shutdown_connections 2024-02-18T11:38:47.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:47.966+0000 7f7c7c948640 1 --2- 172.21.15.12:0/1009513665 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c781046d0 0x7f7c78106ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:47.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:47.966+0000 7f7c7c948640 1 --2- 172.21.15.12:0/1009513665 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c78107000 0x7f7c781093f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:47.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:47.966+0000 7f7c7c948640 1 --2- 172.21.15.12:0/1009513665 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7c78101da0 0x7f7c78104190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:47.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:47.966+0000 7f7c7c948640 1 -- 172.21.15.12:0/1009513665 >> 172.21.15.12:0/1009513665 conn(0x7f7c780fb6f0 msgr2=0x7f7c780fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:47.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:47.966+0000 7f7c7c948640 1 -- 172.21.15.12:0/1009513665 shutdown_connections 2024-02-18T11:38:47.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:47.966+0000 7f7c7c948640 1 -- 172.21.15.12:0/1009513665 wait complete. 2024-02-18T11:38:47.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:47.967+0000 7f7c7c948640 1 Processor -- start 2024-02-18T11:38:47.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:47.967+0000 7f7c7c948640 1 -- start start 2024-02-18T11:38:47.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:47.967+0000 7f7c7c948640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7c78101da0 0x7f7c78197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:47.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:47.967+0000 7f7c7c948640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c781046d0 0x7f7c78197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:47.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:47.967+0000 7f7c7c948640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c78107000 0x7f7c7819e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:47.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:47.967+0000 7f7c7c948640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c7819ee00 con 0x7f7c78107000 2024-02-18T11:38:47.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:47.967+0000 7f7c7c948640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c7819ef70 con 0x7f7c78101da0 2024-02-18T11:38:47.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:47.967+0000 7f7c7c948640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c7819f130 con 0x7f7c781046d0 2024-02-18T11:38:47.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:47.967+0000 7f7c76d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c78107000 0x7f7c7819e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:47.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:47.967+0000 7f7c76d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c78107000 0x7f7c7819e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46708/0 (socket says 172.21.15.12:46708) 2024-02-18T11:38:47.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:47.967+0000 7f7c76d9d640 1 -- 172.21.15.12:0/2371276557 learned_addr learned my addr 172.21.15.12:0/2371276557 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:38:47.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:47.967+0000 7f7c7659c640 1 --2- 172.21.15.12:0/2371276557 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7c78101da0 0x7f7c78197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:47.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:47.967+0000 7f7c6ffff640 1 --2- 172.21.15.12:0/2371276557 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c781046d0 0x7f7c78197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:47.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:47.967+0000 7f7c76d9d640 1 -- 172.21.15.12:0/2371276557 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c781046d0 msgr2=0x7f7c78197850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:47.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:47.967+0000 7f7c76d9d640 1 --2- 172.21.15.12:0/2371276557 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c781046d0 0x7f7c78197850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:47.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:47.967+0000 7f7c76d9d640 1 -- 172.21.15.12:0/2371276557 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7c78101da0 msgr2=0x7f7c78197310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:47.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:47.967+0000 7f7c76d9d640 1 --2- 172.21.15.12:0/2371276557 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7c78101da0 0x7f7c78197310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:47.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:47.967+0000 7f7c76d9d640 1 -- 172.21.15.12:0/2371276557 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c70009900 con 0x7f7c78107000 2024-02-18T11:38:47.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:47.967+0000 7f7c76d9d640 1 --2- 172.21.15.12:0/2371276557 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c78107000 0x7f7c7819e8c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7c70037040 tx=0x7f7c70034d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:47.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:47.968+0000 7f7c6f7fe640 1 -- 172.21.15.12:0/2371276557 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c70003970 con 0x7f7c78107000 2024-02-18T11:38:47.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:47.968+0000 7f7c6f7fe640 1 -- 172.21.15.12:0/2371276557 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7c70004040 con 0x7f7c78107000 2024-02-18T11:38:47.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:47.968+0000 7f7c7c948640 1 -- 172.21.15.12:0/2371276557 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c7819f3b0 con 0x7f7c78107000 2024-02-18T11:38:47.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:47.968+0000 7f7c6f7fe640 1 -- 172.21.15.12:0/2371276557 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c700406b0 con 0x7f7c78107000 2024-02-18T11:38:47.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:47.968+0000 7f7c7c948640 1 -- 172.21.15.12:0/2371276557 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c7819f8e0 con 0x7f7c78107000 2024-02-18T11:38:47.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:47.969+0000 7f7c7c948640 1 -- 172.21.15.12:0/2371276557 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c780fd1e0 con 0x7f7c78107000 2024-02-18T11:38:47.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:47.970+0000 7f7c6f7fe640 1 -- 172.21.15.12:0/2371276557 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) v1 ==== 137245+0+0 (secure 0 0 0) 0x7f7c7000a040 con 0x7f7c78107000 2024-02-18T11:38:47.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:47.970+0000 7f7c6f7fe640 1 --2- 172.21.15.12:0/2371276557 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f7c640a27d0 0x7f7c640a4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:47.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:47.970+0000 7f7c7659c640 1 --2- 172.21.15.12:0/2371276557 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f7c640a27d0 0x7f7c640a4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:47.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:47.970+0000 7f7c6f7fe640 1 -- 172.21.15.12:0/2371276557 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f7c700f16d0 con 0x7f7c78107000 2024-02-18T11:38:47.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:47.971+0000 7f7c7659c640 1 --2- 172.21.15.12:0/2371276557 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f7c640a27d0 0x7f7c640a4c90 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7c68009990 tx=0x7f7c68009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:47.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:47.975+0000 7f7c6f7fe640 1 -- 172.21.15.12:0/2371276557 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7c700ba510 con 0x7f7c78107000 2024-02-18T11:38:48.095 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:47 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/4054359771' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-02-18T11:38:48.095 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:47 smithi012 ceph-mon[32460]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-02-18T11:38:48.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:47 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/4054359771' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-02-18T11:38:48.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:47 smithi188 ceph-mon[36925]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-02-18T11:38:48.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.157+0000 7f7c7c948640 1 -- 172.21.15.12:0/2371276557 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7c780fcc50 con 0x7f7c78107000 2024-02-18T11:38:48.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.159+0000 7f7c6f7fe640 1 -- 172.21.15.12:0/2371276557 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+22242 (secure 0 0 0) 0x7f7c700bdbd0 con 0x7f7c78107000 2024-02-18T11:38:48.153 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:38:48.153 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":86,"fsid":"0c8bb0de-ce51-11ee-95ba-87774f69a715","created":"2024-02-18T11:30:49.110059+0000","modified":"2024-02-18T11:38:46.887798+0000","last_up_change":"2024-02-18T11:37:57.512337+0000","last_in_change":"2024-02-18T11:37:36.567969+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-18T11:34:15.281138+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-18T11:38:44.802134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-18T11:38:46.081188+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"35f15ea6-deb8-48a1-a7dd-ab649ec13533","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":536522426},{"type":"v1","addr":"172.21.15.12:6803","nonce":536522426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":536522426},{"type":"v1","addr":"172.21.15.12:6805","nonce":536522426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":536522426},{"type":"v1","addr":"172.21.15.12:6809","nonce":536522426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":536522426},{"type":"v1","addr":"172.21.15.12:6807","nonce":536522426}]},"public_addr":"172.21.15.12:6803/536522426","cluster_addr":"172.21.15.12:6805/536522426","heartbeat_back_addr":"172.21.15.12:6809/536522426","heartbeat_front_addr":"172.21.15.12:6807/536522426","state":["exists","up"]},{"osd":1,"uuid":"2af937ef-d827-44e6-b805-70cffbfe1126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":4249111488},{"type":"v1","addr":"172.21.15.158:6801","nonce":4249111488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":4249111488},{"type":"v1","addr":"172.21.15.158:6803","nonce":4249111488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":4249111488},{"type":"v1","addr":"172.21.15.158:6807","nonce":4249111488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":4249111488},{"type":"v1","addr":"172.21.15.158:6805","nonce":4249111488}]},"public_addr":"172.21.15.158:6801/4249111488","cluster_addr":"172.21.15.158:6803/4249111488","heartbeat_back_addr":"172.21.15.158:6807/4249111488","heartbeat_front_addr":"172.21.15.158:6805/4249111488","state":["exists","up"]},{"osd":2,"uuid":"17b804dd-31df-45ae-a637-4a5f65bbaf26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":627976150},{"type":"v1","addr":"172.21.15.188:6801","nonce":627976150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":627976150},{"type":"v1","addr":"172.21.15.188:6803","nonce":627976150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":627976150},{"type":"v1","addr":"172.21.15.188:6807","nonce":627976150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":627976150},{"type":"v1","addr":"172.21.15.188:6805","nonce":627976150}]},"public_addr":"172.21.15.188:6801/627976150","cluster_addr":"172.21.15.188:6803/627976150","heartbeat_back_addr":"172.21.15.188:6807/627976150","heartbeat_front_addr":"172.21.15.188:6805/627976150","state":["exists","up"]},{"osd":3,"uuid":"3f9cc423-cdfa-4acf-aaa2-b58b32ccb9cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":3758512503},{"type":"v1","addr":"172.21.15.12:6811","nonce":3758512503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":3758512503},{"type":"v1","addr":"172.21.15.12:6813","nonce":3758512503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3758512503},{"type":"v1","addr":"172.21.15.12:6817","nonce":3758512503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":3758512503},{"type":"v1","addr":"172.21.15.12:6815","nonce":3758512503}]},"public_addr":"172.21.15.12:6811/3758512503","cluster_addr":"172.21.15.12:6813/3758512503","heartbeat_back_addr":"172.21.15.12:6817/3758512503","heartbeat_front_addr":"172.21.15.12:6815/3758512503","state":["exists","up"]},{"osd":4,"uuid":"a92e20ec-af1c-4ef4-a0cb-caba5fcc71aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":3892820997},{"type":"v1","addr":"172.21.15.158:6809","nonce":3892820997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":3892820997},{"type":"v1","addr":"172.21.15.158:6811","nonce":3892820997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":3892820997},{"type":"v1","addr":"172.21.15.158:6815","nonce":3892820997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":3892820997},{"type":"v1","addr":"172.21.15.158:6813","nonce":3892820997}]},"public_addr":"172.21.15.158:6809/3892820997","cluster_addr":"172.21.15.158:6811/3892820997","heartbeat_back_addr":"172.21.15.158:6815/3892820997","heartbeat_front_addr":"172.21.15.158:6813/3892820997","state":["exists","up"]},{"osd":5,"uuid":"d9c2e0c8-85f5-4ae2-bf3e-125c1be3a2db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3939495896},{"type":"v1","addr":"172.21.15.188:6809","nonce":3939495896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3939495896},{"type":"v1","addr":"172.21.15.188:6811","nonce":3939495896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3939495896},{"type":"v1","addr":"172.21.15.188:6815","nonce":3939495896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3939495896},{"type":"v1","addr":"172.21.15.188:6813","nonce":3939495896}]},"public_addr":"172.21.15.188:6809/3939495896","cluster_addr":"172.21.15.188:6811/3939495896","heartbeat_back_addr":"172.21.15.188:6815/3939495896","heartbeat_front_addr":"172.21.15.188:6813/3939495896","state":["exists","up"]},{"osd":6,"uuid":"4a523512-de6b-49f0-be81-00f2f00c1c10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":1206337767},{"type":"v1","addr":"172.21.15.12:6819","nonce":1206337767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":1206337767},{"type":"v1","addr":"172.21.15.12:6821","nonce":1206337767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":1206337767},{"type":"v1","addr":"172.21.15.12:6825","nonce":1206337767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":1206337767},{"type":"v1","addr":"172.21.15.12:6823","nonce":1206337767}]},"public_addr":"172.21.15.12:6819/1206337767","cluster_addr":"172.21.15.12:6821/1206337767","heartbeat_back_addr":"172.21.15.12:6825/1206337767","heartbeat_front_addr":"172.21.15.12:6823/1206337767","state":["exists","up"]},{"osd":7,"uuid":"363271a6-df93-44c9-b0e5-3d29768aa96c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":3283400032},{"type":"v1","addr":"172.21.15.158:6817","nonce":3283400032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":3283400032},{"type":"v1","addr":"172.21.15.158:6819","nonce":3283400032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":3283400032},{"type":"v1","addr":"172.21.15.158:6823","nonce":3283400032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":3283400032},{"type":"v1","addr":"172.21.15.158:6821","nonce":3283400032}]},"public_addr":"172.21.15.158:6817/3283400032","cluster_addr":"172.21.15.158:6819/3283400032","heartbeat_back_addr":"172.21.15.158:6823/3283400032","heartbeat_front_addr":"172.21.15.158:6821/3283400032","state":["exists","up"]},{"osd":8,"uuid":"9a089528-3e8e-45d3-9279-ddb54088b96a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2110170682},{"type":"v1","addr":"172.21.15.188:6817","nonce":2110170682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2110170682},{"type":"v1","addr":"172.21.15.188:6819","nonce":2110170682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":2110170682},{"type":"v1","addr":"172.21.15.188:6823","nonce":2110170682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2110170682},{"type":"v1","addr":"172.21.15.188:6821","nonce":2110170682}]},"public_addr":"172.21.15.188:6817/2110170682","cluster_addr":"172.21.15.188:6819/2110170682","heartbeat_back_addr":"172.21.15.188:6823/2110170682","heartbeat_front_addr":"172.21.15.188:6821/2110170682","state":["exists","up"]},{"osd":9,"uuid":"b12e2727-d649-4464-90e1-fafbc1754832","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1825928986},{"type":"v1","addr":"172.21.15.12:6827","nonce":1825928986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1825928986},{"type":"v1","addr":"172.21.15.12:6829","nonce":1825928986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":1825928986},{"type":"v1","addr":"172.21.15.12:6833","nonce":1825928986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1825928986},{"type":"v1","addr":"172.21.15.12:6831","nonce":1825928986}]},"public_addr":"172.21.15.12:6827/1825928986","cluster_addr":"172.21.15.12:6829/1825928986","heartbeat_back_addr":"172.21.15.12:6833/1825928986","heartbeat_front_addr":"172.21.15.12:6831/1825928986","state":["exists","up"]},{"osd":10,"uuid":"518b4cbf-9d90-4e9a-8ac9-b5cb9b3dee4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":3239052167},{"type":"v1","addr":"172.21.15.158:6825","nonce":3239052167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":3239052167},{"type":"v1","addr":"172.21.15.158:6827","nonce":3239052167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":3239052167},{"type":"v1","addr":"172.21.15.158:6831","nonce":3239052167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":3239052167},{"type":"v1","addr":"172.21.15.158:6829","nonce":3239052167}]},"public_addr":"172.21.15.158:6825/3239052167","cluster_addr":"172.21.15.158:6827/3239052167","heartbeat_back_addr":"172.21.15.158:6831/3239052167","heartbeat_front_addr":"172.21.15.158:6829/3239052167","state":["exists","up"]},{"osd":11,"uuid":"98aa3a0d-4b2f-46c8-9341-375d13794c3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1135773350},{"type":"v1","addr":"172.21.15.188:6825","nonce":1135773350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1135773350},{"type":"v1","addr":"172.21.15.188:6827","nonce":1135773350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1135773350},{"type":"v1","addr":"172.21.15.188:6831","nonce":1135773350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1135773350},{"type":"v1","addr":"172.21.15.188:6829","nonce":1135773350}]},"public_addr":"172.21.15.188:6825/1135773350","cluster_addr":"172.21.15.188:6827/1135773350","heartbeat_back_addr":"172.21.15.188:6831/1135773350","heartbeat_front_addr":"172.21.15.188:6829/1135773350","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:33:24.280879+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:33:47.859461+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:34:11.725283+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:34:35.707629+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:35:00.024197+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:35:24.499803+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:35:48.953870+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:36:13.642456+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:36:38.936457+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:37:04.021466+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:37:29.073178+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:37:54.863919+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:6801/4135274411":"2024-02-19T11:31:50.936928+0000","172.21.15.12:6800/4135274411":"2024-02-19T11:31:50.936928+0000","172.21.15.12:0/118428584":"2024-02-19T11:31:50.936928+0000","172.21.15.12:0/4276810915":"2024-02-19T11:31:50.936928+0000","172.21.15.12:0/3777545262":"2024-02-19T11:31:24.000830+0000","172.21.15.12:0/114707743":"2024-02-19T11:31:24.000830+0000","172.21.15.12:6800/3925204723":"2024-02-19T11:31:24.000830+0000","172.21.15.12:6801/3925204723":"2024-02-19T11:31:24.000830+0000","172.21.15.12:0/74881618":"2024-02-19T11:31:24.000830+0000","172.21.15.12:0/1614805375":"2024-02-19T11:31:50.936928+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-18T11:38:48.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.161+0000 7f7c7c948640 1 -- 172.21.15.12:0/2371276557 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f7c640a27d0 msgr2=0x7f7c640a4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:48.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.161+0000 7f7c7c948640 1 --2- 172.21.15.12:0/2371276557 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f7c640a27d0 0x7f7c640a4c90 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7c68009990 tx=0x7f7c68009110 comp rx=0 tx=0).stop 2024-02-18T11:38:48.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.161+0000 7f7c7c948640 1 -- 172.21.15.12:0/2371276557 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c78107000 msgr2=0x7f7c7819e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:48.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.161+0000 7f7c7c948640 1 --2- 172.21.15.12:0/2371276557 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c78107000 0x7f7c7819e8c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7c70037040 tx=0x7f7c70034d20 comp rx=0 tx=0).stop 2024-02-18T11:38:48.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.161+0000 7f7c7c948640 1 -- 172.21.15.12:0/2371276557 shutdown_connections 2024-02-18T11:38:48.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.161+0000 7f7c7c948640 1 --2- 172.21.15.12:0/2371276557 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f7c640a27d0 0x7f7c640a4c90 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:48.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.161+0000 7f7c7c948640 1 --2- 172.21.15.12:0/2371276557 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c781046d0 0x7f7c78197850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:48.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.161+0000 7f7c7c948640 1 --2- 172.21.15.12:0/2371276557 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c78107000 0x7f7c7819e8c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:48.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.161+0000 7f7c7c948640 1 --2- 172.21.15.12:0/2371276557 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7c78101da0 0x7f7c78197310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:48.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.161+0000 7f7c7c948640 1 -- 172.21.15.12:0/2371276557 >> 172.21.15.12:0/2371276557 conn(0x7f7c780fb6f0 msgr2=0x7f7c781026e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:48.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.161+0000 7f7c7c948640 1 -- 172.21.15.12:0/2371276557 shutdown_connections 2024-02-18T11:38:48.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.161+0000 7f7c7c948640 1 -- 172.21.15.12:0/2371276557 wait complete. 2024-02-18T11:38:48.167 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-18T11:38:48.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:47 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/4054359771' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-02-18T11:38:48.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:47 smithi158 ceph-mon[36626]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-02-18T11:38:48.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.275+0000 7f0c46548640 1 -- 172.21.15.12:0/3843905576 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0c401003b0 msgr2=0x7f0c40100830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:48.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.275+0000 7f0c46548640 1 --2- 172.21.15.12:0/3843905576 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0c401003b0 0x7f0c40100830 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0c30009930 tx=0x7f0c3002bc30 comp rx=0 tx=0).stop 2024-02-18T11:38:48.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.275+0000 7f0c46548640 1 -- 172.21.15.12:0/3843905576 shutdown_connections 2024-02-18T11:38:48.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.275+0000 7f0c46548640 1 --2- 172.21.15.12:0/3843905576 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c40100d70 0x7f0c40108610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:48.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.275+0000 7f0c46548640 1 --2- 172.21.15.12:0/3843905576 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0c401003b0 0x7f0c40100830 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:48.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.275+0000 7f0c46548640 1 --2- 172.21.15.12:0/3843905576 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0c40101680 0x7f0c40101aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:48.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.275+0000 7f0c46548640 1 -- 172.21.15.12:0/3843905576 >> 172.21.15.12:0/3843905576 conn(0x7f0c400fade0 msgr2=0x7f0c400fd220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:48.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.275+0000 7f0c46548640 1 -- 172.21.15.12:0/3843905576 shutdown_connections 2024-02-18T11:38:48.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.275+0000 7f0c46548640 1 -- 172.21.15.12:0/3843905576 wait complete. 2024-02-18T11:38:48.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.275+0000 7f0c46548640 1 Processor -- start 2024-02-18T11:38:48.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.275+0000 7f0c46548640 1 -- start start 2024-02-18T11:38:48.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.276+0000 7f0c46548640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c401003b0 0x7f0c401961a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:48.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.276+0000 7f0c46548640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0c40100d70 0x7f0c401966e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:48.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.276+0000 7f0c46548640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0c40101680 0x7f0c4019d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:48.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.276+0000 7f0c46548640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4019dc90 con 0x7f0c40100d70 2024-02-18T11:38:48.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.276+0000 7f0c46548640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4019de00 con 0x7f0c40101680 2024-02-18T11:38:48.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.276+0000 7f0c46548640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4019dfc0 con 0x7f0c401003b0 2024-02-18T11:38:48.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.276+0000 7f0c3f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0c40100d70 0x7f0c401966e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:48.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.276+0000 7f0c3f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0c40100d70 0x7f0c401966e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46730/0 (socket says 172.21.15.12:46730) 2024-02-18T11:38:48.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.276+0000 7f0c3f7fe640 1 -- 172.21.15.12:0/468889085 learned_addr learned my addr 172.21.15.12:0/468889085 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:38:48.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.276+0000 7f0c44ae5640 1 --2- 172.21.15.12:0/468889085 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0c40101680 0x7f0c4019d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:48.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.276+0000 7f0c3ffff640 1 --2- 172.21.15.12:0/468889085 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c401003b0 0x7f0c401961a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:48.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.276+0000 7f0c3f7fe640 1 -- 172.21.15.12:0/468889085 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c401003b0 msgr2=0x7f0c401961a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:48.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.276+0000 7f0c3f7fe640 1 --2- 172.21.15.12:0/468889085 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c401003b0 0x7f0c401961a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:48.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.276+0000 7f0c3f7fe640 1 -- 172.21.15.12:0/468889085 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0c40101680 msgr2=0x7f0c4019d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:48.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.276+0000 7f0c3f7fe640 1 --2- 172.21.15.12:0/468889085 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0c40101680 0x7f0c4019d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:48.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.276+0000 7f0c3f7fe640 1 -- 172.21.15.12:0/468889085 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c30009560 con 0x7f0c40100d70 2024-02-18T11:38:48.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.276+0000 7f0c3f7fe640 1 --2- 172.21.15.12:0/468889085 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0c40100d70 0x7f0c401966e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0c30009230 tx=0x7f0c300042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:48.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.276+0000 7f0c3d7fa640 1 -- 172.21.15.12:0/468889085 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c300044e0 con 0x7f0c40100d70 2024-02-18T11:38:48.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.276+0000 7f0c3d7fa640 1 -- 172.21.15.12:0/468889085 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0c300054e0 con 0x7f0c40100d70 2024-02-18T11:38:48.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.276+0000 7f0c46548640 1 -- 172.21.15.12:0/468889085 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c4019e240 con 0x7f0c40100d70 2024-02-18T11:38:48.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.276+0000 7f0c3d7fa640 1 -- 172.21.15.12:0/468889085 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3003e450 con 0x7f0c40100d70 2024-02-18T11:38:48.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.276+0000 7f0c46548640 1 -- 172.21.15.12:0/468889085 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c4019e750 con 0x7f0c40100d70 2024-02-18T11:38:48.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.278+0000 7f0c46548640 1 -- 172.21.15.12:0/468889085 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c40107540 con 0x7f0c40100d70 2024-02-18T11:38:48.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.279+0000 7f0c3d7fa640 1 -- 172.21.15.12:0/468889085 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) v1 ==== 137245+0+0 (secure 0 0 0) 0x7f0c30003680 con 0x7f0c40100d70 2024-02-18T11:38:48.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.279+0000 7f0c3d7fa640 1 --2- 172.21.15.12:0/468889085 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f0c280a2750 0x7f0c280a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:48.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.279+0000 7f0c3ffff640 1 --2- 172.21.15.12:0/468889085 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f0c280a2750 0x7f0c280a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:48.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.280+0000 7f0c3d7fa640 1 -- 172.21.15.12:0/468889085 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f0c300f0800 con 0x7f0c40100d70 2024-02-18T11:38:48.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.280+0000 7f0c3ffff640 1 --2- 172.21.15.12:0/468889085 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f0c280a2750 0x7f0c280a4c10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0c34005db0 tx=0x7f0c34005d40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:48.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.284+0000 7f0c3d7fa640 1 -- 172.21.15.12:0/468889085 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0c300b96e0 con 0x7f0c40100d70 2024-02-18T11:38:48.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.468+0000 7f0c46548640 1 -- 172.21.15.12:0/468889085 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0c4010bd70 con 0x7f0c40100d70 2024-02-18T11:38:48.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.469+0000 7f0c3d7fa640 1 -- 172.21.15.12:0/468889085 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+22242 (secure 0 0 0) 0x7f0c300bcda0 con 0x7f0c40100d70 2024-02-18T11:38:48.463 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:38:48.464 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":86,"fsid":"0c8bb0de-ce51-11ee-95ba-87774f69a715","created":"2024-02-18T11:30:49.110059+0000","modified":"2024-02-18T11:38:46.887798+0000","last_up_change":"2024-02-18T11:37:57.512337+0000","last_in_change":"2024-02-18T11:37:36.567969+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-18T11:34:15.281138+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-18T11:38:44.802134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-18T11:38:46.081188+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"35f15ea6-deb8-48a1-a7dd-ab649ec13533","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":536522426},{"type":"v1","addr":"172.21.15.12:6803","nonce":536522426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":536522426},{"type":"v1","addr":"172.21.15.12:6805","nonce":536522426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":536522426},{"type":"v1","addr":"172.21.15.12:6809","nonce":536522426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":536522426},{"type":"v1","addr":"172.21.15.12:6807","nonce":536522426}]},"public_addr":"172.21.15.12:6803/536522426","cluster_addr":"172.21.15.12:6805/536522426","heartbeat_back_addr":"172.21.15.12:6809/536522426","heartbeat_front_addr":"172.21.15.12:6807/536522426","state":["exists","up"]},{"osd":1,"uuid":"2af937ef-d827-44e6-b805-70cffbfe1126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":4249111488},{"type":"v1","addr":"172.21.15.158:6801","nonce":4249111488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":4249111488},{"type":"v1","addr":"172.21.15.158:6803","nonce":4249111488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":4249111488},{"type":"v1","addr":"172.21.15.158:6807","nonce":4249111488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":4249111488},{"type":"v1","addr":"172.21.15.158:6805","nonce":4249111488}]},"public_addr":"172.21.15.158:6801/4249111488","cluster_addr":"172.21.15.158:6803/4249111488","heartbeat_back_addr":"172.21.15.158:6807/4249111488","heartbeat_front_addr":"172.21.15.158:6805/4249111488","state":["exists","up"]},{"osd":2,"uuid":"17b804dd-31df-45ae-a637-4a5f65bbaf26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":627976150},{"type":"v1","addr":"172.21.15.188:6801","nonce":627976150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":627976150},{"type":"v1","addr":"172.21.15.188:6803","nonce":627976150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":627976150},{"type":"v1","addr":"172.21.15.188:6807","nonce":627976150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":627976150},{"type":"v1","addr":"172.21.15.188:6805","nonce":627976150}]},"public_addr":"172.21.15.188:6801/627976150","cluster_addr":"172.21.15.188:6803/627976150","heartbeat_back_addr":"172.21.15.188:6807/627976150","heartbeat_front_addr":"172.21.15.188:6805/627976150","state":["exists","up"]},{"osd":3,"uuid":"3f9cc423-cdfa-4acf-aaa2-b58b32ccb9cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":3758512503},{"type":"v1","addr":"172.21.15.12:6811","nonce":3758512503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":3758512503},{"type":"v1","addr":"172.21.15.12:6813","nonce":3758512503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3758512503},{"type":"v1","addr":"172.21.15.12:6817","nonce":3758512503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":3758512503},{"type":"v1","addr":"172.21.15.12:6815","nonce":3758512503}]},"public_addr":"172.21.15.12:6811/3758512503","cluster_addr":"172.21.15.12:6813/3758512503","heartbeat_back_addr":"172.21.15.12:6817/3758512503","heartbeat_front_addr":"172.21.15.12:6815/3758512503","state":["exists","up"]},{"osd":4,"uuid":"a92e20ec-af1c-4ef4-a0cb-caba5fcc71aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":3892820997},{"type":"v1","addr":"172.21.15.158:6809","nonce":3892820997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":3892820997},{"type":"v1","addr":"172.21.15.158:6811","nonce":3892820997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":3892820997},{"type":"v1","addr":"172.21.15.158:6815","nonce":3892820997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":3892820997},{"type":"v1","addr":"172.21.15.158:6813","nonce":3892820997}]},"public_addr":"172.21.15.158:6809/3892820997","cluster_addr":"172.21.15.158:6811/3892820997","heartbeat_back_addr":"172.21.15.158:6815/3892820997","heartbeat_front_addr":"172.21.15.158:6813/3892820997","state":["exists","up"]},{"osd":5,"uuid":"d9c2e0c8-85f5-4ae2-bf3e-125c1be3a2db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3939495896},{"type":"v1","addr":"172.21.15.188:6809","nonce":3939495896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3939495896},{"type":"v1","addr":"172.21.15.188:6811","nonce":3939495896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3939495896},{"type":"v1","addr":"172.21.15.188:6815","nonce":3939495896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3939495896},{"type":"v1","addr":"172.21.15.188:6813","nonce":3939495896}]},"public_addr":"172.21.15.188:6809/3939495896","cluster_addr":"172.21.15.188:6811/3939495896","heartbeat_back_addr":"172.21.15.188:6815/3939495896","heartbeat_front_addr":"172.21.15.188:6813/3939495896","state":["exists","up"]},{"osd":6,"uuid":"4a523512-de6b-49f0-be81-00f2f00c1c10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":1206337767},{"type":"v1","addr":"172.21.15.12:6819","nonce":1206337767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":1206337767},{"type":"v1","addr":"172.21.15.12:6821","nonce":1206337767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":1206337767},{"type":"v1","addr":"172.21.15.12:6825","nonce":1206337767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":1206337767},{"type":"v1","addr":"172.21.15.12:6823","nonce":1206337767}]},"public_addr":"172.21.15.12:6819/1206337767","cluster_addr":"172.21.15.12:6821/1206337767","heartbeat_back_addr":"172.21.15.12:6825/1206337767","heartbeat_front_addr":"172.21.15.12:6823/1206337767","state":["exists","up"]},{"osd":7,"uuid":"363271a6-df93-44c9-b0e5-3d29768aa96c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":3283400032},{"type":"v1","addr":"172.21.15.158:6817","nonce":3283400032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":3283400032},{"type":"v1","addr":"172.21.15.158:6819","nonce":3283400032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":3283400032},{"type":"v1","addr":"172.21.15.158:6823","nonce":3283400032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":3283400032},{"type":"v1","addr":"172.21.15.158:6821","nonce":3283400032}]},"public_addr":"172.21.15.158:6817/3283400032","cluster_addr":"172.21.15.158:6819/3283400032","heartbeat_back_addr":"172.21.15.158:6823/3283400032","heartbeat_front_addr":"172.21.15.158:6821/3283400032","state":["exists","up"]},{"osd":8,"uuid":"9a089528-3e8e-45d3-9279-ddb54088b96a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2110170682},{"type":"v1","addr":"172.21.15.188:6817","nonce":2110170682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2110170682},{"type":"v1","addr":"172.21.15.188:6819","nonce":2110170682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":2110170682},{"type":"v1","addr":"172.21.15.188:6823","nonce":2110170682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2110170682},{"type":"v1","addr":"172.21.15.188:6821","nonce":2110170682}]},"public_addr":"172.21.15.188:6817/2110170682","cluster_addr":"172.21.15.188:6819/2110170682","heartbeat_back_addr":"172.21.15.188:6823/2110170682","heartbeat_front_addr":"172.21.15.188:6821/2110170682","state":["exists","up"]},{"osd":9,"uuid":"b12e2727-d649-4464-90e1-fafbc1754832","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1825928986},{"type":"v1","addr":"172.21.15.12:6827","nonce":1825928986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1825928986},{"type":"v1","addr":"172.21.15.12:6829","nonce":1825928986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":1825928986},{"type":"v1","addr":"172.21.15.12:6833","nonce":1825928986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1825928986},{"type":"v1","addr":"172.21.15.12:6831","nonce":1825928986}]},"public_addr":"172.21.15.12:6827/1825928986","cluster_addr":"172.21.15.12:6829/1825928986","heartbeat_back_addr":"172.21.15.12:6833/1825928986","heartbeat_front_addr":"172.21.15.12:6831/1825928986","state":["exists","up"]},{"osd":10,"uuid":"518b4cbf-9d90-4e9a-8ac9-b5cb9b3dee4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":3239052167},{"type":"v1","addr":"172.21.15.158:6825","nonce":3239052167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":3239052167},{"type":"v1","addr":"172.21.15.158:6827","nonce":3239052167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":3239052167},{"type":"v1","addr":"172.21.15.158:6831","nonce":3239052167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":3239052167},{"type":"v1","addr":"172.21.15.158:6829","nonce":3239052167}]},"public_addr":"172.21.15.158:6825/3239052167","cluster_addr":"172.21.15.158:6827/3239052167","heartbeat_back_addr":"172.21.15.158:6831/3239052167","heartbeat_front_addr":"172.21.15.158:6829/3239052167","state":["exists","up"]},{"osd":11,"uuid":"98aa3a0d-4b2f-46c8-9341-375d13794c3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1135773350},{"type":"v1","addr":"172.21.15.188:6825","nonce":1135773350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1135773350},{"type":"v1","addr":"172.21.15.188:6827","nonce":1135773350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1135773350},{"type":"v1","addr":"172.21.15.188:6831","nonce":1135773350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1135773350},{"type":"v1","addr":"172.21.15.188:6829","nonce":1135773350}]},"public_addr":"172.21.15.188:6825/1135773350","cluster_addr":"172.21.15.188:6827/1135773350","heartbeat_back_addr":"172.21.15.188:6831/1135773350","heartbeat_front_addr":"172.21.15.188:6829/1135773350","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:33:24.280879+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:33:47.859461+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:34:11.725283+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:34:35.707629+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:35:00.024197+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:35:24.499803+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:35:48.953870+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:36:13.642456+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:36:38.936457+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:37:04.021466+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:37:29.073178+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:37:54.863919+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:6801/4135274411":"2024-02-19T11:31:50.936928+0000","172.21.15.12:6800/4135274411":"2024-02-19T11:31:50.936928+0000","172.21.15.12:0/118428584":"2024-02-19T11:31:50.936928+0000","172.21.15.12:0/4276810915":"2024-02-19T11:31:50.936928+0000","172.21.15.12:0/3777545262":"2024-02-19T11:31:24.000830+0000","172.21.15.12:0/114707743":"2024-02-19T11:31:24.000830+0000","172.21.15.12:6800/3925204723":"2024-02-19T11:31:24.000830+0000","172.21.15.12:6801/3925204723":"2024-02-19T11:31:24.000830+0000","172.21.15.12:0/74881618":"2024-02-19T11:31:24.000830+0000","172.21.15.12:0/1614805375":"2024-02-19T11:31:50.936928+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-18T11:38:48.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.471+0000 7f0c46548640 1 -- 172.21.15.12:0/468889085 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f0c280a2750 msgr2=0x7f0c280a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:48.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.471+0000 7f0c46548640 1 --2- 172.21.15.12:0/468889085 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f0c280a2750 0x7f0c280a4c10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0c34005db0 tx=0x7f0c34005d40 comp rx=0 tx=0).stop 2024-02-18T11:38:48.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.471+0000 7f0c46548640 1 -- 172.21.15.12:0/468889085 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0c40100d70 msgr2=0x7f0c401966e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:48.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.471+0000 7f0c46548640 1 --2- 172.21.15.12:0/468889085 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0c40100d70 0x7f0c401966e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0c30009230 tx=0x7f0c300042f0 comp rx=0 tx=0).stop 2024-02-18T11:38:48.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.471+0000 7f0c46548640 1 -- 172.21.15.12:0/468889085 shutdown_connections 2024-02-18T11:38:48.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.471+0000 7f0c46548640 1 --2- 172.21.15.12:0/468889085 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f0c280a2750 0x7f0c280a4c10 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:48.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.471+0000 7f0c46548640 1 --2- 172.21.15.12:0/468889085 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0c40101680 0x7f0c4019d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:48.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.471+0000 7f0c46548640 1 --2- 172.21.15.12:0/468889085 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0c40100d70 0x7f0c401966e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:48.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.471+0000 7f0c46548640 1 --2- 172.21.15.12:0/468889085 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c401003b0 0x7f0c401961a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:48.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.471+0000 7f0c46548640 1 -- 172.21.15.12:0/468889085 >> 172.21.15.12:0/468889085 conn(0x7f0c400fade0 msgr2=0x7f0c40104710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:48.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.471+0000 7f0c46548640 1 -- 172.21.15.12:0/468889085 shutdown_connections 2024-02-18T11:38:48.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.471+0000 7f0c46548640 1 -- 172.21.15.12:0/468889085 wait complete. 2024-02-18T11:38:48.477 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-02-18T11:38:48.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.587+0000 7f27f2c88640 1 -- 172.21.15.12:0/1286932109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f27ec1012b0 msgr2=0x7f27ec101730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:48.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.587+0000 7f27f2c88640 1 --2- 172.21.15.12:0/1286932109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f27ec1012b0 0x7f27ec101730 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f27dc009ca0 tx=0x7f27dc02c0f0 comp rx=0 tx=0).stop 2024-02-18T11:38:48.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.587+0000 7f27f2c88640 1 -- 172.21.15.12:0/1286932109 shutdown_connections 2024-02-18T11:38:48.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.587+0000 7f27f2c88640 1 --2- 172.21.15.12:0/1286932109 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f27ec101c70 0x7f27ec108500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:48.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.587+0000 7f27f2c88640 1 --2- 172.21.15.12:0/1286932109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f27ec1012b0 0x7f27ec101730 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:48.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.587+0000 7f27f2c88640 1 --2- 172.21.15.12:0/1286932109 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f27ec1000c0 0x7f27ec1004e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:48.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.587+0000 7f27f2c88640 1 -- 172.21.15.12:0/1286932109 >> 172.21.15.12:0/1286932109 conn(0x7f27ec0fb650 msgr2=0x7f27ec0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:48.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.587+0000 7f27f2c88640 1 -- 172.21.15.12:0/1286932109 shutdown_connections 2024-02-18T11:38:48.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.587+0000 7f27f2c88640 1 -- 172.21.15.12:0/1286932109 wait complete. 2024-02-18T11:38:48.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.588+0000 7f27f2c88640 1 Processor -- start 2024-02-18T11:38:48.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.588+0000 7f27f2c88640 1 -- start start 2024-02-18T11:38:48.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.588+0000 7f27f2c88640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f27ec1000c0 0x7f27ec068ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:48.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.588+0000 7f27f2c88640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f27ec1012b0 0x7f27ec19bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:48.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.588+0000 7f27f2c88640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f27ec101c70 0x7f27ec19e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:48.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.588+0000 7f27f2c88640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27ec19ed50 con 0x7f27ec1012b0 2024-02-18T11:38:48.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.588+0000 7f27f2c88640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27ec19eec0 con 0x7f27ec1000c0 2024-02-18T11:38:48.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.588+0000 7f27f2c88640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27ec19f080 con 0x7f27ec101c70 2024-02-18T11:38:48.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.588+0000 7f27ebfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f27ec1012b0 0x7f27ec19bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:48.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.588+0000 7f27ebfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f27ec1012b0 0x7f27ec19bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46742/0 (socket says 172.21.15.12:46742) 2024-02-18T11:38:48.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.588+0000 7f27ebfff640 1 -- 172.21.15.12:0/4036703468 learned_addr learned my addr 172.21.15.12:0/4036703468 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:38:48.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.588+0000 7f27f0a24640 1 --2- 172.21.15.12:0/4036703468 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f27ec1000c0 0x7f27ec068ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:48.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.588+0000 7f27ebfff640 1 -- 172.21.15.12:0/4036703468 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f27ec101c70 msgr2=0x7f27ec19e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:48.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.588+0000 7f27ebfff640 1 --2- 172.21.15.12:0/4036703468 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f27ec101c70 0x7f27ec19e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:48.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.588+0000 7f27ebfff640 1 -- 172.21.15.12:0/4036703468 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f27ec1000c0 msgr2=0x7f27ec068ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:48.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.588+0000 7f27ebfff640 1 --2- 172.21.15.12:0/4036703468 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f27ec1000c0 0x7f27ec068ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:48.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.588+0000 7f27ebfff640 1 -- 172.21.15.12:0/4036703468 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27dc009900 con 0x7f27ec1012b0 2024-02-18T11:38:48.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.589+0000 7f27ebfff640 1 --2- 172.21.15.12:0/4036703468 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f27ec1012b0 0x7f27ec19bf50 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f27dc009dc0 tx=0x7f27dc034d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:48.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.589+0000 7f27e9ffb640 1 -- 172.21.15.12:0/4036703468 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27dc003970 con 0x7f27ec1012b0 2024-02-18T11:38:48.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.589+0000 7f27e9ffb640 1 -- 172.21.15.12:0/4036703468 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f27dc004040 con 0x7f27ec1012b0 2024-02-18T11:38:48.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.589+0000 7f27f2c88640 1 -- 172.21.15.12:0/4036703468 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27ec19f300 con 0x7f27ec1012b0 2024-02-18T11:38:48.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.589+0000 7f27e9ffb640 1 -- 172.21.15.12:0/4036703468 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27dc0406b0 con 0x7f27ec1012b0 2024-02-18T11:38:48.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.589+0000 7f27f2c88640 1 -- 172.21.15.12:0/4036703468 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27ec19f800 con 0x7f27ec1012b0 2024-02-18T11:38:48.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.590+0000 7f27f2c88640 1 -- 172.21.15.12:0/4036703468 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27ec1073e0 con 0x7f27ec1012b0 2024-02-18T11:38:48.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.591+0000 7f27e9ffb640 1 -- 172.21.15.12:0/4036703468 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) v1 ==== 137245+0+0 (secure 0 0 0) 0x7f27dc00a040 con 0x7f27ec1012b0 2024-02-18T11:38:48.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.591+0000 7f27e9ffb640 1 --2- 172.21.15.12:0/4036703468 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f27d80a27d0 0x7f27d80a4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:48.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.592+0000 7f27f0a24640 1 --2- 172.21.15.12:0/4036703468 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f27d80a27d0 0x7f27d80a4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:48.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.592+0000 7f27e9ffb640 1 -- 172.21.15.12:0/4036703468 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f27dc0f13a0 con 0x7f27ec1012b0 2024-02-18T11:38:48.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.592+0000 7f27f0a24640 1 --2- 172.21.15.12:0/4036703468 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f27d80a27d0 0x7f27d80a4c90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f27e4005fd0 tx=0x7f27e4005ce0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:48.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.596+0000 7f27e9ffb640 1 -- 172.21.15.12:0/4036703468 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f27dc0ba280 con 0x7f27ec1012b0 2024-02-18T11:38:48.811 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:48 smithi012 ceph-mon[32460]: pgmap v283: 97 pgs: 7 active+clean, 35 creating+peering, 55 unknown; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-18T11:38:48.811 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:48 smithi012 ceph-mon[32460]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T11:38:48.811 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:48 smithi012 ceph-mon[32460]: osdmap e86: 12 total, 12 up, 12 in 2024-02-18T11:38:48.812 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:48 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/4054359771' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-02-18T11:38:48.812 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:48 smithi012 ceph-mon[32460]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-02-18T11:38:48.812 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:48 smithi012 ceph-mon[32460]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-02-18T11:38:48.812 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:48 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/2371276557' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-18T11:38:48.812 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:48 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/468889085' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-18T11:38:48.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.817+0000 7f27f2c88640 1 -- 172.21.15.12:0/4036703468 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f27ec10bc80 con 0x7f27ec1012b0 2024-02-18T11:38:48.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.833+0000 7f27e9ffb640 1 -- 172.21.15.12:0/4036703468 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) v1 ==== 118+0+0 (secure 0 0 0) 0x7f27dc0041b0 con 0x7f27ec1012b0 2024-02-18T11:38:48.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.835+0000 7f27f2c88640 1 -- 172.21.15.12:0/4036703468 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f27d80a27d0 msgr2=0x7f27d80a4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:48.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.835+0000 7f27f2c88640 1 --2- 172.21.15.12:0/4036703468 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f27d80a27d0 0x7f27d80a4c90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f27e4005fd0 tx=0x7f27e4005ce0 comp rx=0 tx=0).stop 2024-02-18T11:38:48.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.835+0000 7f27f2c88640 1 -- 172.21.15.12:0/4036703468 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f27ec1012b0 msgr2=0x7f27ec19bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:48.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.835+0000 7f27f2c88640 1 --2- 172.21.15.12:0/4036703468 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f27ec1012b0 0x7f27ec19bf50 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f27dc009dc0 tx=0x7f27dc034d40 comp rx=0 tx=0).stop 2024-02-18T11:38:48.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.835+0000 7f27f2c88640 1 -- 172.21.15.12:0/4036703468 shutdown_connections 2024-02-18T11:38:48.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.835+0000 7f27f2c88640 1 --2- 172.21.15.12:0/4036703468 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f27d80a27d0 0x7f27d80a4c90 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:48.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.835+0000 7f27f2c88640 1 --2- 172.21.15.12:0/4036703468 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f27ec101c70 0x7f27ec19e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:48.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.835+0000 7f27f2c88640 1 --2- 172.21.15.12:0/4036703468 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f27ec1012b0 0x7f27ec19bf50 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:48.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.835+0000 7f27f2c88640 1 --2- 172.21.15.12:0/4036703468 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f27ec1000c0 0x7f27ec068ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:48.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.835+0000 7f27f2c88640 1 -- 172.21.15.12:0/4036703468 >> 172.21.15.12:0/4036703468 conn(0x7f27ec0fb650 msgr2=0x7f27ec1045b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:48.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.835+0000 7f27f2c88640 1 -- 172.21.15.12:0/4036703468 shutdown_connections 2024-02-18T11:38:48.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.835+0000 7f27f2c88640 1 -- 172.21.15.12:0/4036703468 wait complete. 2024-02-18T11:38:48.841 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-02-18T11:38:48.841 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-02-18T11:38:48.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.952+0000 7f289d69a640 1 -- 172.21.15.12:0/2935955146 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2898106f00 msgr2=0x7f28981092f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:48.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.952+0000 7f289d69a640 1 --2- 172.21.15.12:0/2935955146 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2898106f00 0x7f28981092f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2890009ca0 tx=0x7f289002c0f0 comp rx=0 tx=0).stop 2024-02-18T11:38:48.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.952+0000 7f289d69a640 1 -- 172.21.15.12:0/2935955146 shutdown_connections 2024-02-18T11:38:48.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.952+0000 7f289d69a640 1 --2- 172.21.15.12:0/2935955146 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28981045d0 0x7f28981069c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:48.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.952+0000 7f289d69a640 1 --2- 172.21.15.12:0/2935955146 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2898106f00 0x7f28981092f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:48.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.952+0000 7f289d69a640 1 --2- 172.21.15.12:0/2935955146 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2898101ca0 0x7f2898104090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:48.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.953+0000 7f289d69a640 1 -- 172.21.15.12:0/2935955146 >> 172.21.15.12:0/2935955146 conn(0x7f28980fb650 msgr2=0x7f28980fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:48.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.953+0000 7f289d69a640 1 -- 172.21.15.12:0/2935955146 shutdown_connections 2024-02-18T11:38:48.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.953+0000 7f289d69a640 1 -- 172.21.15.12:0/2935955146 wait complete. 2024-02-18T11:38:48.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.953+0000 7f289d69a640 1 Processor -- start 2024-02-18T11:38:48.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.953+0000 7f289d69a640 1 -- start start 2024-02-18T11:38:48.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.953+0000 7f289d69a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2898101ca0 0x7f28981972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:48.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.953+0000 7f289d69a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28981045d0 0x7f2898197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:48.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.953+0000 7f289d69a640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2898106f00 0x7f289819e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:48.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.953+0000 7f289d69a640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f289819edc0 con 0x7f2898101ca0 2024-02-18T11:38:48.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.953+0000 7f289d69a640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f289819ef30 con 0x7f2898106f00 2024-02-18T11:38:48.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.953+0000 7f289d69a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f289819f0f0 con 0x7f28981045d0 2024-02-18T11:38:48.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.953+0000 7f2896ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2898101ca0 0x7f28981972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:48.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.953+0000 7f2896ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2898101ca0 0x7f28981972d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46750/0 (socket says 172.21.15.12:46750) 2024-02-18T11:38:48.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.953+0000 7f2896ffd640 1 -- 172.21.15.12:0/1987242148 learned_addr learned my addr 172.21.15.12:0/1987242148 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:38:48.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.954+0000 7f288ffff640 1 --2- 172.21.15.12:0/1987242148 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28981045d0 0x7f2898197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:48.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.954+0000 7f2896ffd640 1 -- 172.21.15.12:0/1987242148 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28981045d0 msgr2=0x7f2898197810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:48.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.954+0000 7f2896ffd640 1 --2- 172.21.15.12:0/1987242148 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28981045d0 0x7f2898197810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:48.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.954+0000 7f28977fe640 1 --2- 172.21.15.12:0/1987242148 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2898106f00 0x7f289819e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:48.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.954+0000 7f2896ffd640 1 -- 172.21.15.12:0/1987242148 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2898106f00 msgr2=0x7f289819e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:48.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.954+0000 7f2896ffd640 1 --2- 172.21.15.12:0/1987242148 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2898106f00 0x7f289819e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:48.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.954+0000 7f2896ffd640 1 -- 172.21.15.12:0/1987242148 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2890009900 con 0x7f2898101ca0 2024-02-18T11:38:48.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.954+0000 7f288ffff640 1 --2- 172.21.15.12:0/1987242148 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28981045d0 0x7f2898197810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:38:48.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.954+0000 7f2896ffd640 1 --2- 172.21.15.12:0/1987242148 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2898101ca0 0x7f28981972d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f288800c890 tx=0x7f288800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:48.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.954+0000 7f2894ff9640 1 -- 172.21.15.12:0/1987242148 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2888004220 con 0x7f2898101ca0 2024-02-18T11:38:48.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.954+0000 7f289d69a640 1 -- 172.21.15.12:0/1987242148 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f289819f3d0 con 0x7f2898101ca0 2024-02-18T11:38:48.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.954+0000 7f2894ff9640 1 -- 172.21.15.12:0/1987242148 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2888007e90 con 0x7f2898101ca0 2024-02-18T11:38:48.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.954+0000 7f2894ff9640 1 -- 172.21.15.12:0/1987242148 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2888004a00 con 0x7f2898101ca0 2024-02-18T11:38:48.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.954+0000 7f289d69a640 1 -- 172.21.15.12:0/1987242148 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f289819f930 con 0x7f2898101ca0 2024-02-18T11:38:48.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.955+0000 7f289d69a640 1 -- 172.21.15.12:0/1987242148 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28980fd140 con 0x7f2898101ca0 2024-02-18T11:38:48.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.956+0000 7f2894ff9640 1 -- 172.21.15.12:0/1987242148 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) v1 ==== 137245+0+0 (secure 0 0 0) 0x7f28880079b0 con 0x7f2898101ca0 2024-02-18T11:38:48.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.956+0000 7f2894ff9640 1 --2- 172.21.15.12:0/1987242148 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f28840a2750 0x7f28840a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:48.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.957+0000 7f288ffff640 1 --2- 172.21.15.12:0/1987242148 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f28840a2750 0x7f28840a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:48.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.957+0000 7f2894ff9640 1 -- 172.21.15.12:0/1987242148 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f28880ce1a0 con 0x7f2898101ca0 2024-02-18T11:38:48.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.957+0000 7f288ffff640 1 --2- 172.21.15.12:0/1987242148 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f28840a2750 0x7f28840a4c10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2880006fd0 tx=0x7f2880008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:48.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:48.961+0000 7f2894ff9640 1 -- 172.21.15.12:0/1987242148 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2888097080 con 0x7f2898101ca0 2024-02-18T11:38:49.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:48 smithi188 ceph-mon[36925]: pgmap v283: 97 pgs: 7 active+clean, 35 creating+peering, 55 unknown; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-18T11:38:49.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:48 smithi188 ceph-mon[36925]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T11:38:49.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:48 smithi188 ceph-mon[36925]: osdmap e86: 12 total, 12 up, 12 in 2024-02-18T11:38:49.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:48 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/4054359771' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-02-18T11:38:49.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:48 smithi188 ceph-mon[36925]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-02-18T11:38:49.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:48 smithi188 ceph-mon[36925]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-02-18T11:38:49.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:48 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/2371276557' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-18T11:38:49.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:48 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/468889085' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-18T11:38:49.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:49.180+0000 7f289d69a640 1 -- 172.21.15.12:0/1987242148 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7f289802ce80 con 0x7f2898101ca0 2024-02-18T11:38:49.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:48 smithi158 ceph-mon[36626]: pgmap v283: 97 pgs: 7 active+clean, 35 creating+peering, 55 unknown; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-18T11:38:49.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:48 smithi158 ceph-mon[36626]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T11:38:49.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:48 smithi158 ceph-mon[36626]: osdmap e86: 12 total, 12 up, 12 in 2024-02-18T11:38:49.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:48 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/4054359771' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-02-18T11:38:49.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:48 smithi158 ceph-mon[36626]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-02-18T11:38:49.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:48 smithi158 ceph-mon[36626]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-02-18T11:38:49.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:48 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/2371276557' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-18T11:38:49.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:48 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/468889085' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-18T11:38:49.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:49.840+0000 7f2894ff9640 1 -- 172.21.15.12:0/1987242148 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v16) v1 ==== 105+0+0 (secure 0 0 0) 0x7f288809a740 con 0x7f2898101ca0 2024-02-18T11:38:49.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:49.842+0000 7f289d69a640 1 -- 172.21.15.12:0/1987242148 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f28840a2750 msgr2=0x7f28840a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:49.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:49.842+0000 7f289d69a640 1 --2- 172.21.15.12:0/1987242148 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f28840a2750 0x7f28840a4c10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2880006fd0 tx=0x7f2880008040 comp rx=0 tx=0).stop 2024-02-18T11:38:49.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:49.842+0000 7f289d69a640 1 -- 172.21.15.12:0/1987242148 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2898101ca0 msgr2=0x7f28981972d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:49.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:49.842+0000 7f289d69a640 1 --2- 172.21.15.12:0/1987242148 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2898101ca0 0x7f28981972d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f288800c890 tx=0x7f288800cd50 comp rx=0 tx=0).stop 2024-02-18T11:38:49.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:49.842+0000 7f289d69a640 1 -- 172.21.15.12:0/1987242148 shutdown_connections 2024-02-18T11:38:49.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:49.842+0000 7f289d69a640 1 --2- 172.21.15.12:0/1987242148 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f28840a2750 0x7f28840a4c10 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:49.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:49.843+0000 7f289d69a640 1 --2- 172.21.15.12:0/1987242148 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28981045d0 0x7f2898197810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:49.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:49.843+0000 7f289d69a640 1 --2- 172.21.15.12:0/1987242148 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2898106f00 0x7f289819e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:49.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:49.843+0000 7f289d69a640 1 --2- 172.21.15.12:0/1987242148 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2898101ca0 0x7f28981972d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:49.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:49.843+0000 7f289d69a640 1 -- 172.21.15.12:0/1987242148 >> 172.21.15.12:0/1987242148 conn(0x7f28980fb650 msgr2=0x7f28981025e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:49.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:49.843+0000 7f289d69a640 1 -- 172.21.15.12:0/1987242148 shutdown_connections 2024-02-18T11:38:49.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:49.843+0000 7f289d69a640 1 -- 172.21.15.12:0/1987242148 wait complete. 2024-02-18T11:38:49.853 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-02-18T11:38:49.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:49.970+0000 7fd76f876640 1 -- 172.21.15.12:0/3698342631 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7681012c0 msgr2=0x7fd768101740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:49.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:49.970+0000 7fd76f876640 1 --2- 172.21.15.12:0/3698342631 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7681012c0 0x7fd768101740 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd758009900 tx=0x7fd75802bc30 comp rx=0 tx=0).stop 2024-02-18T11:38:49.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:49.970+0000 7fd76f876640 1 -- 172.21.15.12:0/3698342631 shutdown_connections 2024-02-18T11:38:49.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:49.970+0000 7fd76f876640 1 --2- 172.21.15.12:0/3698342631 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd768101c80 0x7fd768108510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:49.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:49.970+0000 7fd76f876640 1 --2- 172.21.15.12:0/3698342631 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7681012c0 0x7fd768101740 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:49.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:49.970+0000 7fd76f876640 1 --2- 172.21.15.12:0/3698342631 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7681000d0 0x7fd7681004f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:49.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:49.970+0000 7fd76f876640 1 -- 172.21.15.12:0/3698342631 >> 172.21.15.12:0/3698342631 conn(0x7fd7680fb660 msgr2=0x7fd7680fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:49.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:49.970+0000 7fd76f876640 1 -- 172.21.15.12:0/3698342631 shutdown_connections 2024-02-18T11:38:49.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:49.970+0000 7fd76f876640 1 -- 172.21.15.12:0/3698342631 wait complete. 2024-02-18T11:38:49.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:49.971+0000 7fd76f876640 1 Processor -- start 2024-02-18T11:38:49.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:49.971+0000 7fd76f876640 1 -- start start 2024-02-18T11:38:49.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:49.971+0000 7fd76f876640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd7681000d0 0x7fd768196140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:49.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:49.971+0000 7fd76f876640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7681012c0 0x7fd768196680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:49.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:49.971+0000 7fd76f876640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd768101c80 0x7fd76819d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:49.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:49.971+0000 7fd76f876640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd76819dc30 con 0x7fd7681012c0 2024-02-18T11:38:49.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:49.971+0000 7fd76f876640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd76819dda0 con 0x7fd7681000d0 2024-02-18T11:38:49.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:49.971+0000 7fd76f876640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd76819df60 con 0x7fd768101c80 2024-02-18T11:38:49.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:49.971+0000 7fd76ce11640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7681012c0 0x7fd768196680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:49.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:49.971+0000 7fd76ce11640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7681012c0 0x7fd768196680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46774/0 (socket says 172.21.15.12:46774) 2024-02-18T11:38:49.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:49.971+0000 7fd76ce11640 1 -- 172.21.15.12:0/4284540463 learned_addr learned my addr 172.21.15.12:0/4284540463 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:38:49.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:49.971+0000 7fd76d612640 1 --2- 172.21.15.12:0/4284540463 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd7681000d0 0x7fd768196140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:49.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:49.971+0000 7fd76de13640 1 --2- 172.21.15.12:0/4284540463 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd768101c80 0x7fd76819d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:49.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:49.971+0000 7fd76ce11640 1 -- 172.21.15.12:0/4284540463 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd768101c80 msgr2=0x7fd76819d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:49.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:49.971+0000 7fd76ce11640 1 --2- 172.21.15.12:0/4284540463 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd768101c80 0x7fd76819d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:49.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:49.971+0000 7fd76ce11640 1 -- 172.21.15.12:0/4284540463 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd7681000d0 msgr2=0x7fd768196140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:49.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:49.971+0000 7fd76ce11640 1 --2- 172.21.15.12:0/4284540463 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd7681000d0 0x7fd768196140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:49.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:49.971+0000 7fd76ce11640 1 -- 172.21.15.12:0/4284540463 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd758009560 con 0x7fd7681012c0 2024-02-18T11:38:49.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:49.971+0000 7fd76ce11640 1 --2- 172.21.15.12:0/4284540463 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7681012c0 0x7fd768196680 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd758006010 tx=0x7fd7580042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:49.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:49.972+0000 7fd75e7fc640 1 -- 172.21.15.12:0/4284540463 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7580044e0 con 0x7fd7681012c0 2024-02-18T11:38:49.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:49.972+0000 7fd76f876640 1 -- 172.21.15.12:0/4284540463 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd76819e1e0 con 0x7fd7681012c0 2024-02-18T11:38:49.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:49.972+0000 7fd75e7fc640 1 -- 172.21.15.12:0/4284540463 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd7580054e0 con 0x7fd7681012c0 2024-02-18T11:38:49.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:49.972+0000 7fd75e7fc640 1 -- 172.21.15.12:0/4284540463 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd75803e450 con 0x7fd7681012c0 2024-02-18T11:38:49.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:49.972+0000 7fd76f876640 1 -- 172.21.15.12:0/4284540463 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd76819e740 con 0x7fd7681012c0 2024-02-18T11:38:49.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:49.973+0000 7fd76f876640 1 -- 172.21.15.12:0/4284540463 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7681073f0 con 0x7fd7681012c0 2024-02-18T11:38:49.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:49.974+0000 7fd75e7fc640 1 -- 172.21.15.12:0/4284540463 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) v1 ==== 137245+0+0 (secure 0 0 0) 0x7fd7580035f0 con 0x7fd7681012c0 2024-02-18T11:38:49.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:49.974+0000 7fd75e7fc640 1 --2- 172.21.15.12:0/4284540463 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fd7540a2750 0x7fd7540a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:49.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:49.974+0000 7fd76d612640 1 --2- 172.21.15.12:0/4284540463 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fd7540a2750 0x7fd7540a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:49.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:49.974+0000 7fd75e7fc640 1 -- 172.21.15.12:0/4284540463 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fd7580f07f0 con 0x7fd7681012c0 2024-02-18T11:38:49.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:49.975+0000 7fd76d612640 1 --2- 172.21.15.12:0/4284540463 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fd7540a2750 0x7fd7540a4c10 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd768101120 tx=0x7fd764005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:49.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:49.979+0000 7fd75e7fc640 1 -- 172.21.15.12:0/4284540463 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd7580b96d0 con 0x7fd7681012c0 2024-02-18T11:38:50.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:49 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/4036703468' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-02-18T11:38:50.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:49 smithi188 ceph-mon[36925]: osdmap e87: 12 total, 12 up, 12 in 2024-02-18T11:38:50.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:49 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/4036703468' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-02-18T11:38:50.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:49 smithi188 ceph-mon[36925]: mds.? [v2:172.21.15.158:6838/1127200849,v1:172.21.15.158:6839/1127200849] up:active 2024-02-18T11:38:50.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:49 smithi188 ceph-mon[36925]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-18T11:38:50.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:49 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/1987242148' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-02-18T11:38:50.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:49 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/4036703468' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-02-18T11:38:50.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:49 smithi012 ceph-mon[32460]: osdmap e87: 12 total, 12 up, 12 in 2024-02-18T11:38:50.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:49 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/4036703468' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-02-18T11:38:50.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:49 smithi012 ceph-mon[32460]: mds.? [v2:172.21.15.158:6838/1127200849,v1:172.21.15.158:6839/1127200849] up:active 2024-02-18T11:38:50.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:49 smithi012 ceph-mon[32460]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-18T11:38:50.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:49 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/1987242148' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-02-18T11:38:50.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:50.201+0000 7fd76f876640 1 -- 172.21.15.12:0/4284540463 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7fd76810bce0 con 0x7fd7681012c0 2024-02-18T11:38:50.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:49 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/4036703468' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-02-18T11:38:50.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:49 smithi158 ceph-mon[36626]: osdmap e87: 12 total, 12 up, 12 in 2024-02-18T11:38:50.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:49 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/4036703468' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-02-18T11:38:50.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:49 smithi158 ceph-mon[36626]: mds.? [v2:172.21.15.158:6838/1127200849,v1:172.21.15.158:6839/1127200849] up:active 2024-02-18T11:38:50.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:49 smithi158 ceph-mon[36626]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-18T11:38:50.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:49 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/1987242148' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-02-18T11:38:50.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:50.846+0000 7fd75e7fc640 1 -- 172.21.15.12:0/4284540463 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v18) v1 ==== 121+0+0 (secure 0 0 0) 0x7fd7580bcd90 con 0x7fd7681012c0 2024-02-18T11:38:50.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:50.849+0000 7fd76f876640 1 -- 172.21.15.12:0/4284540463 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fd7540a2750 msgr2=0x7fd7540a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:50.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:50.849+0000 7fd76f876640 1 --2- 172.21.15.12:0/4284540463 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fd7540a2750 0x7fd7540a4c10 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd768101120 tx=0x7fd764005f90 comp rx=0 tx=0).stop 2024-02-18T11:38:50.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:50.849+0000 7fd76f876640 1 -- 172.21.15.12:0/4284540463 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7681012c0 msgr2=0x7fd768196680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:50.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:50.849+0000 7fd76f876640 1 --2- 172.21.15.12:0/4284540463 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7681012c0 0x7fd768196680 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd758006010 tx=0x7fd7580042f0 comp rx=0 tx=0).stop 2024-02-18T11:38:50.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:50.849+0000 7fd76f876640 1 -- 172.21.15.12:0/4284540463 shutdown_connections 2024-02-18T11:38:50.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:50.849+0000 7fd76f876640 1 --2- 172.21.15.12:0/4284540463 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fd7540a2750 0x7fd7540a4c10 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:50.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:50.849+0000 7fd76f876640 1 --2- 172.21.15.12:0/4284540463 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd768101c80 0x7fd76819d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:50.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:50.849+0000 7fd76f876640 1 --2- 172.21.15.12:0/4284540463 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7681012c0 0x7fd768196680 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:50.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:50.849+0000 7fd76f876640 1 --2- 172.21.15.12:0/4284540463 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd7681000d0 0x7fd768196140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:50.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:50.849+0000 7fd76f876640 1 -- 172.21.15.12:0/4284540463 >> 172.21.15.12:0/4284540463 conn(0x7fd7680fb660 msgr2=0x7fd7681045c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:50.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:50.849+0000 7fd76f876640 1 -- 172.21.15.12:0/4284540463 shutdown_connections 2024-02-18T11:38:50.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:50.849+0000 7fd76f876640 1 -- 172.21.15.12:0/4284540463 wait complete. 2024-02-18T11:38:50.860 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-02-18T11:38:50.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:50.978+0000 7fbdbaedf640 1 -- 172.21.15.12:0/923953230 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbdb41011d0 msgr2=0x7fbdb41015f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:50.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:50.978+0000 7fbdbaedf640 1 --2- 172.21.15.12:0/923953230 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbdb41011d0 0x7fbdb41015f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fbda4009900 tx=0x7fbda402bc30 comp rx=0 tx=0).stop 2024-02-18T11:38:50.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:50.978+0000 7fbdbaedf640 1 -- 172.21.15.12:0/923953230 shutdown_connections 2024-02-18T11:38:50.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:50.978+0000 7fbdbaedf640 1 --2- 172.21.15.12:0/923953230 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbdb41023c0 0x7fbdb4102840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:50.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:50.978+0000 7fbdbaedf640 1 --2- 172.21.15.12:0/923953230 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbdb4102d80 0x7fbdb4109610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:50.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:50.978+0000 7fbdbaedf640 1 --2- 172.21.15.12:0/923953230 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbdb41011d0 0x7fbdb41015f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:50.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:50.978+0000 7fbdbaedf640 1 -- 172.21.15.12:0/923953230 >> 172.21.15.12:0/923953230 conn(0x7fbdb40fc760 msgr2=0x7fbdb40feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:50.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:50.978+0000 7fbdbaedf640 1 -- 172.21.15.12:0/923953230 shutdown_connections 2024-02-18T11:38:50.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:50.978+0000 7fbdbaedf640 1 -- 172.21.15.12:0/923953230 wait complete. 2024-02-18T11:38:50.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:50.979+0000 7fbdbaedf640 1 Processor -- start 2024-02-18T11:38:50.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:50.979+0000 7fbdbaedf640 1 -- start start 2024-02-18T11:38:50.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:50.979+0000 7fbdbaedf640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbdb41011d0 0x7fbdb41972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:50.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:50.979+0000 7fbdbaedf640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbdb41023c0 0x7fbdb41977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:50.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:50.979+0000 7fbdbaedf640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbdb4102d80 0x7fbdb419e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:50.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:50.979+0000 7fbdbaedf640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb419eda0 con 0x7fbdb41011d0 2024-02-18T11:38:50.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:50.979+0000 7fbdbaedf640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb419ef10 con 0x7fbdb41023c0 2024-02-18T11:38:50.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:50.979+0000 7fbdbaedf640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb419f0d0 con 0x7fbdb4102d80 2024-02-18T11:38:50.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:50.979+0000 7fbdb8c7b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbdb41011d0 0x7fbdb41972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:50.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:50.979+0000 7fbdb8c7b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbdb41011d0 0x7fbdb41972b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46786/0 (socket says 172.21.15.12:46786) 2024-02-18T11:38:50.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:50.979+0000 7fbdb8c7b640 1 -- 172.21.15.12:0/1006777121 learned_addr learned my addr 172.21.15.12:0/1006777121 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:38:50.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:50.979+0000 7fbdb3fff640 1 --2- 172.21.15.12:0/1006777121 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbdb41023c0 0x7fbdb41977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:50.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:50.979+0000 7fbdb8c7b640 1 -- 172.21.15.12:0/1006777121 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbdb4102d80 msgr2=0x7fbdb419e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:50.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:50.979+0000 7fbdb947c640 1 --2- 172.21.15.12:0/1006777121 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbdb4102d80 0x7fbdb419e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:50.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:50.979+0000 7fbdb8c7b640 1 --2- 172.21.15.12:0/1006777121 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbdb4102d80 0x7fbdb419e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:50.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:50.979+0000 7fbdb8c7b640 1 -- 172.21.15.12:0/1006777121 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbdb41023c0 msgr2=0x7fbdb41977f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:50.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:50.979+0000 7fbdb8c7b640 1 --2- 172.21.15.12:0/1006777121 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbdb41023c0 0x7fbdb41977f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:50.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:50.979+0000 7fbdb8c7b640 1 -- 172.21.15.12:0/1006777121 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbda4009560 con 0x7fbdb41011d0 2024-02-18T11:38:50.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:50.979+0000 7fbdb3fff640 1 --2- 172.21.15.12:0/1006777121 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbdb41023c0 0x7fbdb41977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:38:50.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:50.979+0000 7fbdb947c640 1 --2- 172.21.15.12:0/1006777121 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbdb4102d80 0x7fbdb419e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:38:50.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:50.979+0000 7fbdb8c7b640 1 --2- 172.21.15.12:0/1006777121 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbdb41011d0 0x7fbdb41972b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbda4000c00 tx=0x7fbda4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:50.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:50.980+0000 7fbdb1ffb640 1 -- 172.21.15.12:0/1006777121 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbda4004df0 con 0x7fbdb41011d0 2024-02-18T11:38:50.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:50.980+0000 7fbdb1ffb640 1 -- 172.21.15.12:0/1006777121 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbda4005430 con 0x7fbdb41011d0 2024-02-18T11:38:50.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:50.980+0000 7fbdbaedf640 1 -- 172.21.15.12:0/1006777121 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdb419f350 con 0x7fbdb41011d0 2024-02-18T11:38:50.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:50.980+0000 7fbdb1ffb640 1 -- 172.21.15.12:0/1006777121 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbda403e450 con 0x7fbdb41011d0 2024-02-18T11:38:50.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:50.980+0000 7fbdbaedf640 1 -- 172.21.15.12:0/1006777121 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdb419f8b0 con 0x7fbdb41011d0 2024-02-18T11:38:50.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:50.981+0000 7fbdbaedf640 1 -- 172.21.15.12:0/1006777121 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdb4108610 con 0x7fbdb41011d0 2024-02-18T11:38:50.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:50.982+0000 7fbdb1ffb640 1 -- 172.21.15.12:0/1006777121 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) v1 ==== 137245+0+0 (secure 0 0 0) 0x7fbda4003680 con 0x7fbdb41011d0 2024-02-18T11:38:50.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:50.983+0000 7fbdb1ffb640 1 --2- 172.21.15.12:0/1006777121 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fbd9c0a27d0 0x7fbd9c0a4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:50.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:50.983+0000 7fbdb3fff640 1 --2- 172.21.15.12:0/1006777121 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fbd9c0a27d0 0x7fbd9c0a4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:50.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:50.983+0000 7fbdb1ffb640 1 -- 172.21.15.12:0/1006777121 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fbda40f0f10 con 0x7fbdb41011d0 2024-02-18T11:38:50.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:50.983+0000 7fbdb3fff640 1 --2- 172.21.15.12:0/1006777121 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fbd9c0a27d0 0x7fbd9c0a4c90 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fbdb41987c0 tx=0x7fbda8005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:50.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:50.987+0000 7fbdb1ffb640 1 -- 172.21.15.12:0/1006777121 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbda40b9df0 con 0x7fbdb41011d0 2024-02-18T11:38:51.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:50 smithi012 ceph-mon[32460]: pgmap v286: 97 pgs: 25 active+clean, 49 creating+peering, 23 unknown; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-18T11:38:51.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:50 smithi012 ceph-mon[32460]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-02-18T11:38:51.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:50 smithi012 ceph-mon[32460]: Cluster is now healthy 2024-02-18T11:38:51.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:50 smithi012 ceph-mon[32460]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-02-18T11:38:51.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:50 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/1987242148' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-02-18T11:38:51.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:50 smithi012 ceph-mon[32460]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-02-18T11:38:51.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:50 smithi012 ceph-mon[32460]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-18T11:38:51.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:50 smithi012 ceph-mon[32460]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-02-18T11:38:51.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:50 smithi012 ceph-mon[32460]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-02-18T11:38:51.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:50 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/4284540463' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-02-18T11:38:51.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:50 smithi188 ceph-mon[36925]: pgmap v286: 97 pgs: 25 active+clean, 49 creating+peering, 23 unknown; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-18T11:38:51.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:50 smithi188 ceph-mon[36925]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-02-18T11:38:51.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:50 smithi188 ceph-mon[36925]: Cluster is now healthy 2024-02-18T11:38:51.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:50 smithi188 ceph-mon[36925]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-02-18T11:38:51.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:50 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/1987242148' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-02-18T11:38:51.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:50 smithi188 ceph-mon[36925]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-02-18T11:38:51.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:50 smithi188 ceph-mon[36925]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-18T11:38:51.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:50 smithi188 ceph-mon[36925]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-02-18T11:38:51.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:50 smithi188 ceph-mon[36925]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-02-18T11:38:51.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:50 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/4284540463' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-02-18T11:38:51.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:51.209+0000 7fbdbaedf640 1 -- 172.21.15.12:0/1006777121 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7fbdb410cd20 con 0x7fbdb41011d0 2024-02-18T11:38:51.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:50 smithi158 ceph-mon[36626]: pgmap v286: 97 pgs: 25 active+clean, 49 creating+peering, 23 unknown; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-18T11:38:51.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:50 smithi158 ceph-mon[36626]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-02-18T11:38:51.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:50 smithi158 ceph-mon[36626]: Cluster is now healthy 2024-02-18T11:38:51.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:50 smithi158 ceph-mon[36626]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-02-18T11:38:51.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:50 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/1987242148' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-02-18T11:38:51.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:50 smithi158 ceph-mon[36626]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-02-18T11:38:51.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:50 smithi158 ceph-mon[36626]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-18T11:38:51.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:50 smithi158 ceph-mon[36626]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-02-18T11:38:51.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:50 smithi158 ceph-mon[36626]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-02-18T11:38:51.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:50 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/4284540463' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-02-18T11:38:51.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:51.534+0000 7fbdb1ffb640 1 -- 172.21.15.12:0/1006777121 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v20) v1 ==== 115+0+0 (secure 0 0 0) 0x7fbdb410cd20 con 0x7fbdb41011d0 2024-02-18T11:38:51.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:51.536+0000 7fbdbaedf640 1 -- 172.21.15.12:0/1006777121 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fbd9c0a27d0 msgr2=0x7fbd9c0a4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:51.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:51.536+0000 7fbdbaedf640 1 --2- 172.21.15.12:0/1006777121 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fbd9c0a27d0 0x7fbd9c0a4c90 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fbdb41987c0 tx=0x7fbda8005e50 comp rx=0 tx=0).stop 2024-02-18T11:38:51.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:51.536+0000 7fbdbaedf640 1 -- 172.21.15.12:0/1006777121 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbdb41011d0 msgr2=0x7fbdb41972b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:51.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:51.536+0000 7fbdbaedf640 1 --2- 172.21.15.12:0/1006777121 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbdb41011d0 0x7fbdb41972b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbda4000c00 tx=0x7fbda4004270 comp rx=0 tx=0).stop 2024-02-18T11:38:51.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:51.536+0000 7fbdbaedf640 1 -- 172.21.15.12:0/1006777121 shutdown_connections 2024-02-18T11:38:51.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:51.536+0000 7fbdbaedf640 1 --2- 172.21.15.12:0/1006777121 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fbd9c0a27d0 0x7fbd9c0a4c90 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:51.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:51.536+0000 7fbdbaedf640 1 --2- 172.21.15.12:0/1006777121 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbdb4102d80 0x7fbdb419e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:51.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:51.536+0000 7fbdbaedf640 1 --2- 172.21.15.12:0/1006777121 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbdb41023c0 0x7fbdb41977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:51.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:51.536+0000 7fbdbaedf640 1 --2- 172.21.15.12:0/1006777121 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbdb41011d0 0x7fbdb41972b0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:51.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:51.536+0000 7fbdbaedf640 1 -- 172.21.15.12:0/1006777121 >> 172.21.15.12:0/1006777121 conn(0x7fbdb40fc760 msgr2=0x7fbdb41057e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:51.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:51.536+0000 7fbdbaedf640 1 -- 172.21.15.12:0/1006777121 shutdown_connections 2024-02-18T11:38:51.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:51.536+0000 7fbdbaedf640 1 -- 172.21.15.12:0/1006777121 wait complete. 2024-02-18T11:38:51.544 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-02-18T11:38:51.545 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-02-18T11:38:51.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:51.659+0000 7f3196fd0640 1 -- 172.21.15.12:0/2030521117 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31901003a0 msgr2=0x7f3190100820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:51.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:51.659+0000 7f3196fd0640 1 --2- 172.21.15.12:0/2030521117 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31901003a0 0x7f3190100820 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f317c009930 tx=0x7f317c02bc30 comp rx=0 tx=0).stop 2024-02-18T11:38:51.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:51.659+0000 7f3196fd0640 1 -- 172.21.15.12:0/2030521117 shutdown_connections 2024-02-18T11:38:51.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:51.659+0000 7f3196fd0640 1 --2- 172.21.15.12:0/2030521117 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3190100d60 0x7f3190108600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:51.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:51.659+0000 7f3196fd0640 1 --2- 172.21.15.12:0/2030521117 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31901003a0 0x7f3190100820 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:51.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:51.659+0000 7f3196fd0640 1 --2- 172.21.15.12:0/2030521117 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3190101670 0x7f3190101a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:51.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:51.659+0000 7f3196fd0640 1 -- 172.21.15.12:0/2030521117 >> 172.21.15.12:0/2030521117 conn(0x7f31900fadb0 msgr2=0x7f31900fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:51.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:51.659+0000 7f3196fd0640 1 -- 172.21.15.12:0/2030521117 shutdown_connections 2024-02-18T11:38:51.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:51.659+0000 7f3196fd0640 1 -- 172.21.15.12:0/2030521117 wait complete. 2024-02-18T11:38:51.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:51.660+0000 7f3196fd0640 1 Processor -- start 2024-02-18T11:38:51.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:51.660+0000 7f3196fd0640 1 -- start start 2024-02-18T11:38:51.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:51.660+0000 7f3196fd0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31901003a0 0x7f31901995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:51.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:51.660+0000 7f3196fd0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3190100d60 0x7f3190199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:51.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:51.660+0000 7f3196fd0640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3190101670 0x7f31901a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:51.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:51.660+0000 7f3196fd0640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31901a10a0 con 0x7f31901003a0 2024-02-18T11:38:51.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:51.660+0000 7f3196fd0640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31901a1210 con 0x7f3190101670 2024-02-18T11:38:51.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:51.660+0000 7f3196fd0640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31901a13d0 con 0x7f3190100d60 2024-02-18T11:38:51.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:51.660+0000 7f3194d6c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31901003a0 0x7f31901995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:51.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:51.660+0000 7f3194d6c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31901003a0 0x7f31901995b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43516/0 (socket says 172.21.15.12:43516) 2024-02-18T11:38:51.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:51.660+0000 7f3194d6c640 1 -- 172.21.15.12:0/3318649314 learned_addr learned my addr 172.21.15.12:0/3318649314 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:38:51.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:51.660+0000 7f318ffff640 1 --2- 172.21.15.12:0/3318649314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3190100d60 0x7f3190199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:51.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:51.660+0000 7f319556d640 1 --2- 172.21.15.12:0/3318649314 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3190101670 0x7f31901a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:51.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:51.660+0000 7f3194d6c640 1 -- 172.21.15.12:0/3318649314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3190100d60 msgr2=0x7f3190199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:51.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:51.660+0000 7f3194d6c640 1 --2- 172.21.15.12:0/3318649314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3190100d60 0x7f3190199af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:51.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:51.660+0000 7f3194d6c640 1 -- 172.21.15.12:0/3318649314 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3190101670 msgr2=0x7f31901a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:51.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:51.660+0000 7f3194d6c640 1 --2- 172.21.15.12:0/3318649314 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3190101670 0x7f31901a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:51.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:51.660+0000 7f3194d6c640 1 -- 172.21.15.12:0/3318649314 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f317c009560 con 0x7f31901003a0 2024-02-18T11:38:51.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:51.660+0000 7f319556d640 1 --2- 172.21.15.12:0/3318649314 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3190101670 0x7f31901a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:38:51.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:51.660+0000 7f3194d6c640 1 --2- 172.21.15.12:0/3318649314 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31901003a0 0x7f31901995b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f318400d950 tx=0x7f318400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:51.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:51.661+0000 7f318dffb640 1 -- 172.21.15.12:0/3318649314 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3184009c30 con 0x7f31901003a0 2024-02-18T11:38:51.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:51.661+0000 7f3196fd0640 1 -- 172.21.15.12:0/3318649314 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31901a16b0 con 0x7f31901003a0 2024-02-18T11:38:51.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:51.661+0000 7f318dffb640 1 -- 172.21.15.12:0/3318649314 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3184009d90 con 0x7f31901003a0 2024-02-18T11:38:51.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:51.661+0000 7f318dffb640 1 -- 172.21.15.12:0/3318649314 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f318400f720 con 0x7f31901003a0 2024-02-18T11:38:51.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:51.661+0000 7f3196fd0640 1 -- 172.21.15.12:0/3318649314 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31901a1bc0 con 0x7f31901003a0 2024-02-18T11:38:51.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:51.662+0000 7f3196fd0640 1 -- 172.21.15.12:0/3318649314 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f319010bb70 con 0x7f31901003a0 2024-02-18T11:38:51.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:51.663+0000 7f318dffb640 1 -- 172.21.15.12:0/3318649314 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) v1 ==== 137245+0+0 (secure 0 0 0) 0x7f3184010a60 con 0x7f31901003a0 2024-02-18T11:38:51.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:51.663+0000 7f318dffb640 1 --2- 172.21.15.12:0/3318649314 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f31800a2750 0x7f31800a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:51.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:51.664+0000 7f318ffff640 1 --2- 172.21.15.12:0/3318649314 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f31800a2750 0x7f31800a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:51.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:51.664+0000 7f318dffb640 1 -- 172.21.15.12:0/3318649314 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f31840ce2f0 con 0x7f31901003a0 2024-02-18T11:38:51.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:51.664+0000 7f318ffff640 1 --2- 172.21.15.12:0/3318649314 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f31800a2750 0x7f31800a4c10 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f317c006010 tx=0x7f317c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:51.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:51.668+0000 7f318dffb640 1 -- 172.21.15.12:0/3318649314 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3184097130 con 0x7f31901003a0 2024-02-18T11:38:51.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:51.878+0000 7f3196fd0640 1 -- 172.21.15.12:0/3318649314 --> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f31900fd000 con 0x7f31800a2750 2024-02-18T11:38:51.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:51.902+0000 7f318dffb640 1 -- 172.21.15.12:0/3318649314 <== mgr.14152 v2:172.21.15.12:6800/1153736685 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f31900fd000 con 0x7f31800a2750 2024-02-18T11:38:51.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:51.904+0000 7f3196fd0640 1 -- 172.21.15.12:0/3318649314 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f31800a2750 msgr2=0x7f31800a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:51.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:51.904+0000 7f3196fd0640 1 --2- 172.21.15.12:0/3318649314 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f31800a2750 0x7f31800a4c10 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f317c006010 tx=0x7f317c000bc0 comp rx=0 tx=0).stop 2024-02-18T11:38:51.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:51.904+0000 7f3196fd0640 1 -- 172.21.15.12:0/3318649314 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31901003a0 msgr2=0x7f31901995b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:51.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:51.904+0000 7f3196fd0640 1 --2- 172.21.15.12:0/3318649314 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31901003a0 0x7f31901995b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f318400d950 tx=0x7f318400de10 comp rx=0 tx=0).stop 2024-02-18T11:38:51.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:51.904+0000 7f3196fd0640 1 -- 172.21.15.12:0/3318649314 shutdown_connections 2024-02-18T11:38:51.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:51.904+0000 7f3196fd0640 1 --2- 172.21.15.12:0/3318649314 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f31800a2750 0x7f31800a4c10 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:51.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:51.904+0000 7f3196fd0640 1 --2- 172.21.15.12:0/3318649314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3190100d60 0x7f3190199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:51.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:51.904+0000 7f3196fd0640 1 --2- 172.21.15.12:0/3318649314 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3190101670 0x7f31901a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:51.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:51.904+0000 7f3196fd0640 1 --2- 172.21.15.12:0/3318649314 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31901003a0 0x7f31901995b0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:51.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:51.904+0000 7f3196fd0640 1 -- 172.21.15.12:0/3318649314 >> 172.21.15.12:0/3318649314 conn(0x7f31900fadb0 msgr2=0x7f31900fc8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:51.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:51.904+0000 7f3196fd0640 1 -- 172.21.15.12:0/3318649314 shutdown_connections 2024-02-18T11:38:51.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:51.904+0000 7f3196fd0640 1 -- 172.21.15.12:0/3318649314 wait complete. 2024-02-18T11:38:51.909 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-02-18T11:38:52.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.026+0000 7f0d1d3e2640 1 -- 172.21.15.12:0/2587322464 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d181000c0 msgr2=0x7f0d181004e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:52.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.026+0000 7f0d1d3e2640 1 --2- 172.21.15.12:0/2587322464 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d181000c0 0x7f0d181004e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0d08009ca0 tx=0x7f0d0802c140 comp rx=0 tx=0).stop 2024-02-18T11:38:52.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.026+0000 7f0d1d3e2640 1 -- 172.21.15.12:0/2587322464 shutdown_connections 2024-02-18T11:38:52.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.026+0000 7f0d1d3e2640 1 --2- 172.21.15.12:0/2587322464 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d181012b0 0x7f0d18101730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:52.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.026+0000 7f0d1d3e2640 1 --2- 172.21.15.12:0/2587322464 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0d18101c70 0x7f0d18108500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:52.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.026+0000 7f0d1d3e2640 1 --2- 172.21.15.12:0/2587322464 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d181000c0 0x7f0d181004e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:52.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.026+0000 7f0d1d3e2640 1 -- 172.21.15.12:0/2587322464 >> 172.21.15.12:0/2587322464 conn(0x7f0d180fb650 msgr2=0x7f0d180fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:52.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.026+0000 7f0d1d3e2640 1 -- 172.21.15.12:0/2587322464 shutdown_connections 2024-02-18T11:38:52.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.026+0000 7f0d1d3e2640 1 -- 172.21.15.12:0/2587322464 wait complete. 2024-02-18T11:38:52.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.027+0000 7f0d1d3e2640 1 Processor -- start 2024-02-18T11:38:52.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.027+0000 7f0d1d3e2640 1 -- start start 2024-02-18T11:38:52.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.027+0000 7f0d1d3e2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d181000c0 0x7f0d181973e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:52.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.027+0000 7f0d1d3e2640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0d181012b0 0x7f0d18197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:52.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.027+0000 7f0d1d3e2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d18101c70 0x7f0d1819e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:52.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.027+0000 7f0d1d3e2640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1819eed0 con 0x7f0d181000c0 2024-02-18T11:38:52.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.027+0000 7f0d1d3e2640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1819f040 con 0x7f0d181012b0 2024-02-18T11:38:52.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.027+0000 7f0d1d3e2640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1819f200 con 0x7f0d18101c70 2024-02-18T11:38:52.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.027+0000 7f0d16ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d181000c0 0x7f0d181973e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:52.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.027+0000 7f0d16ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d181000c0 0x7f0d181973e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43558/0 (socket says 172.21.15.12:43558) 2024-02-18T11:38:52.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.027+0000 7f0d16ffd640 1 -- 172.21.15.12:0/3745061173 learned_addr learned my addr 172.21.15.12:0/3745061173 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:38:52.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.028+0000 7f0d177fe640 1 --2- 172.21.15.12:0/3745061173 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d18101c70 0x7f0d1819e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:52.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.028+0000 7f0d16ffd640 1 -- 172.21.15.12:0/3745061173 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d18101c70 msgr2=0x7f0d1819e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:52.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.028+0000 7f0d16ffd640 1 --2- 172.21.15.12:0/3745061173 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d18101c70 0x7f0d1819e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:52.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.028+0000 7f0d16ffd640 1 -- 172.21.15.12:0/3745061173 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0d181012b0 msgr2=0x7f0d18197920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:52.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.028+0000 7f0d16ffd640 1 --2- 172.21.15.12:0/3745061173 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0d181012b0 0x7f0d18197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:52.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.028+0000 7f0d16ffd640 1 -- 172.21.15.12:0/3745061173 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d08009900 con 0x7f0d181000c0 2024-02-18T11:38:52.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.028+0000 7f0d16ffd640 1 --2- 172.21.15.12:0/3745061173 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d181000c0 0x7f0d181973e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0d08037040 tx=0x7f0d08034f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:52.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.028+0000 7f0d0ffff640 1 -- 172.21.15.12:0/3745061173 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d08003970 con 0x7f0d181000c0 2024-02-18T11:38:52.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.028+0000 7f0d1d3e2640 1 -- 172.21.15.12:0/3745061173 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d1819f480 con 0x7f0d181000c0 2024-02-18T11:38:52.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.028+0000 7f0d0ffff640 1 -- 172.21.15.12:0/3745061173 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0d08004050 con 0x7f0d181000c0 2024-02-18T11:38:52.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.028+0000 7f0d0ffff640 1 -- 172.21.15.12:0/3745061173 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0803e740 con 0x7f0d181000c0 2024-02-18T11:38:52.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.028+0000 7f0d1d3e2640 1 -- 172.21.15.12:0/3745061173 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d1819fa00 con 0x7f0d181000c0 2024-02-18T11:38:52.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.029+0000 7f0d1d3e2640 1 -- 172.21.15.12:0/3745061173 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d18107500 con 0x7f0d181000c0 2024-02-18T11:38:52.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.030+0000 7f0d0ffff640 1 -- 172.21.15.12:0/3745061173 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) v1 ==== 137245+0+0 (secure 0 0 0) 0x7f0d0800a040 con 0x7f0d181000c0 2024-02-18T11:38:52.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.031+0000 7f0d0ffff640 1 --2- 172.21.15.12:0/3745061173 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f0d040a2750 0x7f0d040a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:52.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.031+0000 7f0d167fc640 1 --2- 172.21.15.12:0/3745061173 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f0d040a2750 0x7f0d040a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:52.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.031+0000 7f0d0ffff640 1 -- 172.21.15.12:0/3745061173 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f0d080f12b0 con 0x7f0d181000c0 2024-02-18T11:38:52.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.032+0000 7f0d167fc640 1 --2- 172.21.15.12:0/3745061173 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f0d040a2750 0x7f0d040a4c10 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0d181988f0 tx=0x7f0d00006c40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:52.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.036+0000 7f0d0ffff640 1 -- 172.21.15.12:0/3745061173 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0d080ba190 con 0x7f0d181000c0 2024-02-18T11:38:52.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:51 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/4284540463' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-02-18T11:38:52.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:51 smithi188 ceph-mon[36925]: mds.? [v2:172.21.15.188:6838/1342671190,v1:172.21.15.188:6839/1342671190] up:active 2024-02-18T11:38:52.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:51 smithi188 ceph-mon[36925]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-02-18T11:38:52.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:51 smithi188 ceph-mon[36925]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-02-18T11:38:52.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:51 smithi188 ceph-mon[36925]: Cluster is now healthy 2024-02-18T11:38:52.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:51 smithi188 ceph-mon[36925]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-02-18T11:38:52.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:51 smithi188 ceph-mon[36925]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-02-18T11:38:52.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:51 smithi188 ceph-mon[36925]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-02-18T11:38:52.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:51 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/1006777121' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-02-18T11:38:52.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:51 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/1006777121' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-02-18T11:38:52.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:51 smithi188 ceph-mon[36925]: mds.? [v2:172.21.15.188:6836/1036630143,v1:172.21.15.188:6837/1036630143] up:active 2024-02-18T11:38:52.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:51 smithi188 ceph-mon[36925]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-18T11:38:52.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:51 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/4284540463' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-02-18T11:38:52.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:51 smithi012 ceph-mon[32460]: mds.? [v2:172.21.15.188:6838/1342671190,v1:172.21.15.188:6839/1342671190] up:active 2024-02-18T11:38:52.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:51 smithi012 ceph-mon[32460]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-02-18T11:38:52.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:51 smithi012 ceph-mon[32460]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-02-18T11:38:52.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:51 smithi012 ceph-mon[32460]: Cluster is now healthy 2024-02-18T11:38:52.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:51 smithi012 ceph-mon[32460]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-02-18T11:38:52.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:51 smithi012 ceph-mon[32460]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-02-18T11:38:52.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:51 smithi012 ceph-mon[32460]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-02-18T11:38:52.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:51 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/1006777121' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-02-18T11:38:52.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:51 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/1006777121' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-02-18T11:38:52.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:51 smithi012 ceph-mon[32460]: mds.? [v2:172.21.15.188:6836/1036630143,v1:172.21.15.188:6837/1036630143] up:active 2024-02-18T11:38:52.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:51 smithi012 ceph-mon[32460]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-18T11:38:52.174 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:38:51 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:38:51.883+0000 7fb34a383700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-18T11:38:52.174 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:38:51 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:38:51.883+0000 7fb34a383700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-18T11:38:52.174 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:38:51 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:38:51.883+0000 7fb34a383700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-18T11:38:52.174 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:38:51 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:38:51.883+0000 7fb34a383700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-18T11:38:52.174 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:38:51 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:38:51.883+0000 7fb34a383700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-18T11:38:52.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.248+0000 7f0d1d3e2640 1 -- 172.21.15.12:0/3745061173 --> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f0d18061600 con 0x7f0d040a2750 2024-02-18T11:38:52.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:51 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/4284540463' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-02-18T11:38:52.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:51 smithi158 ceph-mon[36626]: mds.? [v2:172.21.15.188:6838/1342671190,v1:172.21.15.188:6839/1342671190] up:active 2024-02-18T11:38:52.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:51 smithi158 ceph-mon[36626]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-02-18T11:38:52.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:51 smithi158 ceph-mon[36626]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-02-18T11:38:52.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:51 smithi158 ceph-mon[36626]: Cluster is now healthy 2024-02-18T11:38:52.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:51 smithi158 ceph-mon[36626]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-02-18T11:38:52.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:51 smithi158 ceph-mon[36626]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-02-18T11:38:52.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:51 smithi158 ceph-mon[36626]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-02-18T11:38:52.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:51 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/1006777121' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-02-18T11:38:52.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:51 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/1006777121' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-02-18T11:38:52.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:51 smithi158 ceph-mon[36626]: mds.? [v2:172.21.15.188:6836/1036630143,v1:172.21.15.188:6837/1036630143] up:active 2024-02-18T11:38:52.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:51 smithi158 ceph-mon[36626]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-18T11:38:52.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.269+0000 7f0d0ffff640 1 -- 172.21.15.12:0/3745061173 <== mgr.14152 v2:172.21.15.12:6800/1153736685 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f0d18061600 con 0x7f0d040a2750 2024-02-18T11:38:52.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.271+0000 7f0d1d3e2640 1 -- 172.21.15.12:0/3745061173 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f0d040a2750 msgr2=0x7f0d040a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:52.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.271+0000 7f0d1d3e2640 1 --2- 172.21.15.12:0/3745061173 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f0d040a2750 0x7f0d040a4c10 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0d181988f0 tx=0x7f0d00006c40 comp rx=0 tx=0).stop 2024-02-18T11:38:52.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.271+0000 7f0d1d3e2640 1 -- 172.21.15.12:0/3745061173 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d181000c0 msgr2=0x7f0d181973e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:52.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.271+0000 7f0d1d3e2640 1 --2- 172.21.15.12:0/3745061173 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d181000c0 0x7f0d181973e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0d08037040 tx=0x7f0d08034f10 comp rx=0 tx=0).stop 2024-02-18T11:38:52.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.271+0000 7f0d1d3e2640 1 -- 172.21.15.12:0/3745061173 shutdown_connections 2024-02-18T11:38:52.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.271+0000 7f0d1d3e2640 1 --2- 172.21.15.12:0/3745061173 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f0d040a2750 0x7f0d040a4c10 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:52.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.271+0000 7f0d1d3e2640 1 --2- 172.21.15.12:0/3745061173 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d18101c70 0x7f0d1819e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:52.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.271+0000 7f0d1d3e2640 1 --2- 172.21.15.12:0/3745061173 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0d181012b0 0x7f0d18197920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:52.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.271+0000 7f0d1d3e2640 1 --2- 172.21.15.12:0/3745061173 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d181000c0 0x7f0d181973e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:52.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.271+0000 7f0d1d3e2640 1 -- 172.21.15.12:0/3745061173 >> 172.21.15.12:0/3745061173 conn(0x7f0d180fb650 msgr2=0x7f0d181046d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:52.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.272+0000 7f0d1d3e2640 1 -- 172.21.15.12:0/3745061173 shutdown_connections 2024-02-18T11:38:52.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.272+0000 7f0d1d3e2640 1 -- 172.21.15.12:0/3745061173 wait complete. 2024-02-18T11:38:52.279 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-02-18T11:38:52.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.395+0000 7f5b23ee1640 1 -- 172.21.15.12:0/3675929090 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b1c1012b0 msgr2=0x7f5b1c101730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:52.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.395+0000 7f5b23ee1640 1 --2- 172.21.15.12:0/3675929090 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b1c1012b0 0x7f5b1c101730 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5b18009900 tx=0x7f5b1802bc30 comp rx=0 tx=0).stop 2024-02-18T11:38:52.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.395+0000 7f5b23ee1640 1 -- 172.21.15.12:0/3675929090 shutdown_connections 2024-02-18T11:38:52.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.395+0000 7f5b23ee1640 1 --2- 172.21.15.12:0/3675929090 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b1c101c70 0x7f5b1c108500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:52.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.395+0000 7f5b23ee1640 1 --2- 172.21.15.12:0/3675929090 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b1c1012b0 0x7f5b1c101730 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:52.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.395+0000 7f5b23ee1640 1 --2- 172.21.15.12:0/3675929090 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5b1c1000c0 0x7f5b1c1004e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:52.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.395+0000 7f5b23ee1640 1 -- 172.21.15.12:0/3675929090 >> 172.21.15.12:0/3675929090 conn(0x7f5b1c0fb650 msgr2=0x7f5b1c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:52.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.395+0000 7f5b23ee1640 1 -- 172.21.15.12:0/3675929090 shutdown_connections 2024-02-18T11:38:52.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.395+0000 7f5b23ee1640 1 -- 172.21.15.12:0/3675929090 wait complete. 2024-02-18T11:38:52.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.395+0000 7f5b23ee1640 1 Processor -- start 2024-02-18T11:38:52.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.395+0000 7f5b23ee1640 1 -- start start 2024-02-18T11:38:52.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.395+0000 7f5b23ee1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b1c1000c0 0x7f5b1c197260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:52.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.395+0000 7f5b23ee1640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5b1c1012b0 0x7f5b1c1977a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:52.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.395+0000 7f5b23ee1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b1c101c70 0x7f5b1c19e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:52.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.395+0000 7f5b23ee1640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1c19ed50 con 0x7f5b1c101c70 2024-02-18T11:38:52.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.395+0000 7f5b23ee1640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1c19eec0 con 0x7f5b1c1012b0 2024-02-18T11:38:52.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.395+0000 7f5b23ee1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1c19f080 con 0x7f5b1c1000c0 2024-02-18T11:38:52.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.395+0000 7f5b2247e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b1c101c70 0x7f5b1c19e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:52.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.395+0000 7f5b2247e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b1c101c70 0x7f5b1c19e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43586/0 (socket says 172.21.15.12:43586) 2024-02-18T11:38:52.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.395+0000 7f5b2247e640 1 -- 172.21.15.12:0/1878311146 learned_addr learned my addr 172.21.15.12:0/1878311146 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:38:52.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.396+0000 7f5b21c7d640 1 --2- 172.21.15.12:0/1878311146 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b1c1000c0 0x7f5b1c197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:52.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.396+0000 7f5b2147c640 1 --2- 172.21.15.12:0/1878311146 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5b1c1012b0 0x7f5b1c1977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:52.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.396+0000 7f5b2247e640 1 -- 172.21.15.12:0/1878311146 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b1c1000c0 msgr2=0x7f5b1c197260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:52.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.396+0000 7f5b2247e640 1 --2- 172.21.15.12:0/1878311146 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b1c1000c0 0x7f5b1c197260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:52.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.396+0000 7f5b2247e640 1 -- 172.21.15.12:0/1878311146 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5b1c1012b0 msgr2=0x7f5b1c1977a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:52.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.396+0000 7f5b2247e640 1 --2- 172.21.15.12:0/1878311146 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5b1c1012b0 0x7f5b1c1977a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:52.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.396+0000 7f5b2247e640 1 -- 172.21.15.12:0/1878311146 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b18009560 con 0x7f5b1c101c70 2024-02-18T11:38:52.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.396+0000 7f5b2247e640 1 --2- 172.21.15.12:0/1878311146 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b1c101c70 0x7f5b1c19e810 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5b1400eaa0 tx=0x7f5b1400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:52.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.396+0000 7f5b0effd640 1 -- 172.21.15.12:0/1878311146 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1400cdb0 con 0x7f5b1c101c70 2024-02-18T11:38:52.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.396+0000 7f5b23ee1640 1 -- 172.21.15.12:0/1878311146 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b1c19f360 con 0x7f5b1c101c70 2024-02-18T11:38:52.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.396+0000 7f5b0effd640 1 -- 172.21.15.12:0/1878311146 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5b14005020 con 0x7f5b1c101c70 2024-02-18T11:38:52.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.396+0000 7f5b0effd640 1 -- 172.21.15.12:0/1878311146 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b140116d0 con 0x7f5b1c101c70 2024-02-18T11:38:52.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.396+0000 7f5b23ee1640 1 -- 172.21.15.12:0/1878311146 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b1c19f870 con 0x7f5b1c101c70 2024-02-18T11:38:52.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.397+0000 7f5b23ee1640 1 -- 172.21.15.12:0/1878311146 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b1c107400 con 0x7f5b1c101c70 2024-02-18T11:38:52.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.398+0000 7f5b0effd640 1 -- 172.21.15.12:0/1878311146 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) v1 ==== 137245+0+0 (secure 0 0 0) 0x7f5b14005490 con 0x7f5b1c101c70 2024-02-18T11:38:52.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.399+0000 7f5b0effd640 1 --2- 172.21.15.12:0/1878311146 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f5b080a27d0 0x7f5b080a4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:52.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.399+0000 7f5b21c7d640 1 --2- 172.21.15.12:0/1878311146 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f5b080a27d0 0x7f5b080a4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:52.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.399+0000 7f5b0effd640 1 -- 172.21.15.12:0/1878311146 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f5b140cea30 con 0x7f5b1c101c70 2024-02-18T11:38:52.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.399+0000 7f5b21c7d640 1 --2- 172.21.15.12:0/1878311146 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f5b080a27d0 0x7f5b080a4c90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c101110 tx=0x7f5b1000a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:52.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.403+0000 7f5b0effd640 1 -- 172.21.15.12:0/1878311146 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5b14097870 con 0x7f5b1c101c70 2024-02-18T11:38:52.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.614+0000 7f5b23ee1640 1 -- 172.21.15.12:0/1878311146 --> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7f5b1c02cef0 con 0x7f5b080a27d0 2024-02-18T11:38:52.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.618+0000 7f5b0effd640 1 -- 172.21.15.12:0/1878311146 <== mgr.14152 v2:172.21.15.12:6800/1153736685 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7f5b1c02cef0 con 0x7f5b080a27d0 2024-02-18T11:38:52.612 INFO:teuthology.orchestra.run.smithi012.stdout:/volumes/qa/sv_0/8b8ca1d0-1a29-4763-8e4e-15646b3d921b 2024-02-18T11:38:52.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.619+0000 7f5b23ee1640 1 -- 172.21.15.12:0/1878311146 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f5b080a27d0 msgr2=0x7f5b080a4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:52.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.619+0000 7f5b23ee1640 1 --2- 172.21.15.12:0/1878311146 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f5b080a27d0 0x7f5b080a4c90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c101110 tx=0x7f5b1000a2d0 comp rx=0 tx=0).stop 2024-02-18T11:38:52.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.619+0000 7f5b23ee1640 1 -- 172.21.15.12:0/1878311146 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b1c101c70 msgr2=0x7f5b1c19e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:52.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.619+0000 7f5b23ee1640 1 --2- 172.21.15.12:0/1878311146 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b1c101c70 0x7f5b1c19e810 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5b1400eaa0 tx=0x7f5b1400ef60 comp rx=0 tx=0).stop 2024-02-18T11:38:52.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.620+0000 7f5b23ee1640 1 -- 172.21.15.12:0/1878311146 shutdown_connections 2024-02-18T11:38:52.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.620+0000 7f5b23ee1640 1 --2- 172.21.15.12:0/1878311146 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f5b080a27d0 0x7f5b080a4c90 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:52.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.620+0000 7f5b23ee1640 1 --2- 172.21.15.12:0/1878311146 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b1c101c70 0x7f5b1c19e810 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:52.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.620+0000 7f5b23ee1640 1 --2- 172.21.15.12:0/1878311146 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5b1c1012b0 0x7f5b1c1977a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:52.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.620+0000 7f5b23ee1640 1 --2- 172.21.15.12:0/1878311146 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b1c1000c0 0x7f5b1c197260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:52.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.620+0000 7f5b23ee1640 1 -- 172.21.15.12:0/1878311146 >> 172.21.15.12:0/1878311146 conn(0x7f5b1c0fb650 msgr2=0x7f5b1c1045d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:52.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.620+0000 7f5b23ee1640 1 -- 172.21.15.12:0/1878311146 shutdown_connections 2024-02-18T11:38:52.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.620+0000 7f5b23ee1640 1 -- 172.21.15.12:0/1878311146 wait complete. 2024-02-18T11:38:52.624 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-02-18T11:38:52.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.739+0000 7f5adc4df640 1 -- 172.21.15.12:0/3681665847 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ad4106ea0 msgr2=0x7f5ad41092f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:52.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.739+0000 7f5adc4df640 1 --2- 172.21.15.12:0/3681665847 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ad4106ea0 0x7f5ad41092f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5acc009ca0 tx=0x7f5acc02c0f0 comp rx=0 tx=0).stop 2024-02-18T11:38:52.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.739+0000 7f5adc4df640 1 -- 172.21.15.12:0/3681665847 shutdown_connections 2024-02-18T11:38:52.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.739+0000 7f5adc4df640 1 --2- 172.21.15.12:0/3681665847 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ad4106ea0 0x7f5ad41092f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:52.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.739+0000 7f5adc4df640 1 --2- 172.21.15.12:0/3681665847 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5ad4104570 0x7f5ad4106960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:52.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.739+0000 7f5adc4df640 1 --2- 172.21.15.12:0/3681665847 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ad4101c40 0x7f5ad4104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:52.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.739+0000 7f5adc4df640 1 -- 172.21.15.12:0/3681665847 >> 172.21.15.12:0/3681665847 conn(0x7f5ad40fb650 msgr2=0x7f5ad40fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:52.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.739+0000 7f5adc4df640 1 -- 172.21.15.12:0/3681665847 shutdown_connections 2024-02-18T11:38:52.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.739+0000 7f5adc4df640 1 -- 172.21.15.12:0/3681665847 wait complete. 2024-02-18T11:38:52.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.740+0000 7f5adc4df640 1 Processor -- start 2024-02-18T11:38:52.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.740+0000 7f5adc4df640 1 -- start start 2024-02-18T11:38:52.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.740+0000 7f5adc4df640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ad4101c40 0x7f5ad4197210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:52.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.740+0000 7f5adc4df640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5ad4104570 0x7f5ad4197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:52.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.740+0000 7f5adc4df640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ad4106ea0 0x7f5ad419e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:52.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.740+0000 7f5adc4df640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad419ed00 con 0x7f5ad4106ea0 2024-02-18T11:38:52.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.740+0000 7f5adc4df640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad419ee70 con 0x7f5ad4104570 2024-02-18T11:38:52.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.740+0000 7f5adc4df640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad419f030 con 0x7f5ad4101c40 2024-02-18T11:38:52.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.740+0000 7f5adaa7c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ad4106ea0 0x7f5ad419e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:52.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.740+0000 7f5adaa7c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ad4106ea0 0x7f5ad419e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43618/0 (socket says 172.21.15.12:43618) 2024-02-18T11:38:52.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.740+0000 7f5adaa7c640 1 -- 172.21.15.12:0/435520579 learned_addr learned my addr 172.21.15.12:0/435520579 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:38:52.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.740+0000 7f5ad9a7a640 1 --2- 172.21.15.12:0/435520579 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5ad4104570 0x7f5ad4197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:52.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.740+0000 7f5adaa7c640 1 -- 172.21.15.12:0/435520579 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ad4101c40 msgr2=0x7f5ad4197210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:52.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.740+0000 7f5adaa7c640 1 --2- 172.21.15.12:0/435520579 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ad4101c40 0x7f5ad4197210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:52.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.740+0000 7f5adaa7c640 1 -- 172.21.15.12:0/435520579 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5ad4104570 msgr2=0x7f5ad4197750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:52.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.740+0000 7f5adaa7c640 1 --2- 172.21.15.12:0/435520579 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5ad4104570 0x7f5ad4197750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:52.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.740+0000 7f5adaa7c640 1 -- 172.21.15.12:0/435520579 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5acc009900 con 0x7f5ad4106ea0 2024-02-18T11:38:52.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.740+0000 7f5ad9a7a640 1 --2- 172.21.15.12:0/435520579 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5ad4104570 0x7f5ad4197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:38:52.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.740+0000 7f5adaa7c640 1 --2- 172.21.15.12:0/435520579 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ad4106ea0 0x7f5ad419e7c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5acc037040 tx=0x7f5acc034d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:52.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.740+0000 7f5ac77fe640 1 -- 172.21.15.12:0/435520579 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5acc032eb0 con 0x7f5ad4106ea0 2024-02-18T11:38:52.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.741+0000 7f5adc4df640 1 -- 172.21.15.12:0/435520579 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ad419f2b0 con 0x7f5ad4106ea0 2024-02-18T11:38:52.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.741+0000 7f5ac77fe640 1 -- 172.21.15.12:0/435520579 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5acc003e50 con 0x7f5ad4106ea0 2024-02-18T11:38:52.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.741+0000 7f5ac77fe640 1 -- 172.21.15.12:0/435520579 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5acc040450 con 0x7f5ad4106ea0 2024-02-18T11:38:52.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.741+0000 7f5adc4df640 1 -- 172.21.15.12:0/435520579 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ad419f760 con 0x7f5ad4106ea0 2024-02-18T11:38:52.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.742+0000 7f5adc4df640 1 -- 172.21.15.12:0/435520579 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ad40fd140 con 0x7f5ad4106ea0 2024-02-18T11:38:52.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.743+0000 7f5ac77fe640 1 -- 172.21.15.12:0/435520579 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) v1 ==== 137245+0+0 (secure 0 0 0) 0x7f5acc00a040 con 0x7f5ad4106ea0 2024-02-18T11:38:52.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.743+0000 7f5ac77fe640 1 --2- 172.21.15.12:0/435520579 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f5ac00a2750 0x7f5ac00a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:52.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.743+0000 7f5ada27b640 1 --2- 172.21.15.12:0/435520579 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f5ac00a2750 0x7f5ac00a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:52.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.744+0000 7f5ac77fe640 1 -- 172.21.15.12:0/435520579 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f5acc0f1090 con 0x7f5ad4106ea0 2024-02-18T11:38:52.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.744+0000 7f5ada27b640 1 --2- 172.21.15.12:0/435520579 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f5ac00a2750 0x7f5ac00a4c10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5ac8005fd0 tx=0x7f5ac8005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:52.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.748+0000 7f5ac77fe640 1 -- 172.21.15.12:0/435520579 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5acc0b9f70 con 0x7f5ad4106ea0 2024-02-18T11:38:52.849 INFO:journalctl@ceph.mds.b.smithi158.stdout:Feb 18 11:38:52 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-b[66835]: 2024-02-18T11:38:52.556+0000 7fea469fd700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-02-18T11:38:52.849 INFO:journalctl@ceph.mds.b.smithi158.stdout:Feb 18 11:38:52 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-b[66835]: 2024-02-18T11:38:52.556+0000 7fea469fd700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-02-18T11:38:52.849 INFO:journalctl@ceph.mds.b.smithi158.stdout:Feb 18 11:38:52 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-b[66835]: 2024-02-18T11:38:52.556+0000 7fea469fd700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-02-18T11:38:52.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:52 smithi012 ceph-mon[32460]: pgmap v287: 97 pgs: 56 active+clean, 39 creating+peering, 2 unknown; 578 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s wr, 10 op/s 2024-02-18T11:38:52.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:52 smithi012 ceph-mon[32460]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:38:52.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.961+0000 7f5adc4df640 1 -- 172.21.15.12:0/435520579 --> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f5ad402cf20 con 0x7f5ac00a2750 2024-02-18T11:38:52.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.982+0000 7f5ac77fe640 1 -- 172.21.15.12:0/435520579 <== mgr.14152 v2:172.21.15.12:6800/1153736685 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f5ad402cf20 con 0x7f5ac00a2750 2024-02-18T11:38:52.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.985+0000 7f5adc4df640 1 -- 172.21.15.12:0/435520579 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f5ac00a2750 msgr2=0x7f5ac00a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:52.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.985+0000 7f5adc4df640 1 --2- 172.21.15.12:0/435520579 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f5ac00a2750 0x7f5ac00a4c10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5ac8005fd0 tx=0x7f5ac8005850 comp rx=0 tx=0).stop 2024-02-18T11:38:52.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.985+0000 7f5adc4df640 1 -- 172.21.15.12:0/435520579 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ad4106ea0 msgr2=0x7f5ad419e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:52.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.985+0000 7f5adc4df640 1 --2- 172.21.15.12:0/435520579 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ad4106ea0 0x7f5ad419e7c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5acc037040 tx=0x7f5acc034d40 comp rx=0 tx=0).stop 2024-02-18T11:38:52.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.985+0000 7f5adc4df640 1 -- 172.21.15.12:0/435520579 shutdown_connections 2024-02-18T11:38:52.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.985+0000 7f5adc4df640 1 --2- 172.21.15.12:0/435520579 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f5ac00a2750 0x7f5ac00a4c10 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:52.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.985+0000 7f5adc4df640 1 --2- 172.21.15.12:0/435520579 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ad4106ea0 0x7f5ad419e7c0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:52.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.985+0000 7f5adc4df640 1 --2- 172.21.15.12:0/435520579 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5ad4104570 0x7f5ad4197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:52.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.985+0000 7f5adc4df640 1 --2- 172.21.15.12:0/435520579 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ad4101c40 0x7f5ad4197210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:52.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.985+0000 7f5adc4df640 1 -- 172.21.15.12:0/435520579 >> 172.21.15.12:0/435520579 conn(0x7f5ad40fb650 msgr2=0x7f5ad4102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:52.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.985+0000 7f5adc4df640 1 -- 172.21.15.12:0/435520579 shutdown_connections 2024-02-18T11:38:52.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:52.985+0000 7f5adc4df640 1 -- 172.21.15.12:0/435520579 wait complete. 2024-02-18T11:38:52.990 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-02-18T11:38:53.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.103+0000 7ff4a3bd0640 1 -- 172.21.15.12:0/4109872292 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff49c106ec0 msgr2=0x7ff49c109310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:53.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.103+0000 7ff4a3bd0640 1 --2- 172.21.15.12:0/4109872292 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff49c106ec0 0x7ff49c109310 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff494009ca0 tx=0x7ff49402c0f0 comp rx=0 tx=0).stop 2024-02-18T11:38:53.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.103+0000 7ff4a3bd0640 1 -- 172.21.15.12:0/4109872292 shutdown_connections 2024-02-18T11:38:53.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.103+0000 7ff4a3bd0640 1 --2- 172.21.15.12:0/4109872292 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff49c106ec0 0x7ff49c109310 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:53.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.103+0000 7ff4a3bd0640 1 --2- 172.21.15.12:0/4109872292 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff49c104590 0x7ff49c106980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:53.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.103+0000 7ff4a3bd0640 1 --2- 172.21.15.12:0/4109872292 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff49c101c60 0x7ff49c104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:53.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.103+0000 7ff4a3bd0640 1 -- 172.21.15.12:0/4109872292 >> 172.21.15.12:0/4109872292 conn(0x7ff49c0fb650 msgr2=0x7ff49c0fda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:53.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.103+0000 7ff4a3bd0640 1 -- 172.21.15.12:0/4109872292 shutdown_connections 2024-02-18T11:38:53.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.103+0000 7ff4a3bd0640 1 -- 172.21.15.12:0/4109872292 wait complete. 2024-02-18T11:38:53.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.104+0000 7ff4a3bd0640 1 Processor -- start 2024-02-18T11:38:53.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.104+0000 7ff4a3bd0640 1 -- start start 2024-02-18T11:38:53.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.104+0000 7ff4a3bd0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff49c101c60 0x7ff49c197180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:53.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.104+0000 7ff4a3bd0640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff49c104590 0x7ff49c1976c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:53.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.104+0000 7ff4a3bd0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff49c106ec0 0x7ff49c19e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:53.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.104+0000 7ff4a3bd0640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff49c19ec70 con 0x7ff49c101c60 2024-02-18T11:38:53.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.104+0000 7ff4a3bd0640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff49c19ede0 con 0x7ff49c104590 2024-02-18T11:38:53.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.104+0000 7ff4a3bd0640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff49c19efa0 con 0x7ff49c106ec0 2024-02-18T11:38:53.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.104+0000 7ff4a196c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff49c101c60 0x7ff49c197180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:53.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.104+0000 7ff4a196c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff49c101c60 0x7ff49c197180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43624/0 (socket says 172.21.15.12:43624) 2024-02-18T11:38:53.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.104+0000 7ff4a196c640 1 -- 172.21.15.12:0/92384432 learned_addr learned my addr 172.21.15.12:0/92384432 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:38:53.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.104+0000 7ff4a116b640 1 --2- 172.21.15.12:0/92384432 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff49c104590 0x7ff49c1976c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:53.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.104+0000 7ff4a196c640 1 -- 172.21.15.12:0/92384432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff49c106ec0 msgr2=0x7ff49c19e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:53.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.104+0000 7ff4a196c640 1 --2- 172.21.15.12:0/92384432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff49c106ec0 0x7ff49c19e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:53.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.104+0000 7ff4a196c640 1 -- 172.21.15.12:0/92384432 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff49c104590 msgr2=0x7ff49c1976c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:53.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.104+0000 7ff4a196c640 1 --2- 172.21.15.12:0/92384432 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff49c104590 0x7ff49c1976c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:53.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.104+0000 7ff4a196c640 1 -- 172.21.15.12:0/92384432 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff494009900 con 0x7ff49c101c60 2024-02-18T11:38:53.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.104+0000 7ff4a196c640 1 --2- 172.21.15.12:0/92384432 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff49c101c60 0x7ff49c197180 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff49800d770 tx=0x7ff49800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:53.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.104+0000 7ff492ffd640 1 -- 172.21.15.12:0/92384432 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff498009c30 con 0x7ff49c101c60 2024-02-18T11:38:53.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.104+0000 7ff492ffd640 1 -- 172.21.15.12:0/92384432 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff49800de80 con 0x7ff49c101c60 2024-02-18T11:38:53.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.104+0000 7ff492ffd640 1 -- 172.21.15.12:0/92384432 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff498004e80 con 0x7ff49c101c60 2024-02-18T11:38:53.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.104+0000 7ff4a3bd0640 1 -- 172.21.15.12:0/92384432 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff49c19f280 con 0x7ff49c101c60 2024-02-18T11:38:53.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.105+0000 7ff4a3bd0640 1 -- 172.21.15.12:0/92384432 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff49c19f790 con 0x7ff49c101c60 2024-02-18T11:38:53.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.106+0000 7ff4a3bd0640 1 -- 172.21.15.12:0/92384432 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff49c0fd140 con 0x7ff49c101c60 2024-02-18T11:38:53.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.107+0000 7ff492ffd640 1 -- 172.21.15.12:0/92384432 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) v1 ==== 137245+0+0 (secure 0 0 0) 0x7ff498005000 con 0x7ff49c101c60 2024-02-18T11:38:53.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.107+0000 7ff492ffd640 1 --2- 172.21.15.12:0/92384432 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7ff4880a27d0 0x7ff4880a4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:53.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.107+0000 7ff4a116b640 1 --2- 172.21.15.12:0/92384432 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7ff4880a27d0 0x7ff4880a4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:53.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.107+0000 7ff492ffd640 1 -- 172.21.15.12:0/92384432 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff4980ce2c0 con 0x7ff49c101c60 2024-02-18T11:38:53.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.108+0000 7ff4a116b640 1 --2- 172.21.15.12:0/92384432 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7ff4880a27d0 0x7ff4880a4c90 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff48c005e10 tx=0x7ff48c005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:53.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:52 smithi188 ceph-mon[36925]: pgmap v287: 97 pgs: 56 active+clean, 39 creating+peering, 2 unknown; 578 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s wr, 10 op/s 2024-02-18T11:38:53.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:52 smithi188 ceph-mon[36925]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:38:53.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.111+0000 7ff492ffd640 1 -- 172.21.15.12:0/92384432 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff498097220 con 0x7ff49c101c60 2024-02-18T11:38:53.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:52 smithi158 ceph-mon[36626]: pgmap v287: 97 pgs: 56 active+clean, 39 creating+peering, 2 unknown; 578 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s wr, 10 op/s 2024-02-18T11:38:53.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:52 smithi158 ceph-mon[36626]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:38:53.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.337+0000 7ff4a3bd0640 1 -- 172.21.15.12:0/92384432 --> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7ff49c02cf50 con 0x7ff4880a27d0 2024-02-18T11:38:53.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.340+0000 7ff492ffd640 1 -- 172.21.15.12:0/92384432 <== mgr.14152 v2:172.21.15.12:6800/1153736685 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7ff49c02cf50 con 0x7ff4880a27d0 2024-02-18T11:38:53.335 INFO:teuthology.orchestra.run.smithi012.stdout:/volumes/qa/sv_1/5fe0f653-6870-4e9c-aa7a-5f0a419fa44a 2024-02-18T11:38:53.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.342+0000 7ff4a3bd0640 1 -- 172.21.15.12:0/92384432 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7ff4880a27d0 msgr2=0x7ff4880a4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:53.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.342+0000 7ff4a3bd0640 1 --2- 172.21.15.12:0/92384432 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7ff4880a27d0 0x7ff4880a4c90 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff48c005e10 tx=0x7ff48c005da0 comp rx=0 tx=0).stop 2024-02-18T11:38:53.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.343+0000 7ff4a3bd0640 1 -- 172.21.15.12:0/92384432 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff49c101c60 msgr2=0x7ff49c197180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:53.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.343+0000 7ff4a3bd0640 1 --2- 172.21.15.12:0/92384432 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff49c101c60 0x7ff49c197180 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff49800d770 tx=0x7ff49800dc30 comp rx=0 tx=0).stop 2024-02-18T11:38:53.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.343+0000 7ff4a3bd0640 1 -- 172.21.15.12:0/92384432 shutdown_connections 2024-02-18T11:38:53.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.343+0000 7ff4a3bd0640 1 --2- 172.21.15.12:0/92384432 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7ff4880a27d0 0x7ff4880a4c90 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:53.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.343+0000 7ff4a3bd0640 1 --2- 172.21.15.12:0/92384432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff49c106ec0 0x7ff49c19e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:53.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.343+0000 7ff4a3bd0640 1 --2- 172.21.15.12:0/92384432 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff49c104590 0x7ff49c1976c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:53.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.343+0000 7ff4a3bd0640 1 --2- 172.21.15.12:0/92384432 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff49c101c60 0x7ff49c197180 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:53.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.343+0000 7ff4a3bd0640 1 -- 172.21.15.12:0/92384432 >> 172.21.15.12:0/92384432 conn(0x7ff49c0fb650 msgr2=0x7ff49c102560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:53.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.343+0000 7ff4a3bd0640 1 -- 172.21.15.12:0/92384432 shutdown_connections 2024-02-18T11:38:53.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.343+0000 7ff4a3bd0640 1 -- 172.21.15.12:0/92384432 wait complete. 2024-02-18T11:38:53.347 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-18T11:38:53.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.456+0000 7f148fc98640 1 -- 172.21.15.12:0/4101347391 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1488101420 msgr2=0x7f1488107cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:53.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.456+0000 7f148fc98640 1 --2- 172.21.15.12:0/4101347391 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1488101420 0x7f1488107cb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1478009900 tx=0x7f147802bc30 comp rx=0 tx=0).stop 2024-02-18T11:38:53.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.456+0000 7f148fc98640 1 -- 172.21.15.12:0/4101347391 shutdown_connections 2024-02-18T11:38:53.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.456+0000 7f148fc98640 1 --2- 172.21.15.12:0/4101347391 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1488100a60 0x7f1488100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:53.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.456+0000 7f148fc98640 1 --2- 172.21.15.12:0/4101347391 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1488101420 0x7f1488107cb0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:53.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.456+0000 7f148fc98640 1 --2- 172.21.15.12:0/4101347391 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f14880ff870 0x7f14880ffc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:53.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.456+0000 7f148fc98640 1 -- 172.21.15.12:0/4101347391 >> 172.21.15.12:0/4101347391 conn(0x7f14880fade0 msgr2=0x7f14880fd220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:53.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.456+0000 7f148fc98640 1 -- 172.21.15.12:0/4101347391 shutdown_connections 2024-02-18T11:38:53.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.456+0000 7f148fc98640 1 -- 172.21.15.12:0/4101347391 wait complete. 2024-02-18T11:38:53.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.456+0000 7f148fc98640 1 Processor -- start 2024-02-18T11:38:53.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.456+0000 7f148fc98640 1 -- start start 2024-02-18T11:38:53.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.456+0000 7f148fc98640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f14880ff870 0x7f14881971e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:53.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.456+0000 7f148fc98640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1488100a60 0x7f1488197720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:53.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.456+0000 7f148fc98640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1488101420 0x7f148819e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:53.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.456+0000 7f148fc98640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f148819ecd0 con 0x7f1488100a60 2024-02-18T11:38:53.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.456+0000 7f148fc98640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f148819ee40 con 0x7f14880ff870 2024-02-18T11:38:53.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.456+0000 7f148fc98640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f148819f000 con 0x7f1488101420 2024-02-18T11:38:53.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.456+0000 7f148d233640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1488100a60 0x7f1488197720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:53.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.457+0000 7f148d233640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1488100a60 0x7f1488197720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43652/0 (socket says 172.21.15.12:43652) 2024-02-18T11:38:53.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.457+0000 7f148d233640 1 -- 172.21.15.12:0/3839470194 learned_addr learned my addr 172.21.15.12:0/3839470194 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:38:53.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.457+0000 7f148da34640 1 --2- 172.21.15.12:0/3839470194 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f14880ff870 0x7f14881971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:53.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.457+0000 7f148e235640 1 --2- 172.21.15.12:0/3839470194 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1488101420 0x7f148819e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:53.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.457+0000 7f148d233640 1 -- 172.21.15.12:0/3839470194 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1488101420 msgr2=0x7f148819e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:53.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.457+0000 7f148d233640 1 --2- 172.21.15.12:0/3839470194 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1488101420 0x7f148819e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:53.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.457+0000 7f148d233640 1 -- 172.21.15.12:0/3839470194 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f14880ff870 msgr2=0x7f14881971e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:53.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.457+0000 7f148d233640 1 --2- 172.21.15.12:0/3839470194 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f14880ff870 0x7f14881971e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:53.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.457+0000 7f148d233640 1 -- 172.21.15.12:0/3839470194 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1478009560 con 0x7f1488100a60 2024-02-18T11:38:53.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.457+0000 7f148da34640 1 --2- 172.21.15.12:0/3839470194 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f14880ff870 0x7f14881971e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:38:53.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.457+0000 7f148d233640 1 --2- 172.21.15.12:0/3839470194 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1488100a60 0x7f1488197720 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f147c00c890 tx=0x7f147c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:53.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.457+0000 7f1486ffd640 1 -- 172.21.15.12:0/3839470194 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f147c004220 con 0x7f1488100a60 2024-02-18T11:38:53.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.457+0000 7f148fc98640 1 -- 172.21.15.12:0/3839470194 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f148819f2e0 con 0x7f1488100a60 2024-02-18T11:38:53.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.457+0000 7f1486ffd640 1 -- 172.21.15.12:0/3839470194 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f147c007e90 con 0x7f1488100a60 2024-02-18T11:38:53.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.457+0000 7f1486ffd640 1 -- 172.21.15.12:0/3839470194 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f147c004a00 con 0x7f1488100a60 2024-02-18T11:38:53.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.457+0000 7f148fc98640 1 -- 172.21.15.12:0/3839470194 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f148819f7f0 con 0x7f1488100a60 2024-02-18T11:38:53.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.458+0000 7f148fc98640 1 -- 172.21.15.12:0/3839470194 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1488106b30 con 0x7f1488100a60 2024-02-18T11:38:53.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.459+0000 7f1486ffd640 1 -- 172.21.15.12:0/3839470194 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) v1 ==== 137245+0+0 (secure 0 0 0) 0x7f147c0079b0 con 0x7f1488100a60 2024-02-18T11:38:53.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.460+0000 7f1486ffd640 1 --2- 172.21.15.12:0/3839470194 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f14700a2750 0x7f14700a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:53.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.460+0000 7f148da34640 1 --2- 172.21.15.12:0/3839470194 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f14700a2750 0x7f14700a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:53.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.460+0000 7f1486ffd640 1 -- 172.21.15.12:0/3839470194 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f147c0ce160 con 0x7f1488100a60 2024-02-18T11:38:53.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.460+0000 7f148da34640 1 --2- 172.21.15.12:0/3839470194 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f14700a2750 0x7f14700a4c10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f14881008c0 tx=0x7f14800095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:53.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.464+0000 7f1486ffd640 1 -- 172.21.15.12:0/3839470194 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f147c097040 con 0x7f1488100a60 2024-02-18T11:38:53.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.683+0000 7f148fc98640 1 -- 172.21.15.12:0/3839470194 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f148810cc10 con 0x7f1488100a60 2024-02-18T11:38:53.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.684+0000 7f1486ffd640 1 -- 172.21.15.12:0/3839470194 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11349 (secure 0 0 0) 0x7f148810cc10 con 0x7f1488100a60 2024-02-18T11:38:53.678 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:38:53.678 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1788335302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1788335302},{"type":"v1","addr":"172.21.15.12:6835","nonce":1788335302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/2144692455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":2144692455},{"type":"v1","addr":"172.21.15.12:6837","nonce":2144692455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3569056036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3569056036},{"type":"v1","addr":"172.21.15.12:6839","nonce":3569056036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14622,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/1973234486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":1973234486},{"type":"v1","addr":"172.21.15.12:6841","nonce":1973234486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1207202465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1207202465},{"type":"v1","addr":"172.21.15.188:6833","nonce":1207202465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6833/2636348323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6832","nonce":2636348323},{"type":"v1","addr":"172.21.15.158:6833","nonce":2636348323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-18T11:38:46.888173+0000","modified":"2024-02-18T11:38:51.531552+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24524,"mds_1":24508,"mds_2":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.158:6835/3673253642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":3673253642},{"type":"v1","addr":"172.21.15.158:6835","nonce":3673253642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/806709644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":806709644},{"type":"v1","addr":"172.21.15.188:6835","nonce":806709644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.158:6837/813251825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6836","nonce":813251825},{"type":"v1","addr":"172.21.15.158:6837","nonce":813251825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.188:6837/1036630143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1036630143},{"type":"v1","addr":"172.21.15.188:6837","nonce":1036630143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.188:6839/1342671190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":1342671190},{"type":"v1","addr":"172.21.15.188:6839","nonce":1342671190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.158:6839/1127200849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6838","nonce":1127200849},{"type":"v1","addr":"172.21.15.158:6839","nonce":1127200849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-18T11:38:53.679 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 20 2024-02-18T11:38:53.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.686+0000 7f148fc98640 1 -- 172.21.15.12:0/3839470194 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f14700a2750 msgr2=0x7f14700a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:53.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.686+0000 7f148fc98640 1 --2- 172.21.15.12:0/3839470194 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f14700a2750 0x7f14700a4c10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f14881008c0 tx=0x7f14800095c0 comp rx=0 tx=0).stop 2024-02-18T11:38:53.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.686+0000 7f148fc98640 1 -- 172.21.15.12:0/3839470194 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1488100a60 msgr2=0x7f1488197720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:53.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.686+0000 7f148fc98640 1 --2- 172.21.15.12:0/3839470194 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1488100a60 0x7f1488197720 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f147c00c890 tx=0x7f147c00cd50 comp rx=0 tx=0).stop 2024-02-18T11:38:53.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.686+0000 7f148fc98640 1 -- 172.21.15.12:0/3839470194 shutdown_connections 2024-02-18T11:38:53.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.686+0000 7f148fc98640 1 --2- 172.21.15.12:0/3839470194 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f14700a2750 0x7f14700a4c10 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:53.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.686+0000 7f148fc98640 1 --2- 172.21.15.12:0/3839470194 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1488101420 0x7f148819e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:53.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.686+0000 7f148fc98640 1 --2- 172.21.15.12:0/3839470194 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1488100a60 0x7f1488197720 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:53.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.686+0000 7f148fc98640 1 --2- 172.21.15.12:0/3839470194 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f14880ff870 0x7f14881971e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:53.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.686+0000 7f148fc98640 1 -- 172.21.15.12:0/3839470194 >> 172.21.15.12:0/3839470194 conn(0x7f14880fade0 msgr2=0x7f1488103d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:53.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.686+0000 7f148fc98640 1 -- 172.21.15.12:0/3839470194 shutdown_connections 2024-02-18T11:38:53.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.686+0000 7f148fc98640 1 -- 172.21.15.12:0/3839470194 wait complete. 2024-02-18T11:38:53.691 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-18T11:38:53.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.799+0000 7f07732f1640 1 -- 172.21.15.12:0/3336817477 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f076c1000d0 msgr2=0x7f076c1004f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:53.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.799+0000 7f07732f1640 1 --2- 172.21.15.12:0/3336817477 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f076c1000d0 0x7f076c1004f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0768009900 tx=0x7f076802bc30 comp rx=0 tx=0).stop 2024-02-18T11:38:53.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.799+0000 7f07732f1640 1 -- 172.21.15.12:0/3336817477 shutdown_connections 2024-02-18T11:38:53.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.799+0000 7f07732f1640 1 --2- 172.21.15.12:0/3336817477 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f076c101c80 0x7f076c108510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:53.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.799+0000 7f07732f1640 1 --2- 172.21.15.12:0/3336817477 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f076c1012c0 0x7f076c101740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:53.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.799+0000 7f07732f1640 1 --2- 172.21.15.12:0/3336817477 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f076c1000d0 0x7f076c1004f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:53.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.799+0000 7f07732f1640 1 -- 172.21.15.12:0/3336817477 >> 172.21.15.12:0/3336817477 conn(0x7f076c0fb660 msgr2=0x7f076c0fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:53.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.799+0000 7f07732f1640 1 -- 172.21.15.12:0/3336817477 shutdown_connections 2024-02-18T11:38:53.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.799+0000 7f07732f1640 1 -- 172.21.15.12:0/3336817477 wait complete. 2024-02-18T11:38:53.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.799+0000 7f07732f1640 1 Processor -- start 2024-02-18T11:38:53.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.800+0000 7f07732f1640 1 -- start start 2024-02-18T11:38:53.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.800+0000 7f07732f1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f076c1000d0 0x7f076c06a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:53.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.800+0000 7f07732f1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f076c1012c0 0x7f076c19c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:53.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.800+0000 7f07732f1640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f076c101c80 0x7f076c19e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:53.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.800+0000 7f07732f1640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f076c19eda0 con 0x7f076c1000d0 2024-02-18T11:38:53.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.800+0000 7f07732f1640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f076c19ef10 con 0x7f076c101c80 2024-02-18T11:38:53.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.800+0000 7f07732f1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f076c19f0d0 con 0x7f076c1012c0 2024-02-18T11:38:53.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.800+0000 7f077108d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f076c1000d0 0x7f076c06a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:53.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.800+0000 7f077108d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f076c1000d0 0x7f076c06a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43674/0 (socket says 172.21.15.12:43674) 2024-02-18T11:38:53.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.800+0000 7f077108d640 1 -- 172.21.15.12:0/3275697730 learned_addr learned my addr 172.21.15.12:0/3275697730 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:38:53.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.800+0000 7f077188e640 1 --2- 172.21.15.12:0/3275697730 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f076c101c80 0x7f076c19e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:53.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.800+0000 7f077108d640 1 -- 172.21.15.12:0/3275697730 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f076c1012c0 msgr2=0x7f076c19c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:53.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.800+0000 7f077108d640 1 --2- 172.21.15.12:0/3275697730 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f076c1012c0 0x7f076c19c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:53.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.800+0000 7f077108d640 1 -- 172.21.15.12:0/3275697730 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f076c101c80 msgr2=0x7f076c19e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:53.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.800+0000 7f077108d640 1 --2- 172.21.15.12:0/3275697730 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f076c101c80 0x7f076c19e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:53.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.800+0000 7f077108d640 1 -- 172.21.15.12:0/3275697730 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0768009560 con 0x7f076c1000d0 2024-02-18T11:38:53.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.800+0000 7f077188e640 1 --2- 172.21.15.12:0/3275697730 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f076c101c80 0x7f076c19e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:38:53.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.800+0000 7f077108d640 1 --2- 172.21.15.12:0/3275697730 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f076c1000d0 0x7f076c06a350 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0768005ab0 tx=0x7f07680042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:53.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.800+0000 7f07627fc640 1 -- 172.21.15.12:0/3275697730 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07680044e0 con 0x7f076c1000d0 2024-02-18T11:38:53.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.800+0000 7f07732f1640 1 -- 172.21.15.12:0/3275697730 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f076c19f350 con 0x7f076c1000d0 2024-02-18T11:38:53.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.800+0000 7f07627fc640 1 -- 172.21.15.12:0/3275697730 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f07680054e0 con 0x7f076c1000d0 2024-02-18T11:38:53.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.800+0000 7f07627fc640 1 -- 172.21.15.12:0/3275697730 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f076803e6d0 con 0x7f076c1000d0 2024-02-18T11:38:53.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.800+0000 7f07732f1640 1 -- 172.21.15.12:0/3275697730 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f076c19f930 con 0x7f076c1000d0 2024-02-18T11:38:53.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.801+0000 7f07732f1640 1 -- 172.21.15.12:0/3275697730 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f076c107390 con 0x7f076c1000d0 2024-02-18T11:38:53.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.803+0000 7f07627fc640 1 -- 172.21.15.12:0/3275697730 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) v1 ==== 137245+0+0 (secure 0 0 0) 0x7f0768003780 con 0x7f076c1000d0 2024-02-18T11:38:53.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.804+0000 7f07627fc640 1 --2- 172.21.15.12:0/3275697730 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f07580a2750 0x7f07580a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:53.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.804+0000 7f077088c640 1 --2- 172.21.15.12:0/3275697730 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f07580a2750 0x7f07580a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:53.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.804+0000 7f07627fc640 1 -- 172.21.15.12:0/3275697730 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f07680f0f40 con 0x7f076c1000d0 2024-02-18T11:38:53.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.804+0000 7f077088c640 1 --2- 172.21.15.12:0/3275697730 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f07580a2750 0x7f07580a4c10 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f076c06a0c0 tx=0x7f075c005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:53.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.807+0000 7f07627fc640 1 -- 172.21.15.12:0/3275697730 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f07680b9d80 con 0x7f076c1000d0 2024-02-18T11:38:53.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.869+0000 7f07627fc640 1 -- 172.21.15.12:0/3275697730 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mgrmap(e 17) v1 ==== 137296+0+0 (secure 0 0 0) 0x7f07680ba150 con 0x7f076c1000d0 2024-02-18T11:38:53.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.995+0000 7f07732f1640 1 -- 172.21.15.12:0/3275697730 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f076c02ce80 con 0x7f076c1000d0 2024-02-18T11:38:53.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.996+0000 7f07627fc640 1 -- 172.21.15.12:0/3275697730 <== mon.0 v2:172.21.15.12:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22229 (secure 0 0 0) 0x7f07680ad8a0 con 0x7f076c1000d0 2024-02-18T11:38:53.991 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:38:53.991 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":87,"fsid":"0c8bb0de-ce51-11ee-95ba-87774f69a715","created":"2024-02-18T11:30:49.110059+0000","modified":"2024-02-18T11:38:48.817117+0000","last_up_change":"2024-02-18T11:37:57.512337+0000","last_in_change":"2024-02-18T11:37:36.567969+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-18T11:34:15.281138+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-18T11:38:44.802134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-18T11:38:46.081188+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"35f15ea6-deb8-48a1-a7dd-ab649ec13533","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":536522426},{"type":"v1","addr":"172.21.15.12:6803","nonce":536522426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":536522426},{"type":"v1","addr":"172.21.15.12:6805","nonce":536522426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":536522426},{"type":"v1","addr":"172.21.15.12:6809","nonce":536522426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":536522426},{"type":"v1","addr":"172.21.15.12:6807","nonce":536522426}]},"public_addr":"172.21.15.12:6803/536522426","cluster_addr":"172.21.15.12:6805/536522426","heartbeat_back_addr":"172.21.15.12:6809/536522426","heartbeat_front_addr":"172.21.15.12:6807/536522426","state":["exists","up"]},{"osd":1,"uuid":"2af937ef-d827-44e6-b805-70cffbfe1126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":4249111488},{"type":"v1","addr":"172.21.15.158:6801","nonce":4249111488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":4249111488},{"type":"v1","addr":"172.21.15.158:6803","nonce":4249111488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":4249111488},{"type":"v1","addr":"172.21.15.158:6807","nonce":4249111488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":4249111488},{"type":"v1","addr":"172.21.15.158:6805","nonce":4249111488}]},"public_addr":"172.21.15.158:6801/4249111488","cluster_addr":"172.21.15.158:6803/4249111488","heartbeat_back_addr":"172.21.15.158:6807/4249111488","heartbeat_front_addr":"172.21.15.158:6805/4249111488","state":["exists","up"]},{"osd":2,"uuid":"17b804dd-31df-45ae-a637-4a5f65bbaf26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":627976150},{"type":"v1","addr":"172.21.15.188:6801","nonce":627976150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":627976150},{"type":"v1","addr":"172.21.15.188:6803","nonce":627976150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":627976150},{"type":"v1","addr":"172.21.15.188:6807","nonce":627976150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":627976150},{"type":"v1","addr":"172.21.15.188:6805","nonce":627976150}]},"public_addr":"172.21.15.188:6801/627976150","cluster_addr":"172.21.15.188:6803/627976150","heartbeat_back_addr":"172.21.15.188:6807/627976150","heartbeat_front_addr":"172.21.15.188:6805/627976150","state":["exists","up"]},{"osd":3,"uuid":"3f9cc423-cdfa-4acf-aaa2-b58b32ccb9cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":3758512503},{"type":"v1","addr":"172.21.15.12:6811","nonce":3758512503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":3758512503},{"type":"v1","addr":"172.21.15.12:6813","nonce":3758512503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3758512503},{"type":"v1","addr":"172.21.15.12:6817","nonce":3758512503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":3758512503},{"type":"v1","addr":"172.21.15.12:6815","nonce":3758512503}]},"public_addr":"172.21.15.12:6811/3758512503","cluster_addr":"172.21.15.12:6813/3758512503","heartbeat_back_addr":"172.21.15.12:6817/3758512503","heartbeat_front_addr":"172.21.15.12:6815/3758512503","state":["exists","up"]},{"osd":4,"uuid":"a92e20ec-af1c-4ef4-a0cb-caba5fcc71aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":3892820997},{"type":"v1","addr":"172.21.15.158:6809","nonce":3892820997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":3892820997},{"type":"v1","addr":"172.21.15.158:6811","nonce":3892820997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":3892820997},{"type":"v1","addr":"172.21.15.158:6815","nonce":3892820997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":3892820997},{"type":"v1","addr":"172.21.15.158:6813","nonce":3892820997}]},"public_addr":"172.21.15.158:6809/3892820997","cluster_addr":"172.21.15.158:6811/3892820997","heartbeat_back_addr":"172.21.15.158:6815/3892820997","heartbeat_front_addr":"172.21.15.158:6813/3892820997","state":["exists","up"]},{"osd":5,"uuid":"d9c2e0c8-85f5-4ae2-bf3e-125c1be3a2db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3939495896},{"type":"v1","addr":"172.21.15.188:6809","nonce":3939495896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3939495896},{"type":"v1","addr":"172.21.15.188:6811","nonce":3939495896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3939495896},{"type":"v1","addr":"172.21.15.188:6815","nonce":3939495896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3939495896},{"type":"v1","addr":"172.21.15.188:6813","nonce":3939495896}]},"public_addr":"172.21.15.188:6809/3939495896","cluster_addr":"172.21.15.188:6811/3939495896","heartbeat_back_addr":"172.21.15.188:6815/3939495896","heartbeat_front_addr":"172.21.15.188:6813/3939495896","state":["exists","up"]},{"osd":6,"uuid":"4a523512-de6b-49f0-be81-00f2f00c1c10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":1206337767},{"type":"v1","addr":"172.21.15.12:6819","nonce":1206337767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":1206337767},{"type":"v1","addr":"172.21.15.12:6821","nonce":1206337767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":1206337767},{"type":"v1","addr":"172.21.15.12:6825","nonce":1206337767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":1206337767},{"type":"v1","addr":"172.21.15.12:6823","nonce":1206337767}]},"public_addr":"172.21.15.12:6819/1206337767","cluster_addr":"172.21.15.12:6821/1206337767","heartbeat_back_addr":"172.21.15.12:6825/1206337767","heartbeat_front_addr":"172.21.15.12:6823/1206337767","state":["exists","up"]},{"osd":7,"uuid":"363271a6-df93-44c9-b0e5-3d29768aa96c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":3283400032},{"type":"v1","addr":"172.21.15.158:6817","nonce":3283400032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":3283400032},{"type":"v1","addr":"172.21.15.158:6819","nonce":3283400032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":3283400032},{"type":"v1","addr":"172.21.15.158:6823","nonce":3283400032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":3283400032},{"type":"v1","addr":"172.21.15.158:6821","nonce":3283400032}]},"public_addr":"172.21.15.158:6817/3283400032","cluster_addr":"172.21.15.158:6819/3283400032","heartbeat_back_addr":"172.21.15.158:6823/3283400032","heartbeat_front_addr":"172.21.15.158:6821/3283400032","state":["exists","up"]},{"osd":8,"uuid":"9a089528-3e8e-45d3-9279-ddb54088b96a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2110170682},{"type":"v1","addr":"172.21.15.188:6817","nonce":2110170682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2110170682},{"type":"v1","addr":"172.21.15.188:6819","nonce":2110170682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":2110170682},{"type":"v1","addr":"172.21.15.188:6823","nonce":2110170682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2110170682},{"type":"v1","addr":"172.21.15.188:6821","nonce":2110170682}]},"public_addr":"172.21.15.188:6817/2110170682","cluster_addr":"172.21.15.188:6819/2110170682","heartbeat_back_addr":"172.21.15.188:6823/2110170682","heartbeat_front_addr":"172.21.15.188:6821/2110170682","state":["exists","up"]},{"osd":9,"uuid":"b12e2727-d649-4464-90e1-fafbc1754832","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1825928986},{"type":"v1","addr":"172.21.15.12:6827","nonce":1825928986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1825928986},{"type":"v1","addr":"172.21.15.12:6829","nonce":1825928986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":1825928986},{"type":"v1","addr":"172.21.15.12:6833","nonce":1825928986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1825928986},{"type":"v1","addr":"172.21.15.12:6831","nonce":1825928986}]},"public_addr":"172.21.15.12:6827/1825928986","cluster_addr":"172.21.15.12:6829/1825928986","heartbeat_back_addr":"172.21.15.12:6833/1825928986","heartbeat_front_addr":"172.21.15.12:6831/1825928986","state":["exists","up"]},{"osd":10,"uuid":"518b4cbf-9d90-4e9a-8ac9-b5cb9b3dee4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":3239052167},{"type":"v1","addr":"172.21.15.158:6825","nonce":3239052167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":3239052167},{"type":"v1","addr":"172.21.15.158:6827","nonce":3239052167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":3239052167},{"type":"v1","addr":"172.21.15.158:6831","nonce":3239052167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":3239052167},{"type":"v1","addr":"172.21.15.158:6829","nonce":3239052167}]},"public_addr":"172.21.15.158:6825/3239052167","cluster_addr":"172.21.15.158:6827/3239052167","heartbeat_back_addr":"172.21.15.158:6831/3239052167","heartbeat_front_addr":"172.21.15.158:6829/3239052167","state":["exists","up"]},{"osd":11,"uuid":"98aa3a0d-4b2f-46c8-9341-375d13794c3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1135773350},{"type":"v1","addr":"172.21.15.188:6825","nonce":1135773350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1135773350},{"type":"v1","addr":"172.21.15.188:6827","nonce":1135773350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1135773350},{"type":"v1","addr":"172.21.15.188:6831","nonce":1135773350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1135773350},{"type":"v1","addr":"172.21.15.188:6829","nonce":1135773350}]},"public_addr":"172.21.15.188:6825/1135773350","cluster_addr":"172.21.15.188:6827/1135773350","heartbeat_back_addr":"172.21.15.188:6831/1135773350","heartbeat_front_addr":"172.21.15.188:6829/1135773350","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:33:24.280879+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:33:47.859461+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:34:11.725283+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:34:35.707629+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:35:00.024197+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:35:24.499803+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:35:48.953870+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:36:13.642456+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:36:38.936457+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:37:04.021466+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:37:29.073178+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:37:54.863919+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:6801/4135274411":"2024-02-19T11:31:50.936928+0000","172.21.15.12:6800/4135274411":"2024-02-19T11:31:50.936928+0000","172.21.15.12:0/118428584":"2024-02-19T11:31:50.936928+0000","172.21.15.12:0/4276810915":"2024-02-19T11:31:50.936928+0000","172.21.15.12:0/3777545262":"2024-02-19T11:31:24.000830+0000","172.21.15.12:0/114707743":"2024-02-19T11:31:24.000830+0000","172.21.15.12:6800/3925204723":"2024-02-19T11:31:24.000830+0000","172.21.15.12:6801/3925204723":"2024-02-19T11:31:24.000830+0000","172.21.15.12:0/74881618":"2024-02-19T11:31:24.000830+0000","172.21.15.12:0/1614805375":"2024-02-19T11:31:50.936928+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-18T11:38:53.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.998+0000 7f07732f1640 1 -- 172.21.15.12:0/3275697730 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f07580a2750 msgr2=0x7f07580a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:53.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.998+0000 7f07732f1640 1 --2- 172.21.15.12:0/3275697730 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f07580a2750 0x7f07580a4c10 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f076c06a0c0 tx=0x7f075c005c70 comp rx=0 tx=0).stop 2024-02-18T11:38:53.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.998+0000 7f07732f1640 1 -- 172.21.15.12:0/3275697730 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f076c1000d0 msgr2=0x7f076c06a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:53.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.998+0000 7f07732f1640 1 --2- 172.21.15.12:0/3275697730 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f076c1000d0 0x7f076c06a350 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0768005ab0 tx=0x7f07680042f0 comp rx=0 tx=0).stop 2024-02-18T11:38:53.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.998+0000 7f07732f1640 1 -- 172.21.15.12:0/3275697730 shutdown_connections 2024-02-18T11:38:53.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.998+0000 7f07732f1640 1 --2- 172.21.15.12:0/3275697730 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f07580a2750 0x7f07580a4c10 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:53.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.998+0000 7f07732f1640 1 --2- 172.21.15.12:0/3275697730 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f076c1012c0 0x7f076c19c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:53.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.998+0000 7f07732f1640 1 --2- 172.21.15.12:0/3275697730 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f076c101c80 0x7f076c19e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:53.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.998+0000 7f07732f1640 1 --2- 172.21.15.12:0/3275697730 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f076c1000d0 0x7f076c06a350 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:53.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.998+0000 7f07732f1640 1 -- 172.21.15.12:0/3275697730 >> 172.21.15.12:0/3275697730 conn(0x7f076c0fb660 msgr2=0x7f076c104560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:53.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.998+0000 7f07732f1640 1 -- 172.21.15.12:0/3275697730 shutdown_connections 2024-02-18T11:38:53.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:53.999+0000 7f07732f1640 1 -- 172.21.15.12:0/3275697730 wait complete. 2024-02-18T11:38:54.002 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:53 smithi012 ceph-mon[32460]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:38:54.002 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:53 smithi012 ceph-mon[32460]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:38:54.002 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:53 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/3839470194' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-18T11:38:54.004 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-02-18T11:38:54.004 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-18T11:38:54.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.112+0000 7fba6a8fb640 1 -- 172.21.15.12:0/2301498354 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba64101440 msgr2=0x7fba64107cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:54.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.112+0000 7fba6a8fb640 1 --2- 172.21.15.12:0/2301498354 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba64101440 0x7fba64107cd0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fba5c009ca0 tx=0x7fba5c02c0f0 comp rx=0 tx=0).stop 2024-02-18T11:38:54.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.112+0000 7fba6a8fb640 1 -- 172.21.15.12:0/2301498354 shutdown_connections 2024-02-18T11:38:54.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.112+0000 7fba6a8fb640 1 --2- 172.21.15.12:0/2301498354 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba64101440 0x7fba64107cd0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:54.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.112+0000 7fba6a8fb640 1 --2- 172.21.15.12:0/2301498354 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fba64100a80 0x7fba64100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:54.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.112+0000 7fba6a8fb640 1 --2- 172.21.15.12:0/2301498354 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba640ff890 0x7fba640ffcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:54.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.112+0000 7fba6a8fb640 1 -- 172.21.15.12:0/2301498354 >> 172.21.15.12:0/2301498354 conn(0x7fba640fae00 msgr2=0x7fba640fd240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:54.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.112+0000 7fba6a8fb640 1 -- 172.21.15.12:0/2301498354 shutdown_connections 2024-02-18T11:38:54.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.112+0000 7fba6a8fb640 1 -- 172.21.15.12:0/2301498354 wait complete. 2024-02-18T11:38:54.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.112+0000 7fba6a8fb640 1 Processor -- start 2024-02-18T11:38:54.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.112+0000 7fba6a8fb640 1 -- start start 2024-02-18T11:38:54.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.113+0000 7fba6a8fb640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba640ff890 0x7fba641950b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:54.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.113+0000 7fba6a8fb640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fba64100a80 0x7fba641955f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:54.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.113+0000 7fba6a8fb640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba64101440 0x7fba6419c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:54.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.113+0000 7fba6a8fb640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba6419cba0 con 0x7fba64101440 2024-02-18T11:38:54.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.113+0000 7fba6a8fb640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba6419cd10 con 0x7fba64100a80 2024-02-18T11:38:54.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.113+0000 7fba6a8fb640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba6419ced0 con 0x7fba640ff890 2024-02-18T11:38:54.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.113+0000 7fba68e98640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba64101440 0x7fba6419c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:54.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.113+0000 7fba68e98640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba64101440 0x7fba6419c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43702/0 (socket says 172.21.15.12:43702) 2024-02-18T11:38:54.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.113+0000 7fba68e98640 1 -- 172.21.15.12:0/4281760485 learned_addr learned my addr 172.21.15.12:0/4281760485 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:38:54.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.113+0000 7fba63fff640 1 --2- 172.21.15.12:0/4281760485 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba640ff890 0x7fba641950b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:54.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.113+0000 7fba637fe640 1 --2- 172.21.15.12:0/4281760485 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fba64100a80 0x7fba641955f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:54.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.113+0000 7fba68e98640 1 -- 172.21.15.12:0/4281760485 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba640ff890 msgr2=0x7fba641950b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:54.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.113+0000 7fba68e98640 1 --2- 172.21.15.12:0/4281760485 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba640ff890 0x7fba641950b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:54.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.113+0000 7fba68e98640 1 -- 172.21.15.12:0/4281760485 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fba64100a80 msgr2=0x7fba641955f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:54.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.113+0000 7fba68e98640 1 --2- 172.21.15.12:0/4281760485 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fba64100a80 0x7fba641955f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:54.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.113+0000 7fba68e98640 1 -- 172.21.15.12:0/4281760485 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba5c009900 con 0x7fba64101440 2024-02-18T11:38:54.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.113+0000 7fba68e98640 1 --2- 172.21.15.12:0/4281760485 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba64101440 0x7fba6419c660 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fba5c032650 tx=0x7fba5c034d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:54.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.113+0000 7fba617fa640 1 -- 172.21.15.12:0/4281760485 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba5c032ed0 con 0x7fba64101440 2024-02-18T11:38:54.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.114+0000 7fba617fa640 1 -- 172.21.15.12:0/4281760485 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fba5c003e50 con 0x7fba64101440 2024-02-18T11:38:54.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.114+0000 7fba6a8fb640 1 -- 172.21.15.12:0/4281760485 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba6419d150 con 0x7fba64101440 2024-02-18T11:38:54.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.114+0000 7fba617fa640 1 -- 172.21.15.12:0/4281760485 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba5c040450 con 0x7fba64101440 2024-02-18T11:38:54.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.114+0000 7fba6a8fb640 1 -- 172.21.15.12:0/4281760485 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba6419d5d0 con 0x7fba64101440 2024-02-18T11:38:54.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.115+0000 7fba6a8fb640 1 -- 172.21.15.12:0/4281760485 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba64106b50 con 0x7fba64101440 2024-02-18T11:38:54.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.115+0000 7fba617fa640 1 -- 172.21.15.12:0/4281760485 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 137296+0+0 (secure 0 0 0) 0x7fba5c00a040 con 0x7fba64101440 2024-02-18T11:38:54.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.116+0000 7fba617fa640 1 --2- 172.21.15.12:0/4281760485 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fba4c0a27e0 0x7fba4c0a4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:54.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.116+0000 7fba63fff640 1 --2- 172.21.15.12:0/4281760485 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fba4c0a27e0 0x7fba4c0a4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:54.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.116+0000 7fba617fa640 1 -- 172.21.15.12:0/4281760485 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fba5c0f1050 con 0x7fba64101440 2024-02-18T11:38:54.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.117+0000 7fba63fff640 1 --2- 172.21.15.12:0/4281760485 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fba4c0a27e0 0x7fba4c0a4ca0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fba54005fd0 tx=0x7fba54010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:54.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.121+0000 7fba617fa640 1 -- 172.21.15.12:0/4281760485 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fba5c0b9e40 con 0x7fba64101440 2024-02-18T11:38:54.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:53 smithi158 ceph-mon[36626]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:38:54.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:53 smithi158 ceph-mon[36626]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:38:54.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:53 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/3839470194' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-18T11:38:54.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.298+0000 7fba6a8fb640 1 -- 172.21.15.12:0/4281760485 --> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fba64061600 con 0x7fba4c0a27e0 2024-02-18T11:38:54.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.306+0000 7fba617fa640 1 -- 172.21.15.12:0/4281760485 <== mgr.14152 v2:172.21.15.12:6800/1153736685 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+258779 (secure 0 0 0) 0x7fba64061600 con 0x7fba4c0a27e0 2024-02-18T11:38:54.303 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:38:54.305 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-02-18T11:38:54.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.309+0000 7fba6a8fb640 1 -- 172.21.15.12:0/4281760485 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fba4c0a27e0 msgr2=0x7fba4c0a4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:54.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.309+0000 7fba6a8fb640 1 --2- 172.21.15.12:0/4281760485 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fba4c0a27e0 0x7fba4c0a4ca0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fba54005fd0 tx=0x7fba54010040 comp rx=0 tx=0).stop 2024-02-18T11:38:54.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.309+0000 7fba6a8fb640 1 -- 172.21.15.12:0/4281760485 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba64101440 msgr2=0x7fba6419c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:54.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.309+0000 7fba6a8fb640 1 --2- 172.21.15.12:0/4281760485 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba64101440 0x7fba6419c660 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fba5c032650 tx=0x7fba5c034d40 comp rx=0 tx=0).stop 2024-02-18T11:38:54.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.309+0000 7fba6a8fb640 1 -- 172.21.15.12:0/4281760485 shutdown_connections 2024-02-18T11:38:54.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.309+0000 7fba6a8fb640 1 --2- 172.21.15.12:0/4281760485 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fba4c0a27e0 0x7fba4c0a4ca0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:54.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.309+0000 7fba6a8fb640 1 --2- 172.21.15.12:0/4281760485 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba64101440 0x7fba6419c660 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:54.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.309+0000 7fba6a8fb640 1 --2- 172.21.15.12:0/4281760485 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fba64100a80 0x7fba641955f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:54.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.309+0000 7fba6a8fb640 1 --2- 172.21.15.12:0/4281760485 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba640ff890 0x7fba641950b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:54.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.309+0000 7fba6a8fb640 1 -- 172.21.15.12:0/4281760485 >> 172.21.15.12:0/4281760485 conn(0x7fba640fae00 msgr2=0x7fba64103d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:54.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.309+0000 7fba6a8fb640 1 -- 172.21.15.12:0/4281760485 shutdown_connections 2024-02-18T11:38:54.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.309+0000 7fba6a8fb640 1 -- 172.21.15.12:0/4281760485 wait complete. 2024-02-18T11:38:54.314 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":288,"stamp":"2024-02-18T11:38:53.016022+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595309,"num_objects":54,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":139,"num_read_kb":120,"num_write":291,"num_write_kb":2975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":234,"ondisk_log_size":234,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":234,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":391128,"kb_used_data":12816,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124550696,"statfs":{"total":1151940427776,"available":1151539912704,"internally_reserved":0,"allocated":13123584,"data_stored":7110837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":387233082},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4941,"num_objects":52,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":13,"num_read_kb":11,"num_write":110,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.501108"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.855808+0000","last_change":"2024-02-18T11:38:47.847524+0000","last_active":"2024-02-18T11:38:49.855808+0000","last_peered":"2024-02-18T11:38:49.855808+0000","last_clean":"2024-02-18T11:38:49.855808+0000","last_became_active":"2024-02-18T11:38:47.846756+0000","last_became_peered":"2024-02-18T11:38:47.846756+0000","last_unstale":"2024-02-18T11:38:49.855808+0000","last_undegraded":"2024-02-18T11:38:49.855808+0000","last_fullsized":"2024-02-18T11:38:49.855808+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:23:02.715418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:48.827338+0000","last_change":"2024-02-18T11:38:47.847391+0000","last_active":"2024-02-18T11:38:48.827338+0000","last_peered":"2024-02-18T11:38:48.827338+0000","last_clean":"2024-02-18T11:38:48.827338+0000","last_became_active":"2024-02-18T11:38:47.846558+0000","last_became_peered":"2024-02-18T11:38:47.846558+0000","last_unstale":"2024-02-18T11:38:48.827338+0000","last_undegraded":"2024-02-18T11:38:48.827338+0000","last_fullsized":"2024-02-18T11:38:48.827338+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:37:34.940328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:48.827305+0000","last_change":"2024-02-18T11:38:47.850143+0000","last_active":"2024-02-18T11:38:48.827305+0000","last_peered":"2024-02-18T11:38:48.827305+0000","last_clean":"2024-02-18T11:38:48.827305+0000","last_became_active":"2024-02-18T11:38:47.845844+0000","last_became_peered":"2024-02-18T11:38:47.845844+0000","last_unstale":"2024-02-18T11:38:48.827305+0000","last_undegraded":"2024-02-18T11:38:48.827305+0000","last_fullsized":"2024-02-18T11:38:48.827305+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T12:45:15.638456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.088979+0000","last_change":"2024-02-18T11:38:47.841317+0000","last_active":"2024-02-18T11:38:49.088979+0000","last_peered":"2024-02-18T11:38:49.088979+0000","last_clean":"2024-02-18T11:38:49.088979+0000","last_became_active":"2024-02-18T11:38:47.840982+0000","last_became_peered":"2024-02-18T11:38:47.840982+0000","last_unstale":"2024-02-18T11:38:49.088979+0000","last_undegraded":"2024-02-18T11:38:49.088979+0000","last_fullsized":"2024-02-18T11:38:49.088979+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:53:40.115908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:48.832632+0000","last_change":"2024-02-18T11:38:47.851422+0000","last_active":"2024-02-18T11:38:48.832632+0000","last_peered":"2024-02-18T11:38:48.832632+0000","last_clean":"2024-02-18T11:38:48.832632+0000","last_became_active":"2024-02-18T11:38:47.850479+0000","last_became_peered":"2024-02-18T11:38:47.850479+0000","last_unstale":"2024-02-18T11:38:48.832632+0000","last_undegraded":"2024-02-18T11:38:48.832632+0000","last_fullsized":"2024-02-18T11:38:48.832632+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:09:21.200282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.853607+0000","last_change":"2024-02-18T11:38:47.837468+0000","last_active":"2024-02-18T11:38:49.853607+0000","last_peered":"2024-02-18T11:38:49.853607+0000","last_clean":"2024-02-18T11:38:49.853607+0000","last_became_active":"2024-02-18T11:38:47.837129+0000","last_became_peered":"2024-02-18T11:38:47.837129+0000","last_unstale":"2024-02-18T11:38:49.853607+0000","last_undegraded":"2024-02-18T11:38:49.853607+0000","last_fullsized":"2024-02-18T11:38:49.853607+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:12:01.954496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:48.827918+0000","last_change":"2024-02-18T11:38:47.849467+0000","last_active":"2024-02-18T11:38:48.827918+0000","last_peered":"2024-02-18T11:38:48.827918+0000","last_clean":"2024-02-18T11:38:48.827918+0000","last_became_active":"2024-02-18T11:38:47.848938+0000","last_became_peered":"2024-02-18T11:38:47.848938+0000","last_unstale":"2024-02-18T11:38:48.827918+0000","last_undegraded":"2024-02-18T11:38:48.827918+0000","last_fullsized":"2024-02-18T11:38:48.827918+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:39:16.680834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-18T11:38:47.841833+0000","last_change":"2024-02-18T11:38:47.830143+0000","last_active":"2024-02-18T11:38:47.841833+0000","last_peered":"2024-02-18T11:38:47.841833+0000","last_clean":"2024-02-18T11:38:47.841833+0000","last_became_active":"2024-02-18T11:38:47.829765+0000","last_became_peered":"2024-02-18T11:38:47.829765+0000","last_unstale":"2024-02-18T11:38:47.841833+0000","last_undegraded":"2024-02-18T11:38:47.841833+0000","last_fullsized":"2024-02-18T11:38:47.841833+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T12:43:49.777465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.857235+0000","last_change":"2024-02-18T11:38:47.840477+0000","last_active":"2024-02-18T11:38:49.857235+0000","last_peered":"2024-02-18T11:38:49.857235+0000","last_clean":"2024-02-18T11:38:49.857235+0000","last_became_active":"2024-02-18T11:38:47.840136+0000","last_became_peered":"2024-02-18T11:38:47.840136+0000","last_unstale":"2024-02-18T11:38:49.857235+0000","last_undegraded":"2024-02-18T11:38:49.857235+0000","last_fullsized":"2024-02-18T11:38:49.857235+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:27:28.272633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.854739+0000","last_change":"2024-02-18T11:38:47.836478+0000","last_active":"2024-02-18T11:38:49.854739+0000","last_peered":"2024-02-18T11:38:49.854739+0000","last_clean":"2024-02-18T11:38:49.854739+0000","last_became_active":"2024-02-18T11:38:47.836161+0000","last_became_peered":"2024-02-18T11:38:47.836161+0000","last_unstale":"2024-02-18T11:38:49.854739+0000","last_undegraded":"2024-02-18T11:38:49.854739+0000","last_fullsized":"2024-02-18T11:38:49.854739+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:28:13.828518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:48.828574+0000","last_change":"2024-02-18T11:38:47.849435+0000","last_active":"2024-02-18T11:38:48.828574+0000","last_peered":"2024-02-18T11:38:48.828574+0000","last_clean":"2024-02-18T11:38:48.828574+0000","last_became_active":"2024-02-18T11:38:47.848947+0000","last_became_peered":"2024-02-18T11:38:47.848947+0000","last_unstale":"2024-02-18T11:38:48.828574+0000","last_undegraded":"2024-02-18T11:38:48.828574+0000","last_fullsized":"2024-02-18T11:38:48.828574+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T11:48:14.101732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.089038+0000","last_change":"2024-02-18T11:38:47.840366+0000","last_active":"2024-02-18T11:38:49.089038+0000","last_peered":"2024-02-18T11:38:49.089038+0000","last_clean":"2024-02-18T11:38:49.089038+0000","last_became_active":"2024-02-18T11:38:47.840044+0000","last_became_peered":"2024-02-18T11:38:47.840044+0000","last_unstale":"2024-02-18T11:38:49.089038+0000","last_undegraded":"2024-02-18T11:38:49.089038+0000","last_fullsized":"2024-02-18T11:38:49.089038+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:31:05.184418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.853951+0000","last_change":"2024-02-18T11:38:47.844433+0000","last_active":"2024-02-18T11:38:49.853951+0000","last_peered":"2024-02-18T11:38:49.853951+0000","last_clean":"2024-02-18T11:38:49.853951+0000","last_became_active":"2024-02-18T11:38:47.844048+0000","last_became_peered":"2024-02-18T11:38:47.844048+0000","last_unstale":"2024-02-18T11:38:49.853951+0000","last_undegraded":"2024-02-18T11:38:49.853951+0000","last_fullsized":"2024-02-18T11:38:49.853951+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:08:27.698272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:48.827089+0000","last_change":"2024-02-18T11:38:47.842819+0000","last_active":"2024-02-18T11:38:48.827089+0000","last_peered":"2024-02-18T11:38:48.827089+0000","last_clean":"2024-02-18T11:38:48.827089+0000","last_became_active":"2024-02-18T11:38:47.842489+0000","last_became_peered":"2024-02-18T11:38:47.842489+0000","last_unstale":"2024-02-18T11:38:48.827089+0000","last_undegraded":"2024-02-18T11:38:48.827089+0000","last_fullsized":"2024-02-18T11:38:48.827089+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:03:21.808910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:48.826981+0000","last_change":"2024-02-18T11:38:47.838399+0000","last_active":"2024-02-18T11:38:48.826981+0000","last_peered":"2024-02-18T11:38:48.826981+0000","last_clean":"2024-02-18T11:38:48.826981+0000","last_became_active":"2024-02-18T11:38:47.838076+0000","last_became_peered":"2024-02-18T11:38:47.838076+0000","last_unstale":"2024-02-18T11:38:48.826981+0000","last_undegraded":"2024-02-18T11:38:48.826981+0000","last_fullsized":"2024-02-18T11:38:48.826981+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:26:58.017359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.855748+0000","last_change":"2024-02-18T11:38:47.838729+0000","last_active":"2024-02-18T11:38:49.855748+0000","last_peered":"2024-02-18T11:38:49.855748+0000","last_clean":"2024-02-18T11:38:49.855748+0000","last_became_active":"2024-02-18T11:38:47.838267+0000","last_became_peered":"2024-02-18T11:38:47.838267+0000","last_unstale":"2024-02-18T11:38:49.855748+0000","last_undegraded":"2024-02-18T11:38:49.855748+0000","last_fullsized":"2024-02-18T11:38:49.855748+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:08:37.797780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.855825+0000","last_change":"2024-02-18T11:38:47.847817+0000","last_active":"2024-02-18T11:38:49.855825+0000","last_peered":"2024-02-18T11:38:49.855825+0000","last_clean":"2024-02-18T11:38:49.855825+0000","last_became_active":"2024-02-18T11:38:47.847200+0000","last_became_peered":"2024-02-18T11:38:47.847200+0000","last_unstale":"2024-02-18T11:38:49.855825+0000","last_undegraded":"2024-02-18T11:38:49.855825+0000","last_fullsized":"2024-02-18T11:38:49.855825+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:22:12.425623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:48.844543+0000","last_change":"2024-02-18T11:38:47.843954+0000","last_active":"2024-02-18T11:38:48.844543+0000","last_peered":"2024-02-18T11:38:48.844543+0000","last_clean":"2024-02-18T11:38:48.844543+0000","last_became_active":"2024-02-18T11:38:47.840084+0000","last_became_peered":"2024-02-18T11:38:47.840084+0000","last_unstale":"2024-02-18T11:38:48.844543+0000","last_undegraded":"2024-02-18T11:38:48.844543+0000","last_fullsized":"2024-02-18T11:38:48.844543+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:21:13.124696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-18T11:38:47.835228+0000","last_change":"2024-02-18T11:38:47.834262+0000","last_active":"2024-02-18T11:38:47.835228+0000","last_peered":"2024-02-18T11:38:47.835228+0000","last_clean":"2024-02-18T11:38:47.835228+0000","last_became_active":"2024-02-18T11:38:47.833948+0000","last_became_peered":"2024-02-18T11:38:47.833948+0000","last_unstale":"2024-02-18T11:38:47.835228+0000","last_undegraded":"2024-02-18T11:38:47.835228+0000","last_fullsized":"2024-02-18T11:38:47.835228+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:34:14.508236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'9","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:50.863816+0000","last_change":"2024-02-18T11:38:46.533805+0000","last_active":"2024-02-18T11:38:50.863816+0000","last_peered":"2024-02-18T11:38:50.863816+0000","last_clean":"2024-02-18T11:38:50.863816+0000","last_became_active":"2024-02-18T11:38:46.533503+0000","last_became_peered":"2024-02-18T11:38:46.533503+0000","last_unstale":"2024-02-18T11:38:50.863816+0000","last_undegraded":"2024-02-18T11:38:50.863816+0000","last_fullsized":"2024-02-18T11:38:50.863816+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:05:28.087740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-18T11:38:47.841388+0000","last_change":"2024-02-18T11:38:47.833802+0000","last_active":"2024-02-18T11:38:47.841388+0000","last_peered":"2024-02-18T11:38:47.841388+0000","last_clean":"2024-02-18T11:38:47.841388+0000","last_became_active":"2024-02-18T11:38:47.833441+0000","last_became_peered":"2024-02-18T11:38:47.833441+0000","last_unstale":"2024-02-18T11:38:47.841388+0000","last_undegraded":"2024-02-18T11:38:47.841388+0000","last_fullsized":"2024-02-18T11:38:47.841388+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T12:10:09.276997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'1","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-18T11:38:47.855770+0000","last_change":"2024-02-18T11:38:46.540718+0000","last_active":"2024-02-18T11:38:47.855770+0000","last_peered":"2024-02-18T11:38:47.855770+0000","last_clean":"2024-02-18T11:38:47.855770+0000","last_became_active":"2024-02-18T11:38:46.540413+0000","last_became_peered":"2024-02-18T11:38:46.540413+0000","last_unstale":"2024-02-18T11:38:47.855770+0000","last_undegraded":"2024-02-18T11:38:47.855770+0000","last_fullsized":"2024-02-18T11:38:47.855770+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:17:53.287376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:48.827293+0000","last_change":"2024-02-18T11:38:47.846400+0000","last_active":"2024-02-18T11:38:48.827293+0000","last_peered":"2024-02-18T11:38:48.827293+0000","last_clean":"2024-02-18T11:38:48.827293+0000","last_became_active":"2024-02-18T11:38:47.845934+0000","last_became_peered":"2024-02-18T11:38:47.845934+0000","last_unstale":"2024-02-18T11:38:48.827293+0000","last_undegraded":"2024-02-18T11:38:48.827293+0000","last_fullsized":"2024-02-18T11:38:48.827293+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:20:12.093697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"87'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:50.860988+0000","last_change":"2024-02-18T11:38:46.537340+0000","last_active":"2024-02-18T11:38:50.860988+0000","last_peered":"2024-02-18T11:38:50.860988+0000","last_clean":"2024-02-18T11:38:50.860988+0000","last_became_active":"2024-02-18T11:38:46.537019+0000","last_became_peered":"2024-02-18T11:38:46.537019+0000","last_unstale":"2024-02-18T11:38:50.860988+0000","last_undegraded":"2024-02-18T11:38:50.860988+0000","last_fullsized":"2024-02-18T11:38:50.860988+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T23:00:50.052654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.856118+0000","last_change":"2024-02-18T11:38:47.852656+0000","last_active":"2024-02-18T11:38:49.856118+0000","last_peered":"2024-02-18T11:38:49.856118+0000","last_clean":"2024-02-18T11:38:49.856118+0000","last_became_active":"2024-02-18T11:38:47.852279+0000","last_became_peered":"2024-02-18T11:38:47.852279+0000","last_unstale":"2024-02-18T11:38:49.856118+0000","last_undegraded":"2024-02-18T11:38:49.856118+0000","last_fullsized":"2024-02-18T11:38:49.856118+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:51:48.911504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:48.832752+0000","last_change":"2024-02-18T11:38:46.533091+0000","last_active":"2024-02-18T11:38:48.832752+0000","last_peered":"2024-02-18T11:38:48.832752+0000","last_clean":"2024-02-18T11:38:48.832752+0000","last_became_active":"2024-02-18T11:38:46.532504+0000","last_became_peered":"2024-02-18T11:38:46.532504+0000","last_unstale":"2024-02-18T11:38:48.832752+0000","last_undegraded":"2024-02-18T11:38:48.832752+0000","last_fullsized":"2024-02-18T11:38:48.832752+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:12:00.529365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.456298+0000","last_change":"2024-02-18T11:38:47.836578+0000","last_active":"2024-02-18T11:38:49.456298+0000","last_peered":"2024-02-18T11:38:49.456298+0000","last_clean":"2024-02-18T11:38:49.456298+0000","last_became_active":"2024-02-18T11:38:47.836217+0000","last_became_peered":"2024-02-18T11:38:47.836217+0000","last_unstale":"2024-02-18T11:38:49.456298+0000","last_undegraded":"2024-02-18T11:38:49.456298+0000","last_fullsized":"2024-02-18T11:38:49.456298+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T21:53:39.011822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"87'6","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:51.545729+0000","last_change":"2024-02-18T11:38:46.532196+0000","last_active":"2024-02-18T11:38:51.545729+0000","last_peered":"2024-02-18T11:38:51.545729+0000","last_clean":"2024-02-18T11:38:51.545729+0000","last_became_active":"2024-02-18T11:38:46.531868+0000","last_became_peered":"2024-02-18T11:38:46.531868+0000","last_unstale":"2024-02-18T11:38:51.545729+0000","last_undegraded":"2024-02-18T11:38:51.545729+0000","last_fullsized":"2024-02-18T11:38:51.545729+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:52:07.396279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:48.832577+0000","last_change":"2024-02-18T11:38:47.837329+0000","last_active":"2024-02-18T11:38:48.832577+0000","last_peered":"2024-02-18T11:38:48.832577+0000","last_clean":"2024-02-18T11:38:48.832577+0000","last_became_active":"2024-02-18T11:38:47.836989+0000","last_became_peered":"2024-02-18T11:38:47.836989+0000","last_unstale":"2024-02-18T11:38:48.832577+0000","last_undegraded":"2024-02-18T11:38:48.832577+0000","last_fullsized":"2024-02-18T11:38:48.832577+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:21:16.801916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"87'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:50.862610+0000","last_change":"2024-02-18T11:38:46.533244+0000","last_active":"2024-02-18T11:38:50.862610+0000","last_peered":"2024-02-18T11:38:50.862610+0000","last_clean":"2024-02-18T11:38:50.862610+0000","last_became_active":"2024-02-18T11:38:46.532326+0000","last_became_peered":"2024-02-18T11:38:46.532326+0000","last_unstale":"2024-02-18T11:38:50.862610+0000","last_undegraded":"2024-02-18T11:38:50.862610+0000","last_fullsized":"2024-02-18T11:38:50.862610+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:32:32.979056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.456426+0000","last_change":"2024-02-18T11:38:47.824560+0000","last_active":"2024-02-18T11:38:49.456426+0000","last_peered":"2024-02-18T11:38:49.456426+0000","last_clean":"2024-02-18T11:38:49.456426+0000","last_became_active":"2024-02-18T11:38:47.824095+0000","last_became_peered":"2024-02-18T11:38:47.824095+0000","last_unstale":"2024-02-18T11:38:49.456426+0000","last_undegraded":"2024-02-18T11:38:49.456426+0000","last_fullsized":"2024-02-18T11:38:49.456426+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:27:09.870585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"87'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:50.862291+0000","last_change":"2024-02-18T11:38:46.526024+0000","last_active":"2024-02-18T11:38:50.862291+0000","last_peered":"2024-02-18T11:38:50.862291+0000","last_clean":"2024-02-18T11:38:50.862291+0000","last_became_active":"2024-02-18T11:38:46.525735+0000","last_became_peered":"2024-02-18T11:38:46.525735+0000","last_unstale":"2024-02-18T11:38:50.862291+0000","last_undegraded":"2024-02-18T11:38:50.862291+0000","last_fullsized":"2024-02-18T11:38:50.862291+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:25:10.659210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-18T11:38:47.843658+0000","last_change":"2024-02-18T11:38:47.841743+0000","last_active":"2024-02-18T11:38:47.843658+0000","last_peered":"2024-02-18T11:38:47.843658+0000","last_clean":"2024-02-18T11:38:47.843658+0000","last_became_active":"2024-02-18T11:38:47.840848+0000","last_became_peered":"2024-02-18T11:38:47.840848+0000","last_unstale":"2024-02-18T11:38:47.843658+0000","last_undegraded":"2024-02-18T11:38:47.843658+0000","last_fullsized":"2024-02-18T11:38:47.843658+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T11:51:35.764831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"87'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.456175+0000","last_change":"2024-02-18T11:38:46.531355+0000","last_active":"2024-02-18T11:38:49.456175+0000","last_peered":"2024-02-18T11:38:49.456175+0000","last_clean":"2024-02-18T11:38:49.456175+0000","last_became_active":"2024-02-18T11:38:46.530601+0000","last_became_peered":"2024-02-18T11:38:46.530601+0000","last_unstale":"2024-02-18T11:38:49.456175+0000","last_undegraded":"2024-02-18T11:38:49.456175+0000","last_fullsized":"2024-02-18T11:38:49.456175+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:55:29.670274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:48.844447+0000","last_change":"2024-02-18T11:38:47.844506+0000","last_active":"2024-02-18T11:38:48.844447+0000","last_peered":"2024-02-18T11:38:48.844447+0000","last_clean":"2024-02-18T11:38:48.844447+0000","last_became_active":"2024-02-18T11:38:47.844158+0000","last_became_peered":"2024-02-18T11:38:47.844158+0000","last_unstale":"2024-02-18T11:38:48.844447+0000","last_undegraded":"2024-02-18T11:38:48.844447+0000","last_fullsized":"2024-02-18T11:38:48.844447+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T23:13:17.360077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"87'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.858291+0000","last_change":"2024-02-18T11:38:46.536313+0000","last_active":"2024-02-18T11:38:49.858291+0000","last_peered":"2024-02-18T11:38:49.858291+0000","last_clean":"2024-02-18T11:38:49.858291+0000","last_became_active":"2024-02-18T11:38:46.535910+0000","last_became_peered":"2024-02-18T11:38:46.535910+0000","last_unstale":"2024-02-18T11:38:49.858291+0000","last_undegraded":"2024-02-18T11:38:49.858291+0000","last_fullsized":"2024-02-18T11:38:49.858291+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:56:16.606415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.856993+0000","last_change":"2024-02-18T11:38:47.836131+0000","last_active":"2024-02-18T11:38:49.856993+0000","last_peered":"2024-02-18T11:38:49.856993+0000","last_clean":"2024-02-18T11:38:49.856993+0000","last_became_active":"2024-02-18T11:38:47.835702+0000","last_became_peered":"2024-02-18T11:38:47.835702+0000","last_unstale":"2024-02-18T11:38:49.856993+0000","last_undegraded":"2024-02-18T11:38:49.856993+0000","last_fullsized":"2024-02-18T11:38:49.856993+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:58:11.622972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"87'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.856996+0000","last_change":"2024-02-18T11:38:46.530031+0000","last_active":"2024-02-18T11:38:49.856996+0000","last_peered":"2024-02-18T11:38:49.856996+0000","last_clean":"2024-02-18T11:38:49.856996+0000","last_became_active":"2024-02-18T11:38:46.529674+0000","last_became_peered":"2024-02-18T11:38:46.529674+0000","last_unstale":"2024-02-18T11:38:49.856996+0000","last_undegraded":"2024-02-18T11:38:49.856996+0000","last_fullsized":"2024-02-18T11:38:49.856996+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T21:13:53.955970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:48.827261+0000","last_change":"2024-02-18T11:38:47.852340+0000","last_active":"2024-02-18T11:38:48.827261+0000","last_peered":"2024-02-18T11:38:48.827261+0000","last_clean":"2024-02-18T11:38:48.827261+0000","last_became_active":"2024-02-18T11:38:47.847978+0000","last_became_peered":"2024-02-18T11:38:47.847978+0000","last_unstale":"2024-02-18T11:38:48.827261+0000","last_undegraded":"2024-02-18T11:38:48.827261+0000","last_fullsized":"2024-02-18T11:38:48.827261+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:47:09.667619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'4","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.857723+0000","last_change":"2024-02-18T11:38:46.533511+0000","last_active":"2024-02-18T11:38:49.857723+0000","last_peered":"2024-02-18T11:38:49.857723+0000","last_clean":"2024-02-18T11:38:49.857723+0000","last_became_active":"2024-02-18T11:38:46.533162+0000","last_became_peered":"2024-02-18T11:38:46.533162+0000","last_unstale":"2024-02-18T11:38:49.857723+0000","last_undegraded":"2024-02-18T11:38:49.857723+0000","last_fullsized":"2024-02-18T11:38:49.857723+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:00:49.307489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.856288+0000","last_change":"2024-02-18T11:38:47.843835+0000","last_active":"2024-02-18T11:38:49.856288+0000","last_peered":"2024-02-18T11:38:49.856288+0000","last_clean":"2024-02-18T11:38:49.856288+0000","last_became_active":"2024-02-18T11:38:47.843517+0000","last_became_peered":"2024-02-18T11:38:47.843517+0000","last_unstale":"2024-02-18T11:38:49.856288+0000","last_undegraded":"2024-02-18T11:38:49.856288+0000","last_fullsized":"2024-02-18T11:38:49.856288+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:54:02.259854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'8","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.858239+0000","last_change":"2024-02-18T11:38:46.536900+0000","last_active":"2024-02-18T11:38:49.858239+0000","last_peered":"2024-02-18T11:38:49.858239+0000","last_clean":"2024-02-18T11:38:49.858239+0000","last_became_active":"2024-02-18T11:38:46.536563+0000","last_became_peered":"2024-02-18T11:38:46.536563+0000","last_unstale":"2024-02-18T11:38:49.858239+0000","last_undegraded":"2024-02-18T11:38:49.858239+0000","last_fullsized":"2024-02-18T11:38:49.858239+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T12:56:57.959482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:48.832845+0000","last_change":"2024-02-18T11:38:47.842452+0000","last_active":"2024-02-18T11:38:48.832845+0000","last_peered":"2024-02-18T11:38:48.832845+0000","last_clean":"2024-02-18T11:38:48.832845+0000","last_became_active":"2024-02-18T11:38:47.842105+0000","last_became_peered":"2024-02-18T11:38:47.842105+0000","last_unstale":"2024-02-18T11:38:48.832845+0000","last_undegraded":"2024-02-18T11:38:48.832845+0000","last_fullsized":"2024-02-18T11:38:48.832845+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:10:05.131880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'2","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-18T11:38:47.856142+0000","last_change":"2024-02-18T11:38:46.534220+0000","last_active":"2024-02-18T11:38:47.856142+0000","last_peered":"2024-02-18T11:38:47.856142+0000","last_clean":"2024-02-18T11:38:47.856142+0000","last_became_active":"2024-02-18T11:38:46.532929+0000","last_became_peered":"2024-02-18T11:38:46.532929+0000","last_unstale":"2024-02-18T11:38:47.856142+0000","last_undegraded":"2024-02-18T11:38:47.856142+0000","last_fullsized":"2024-02-18T11:38:47.856142+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:41:42.674196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.853814+0000","last_change":"2024-02-18T11:38:47.848434+0000","last_active":"2024-02-18T11:38:49.853814+0000","last_peered":"2024-02-18T11:38:49.853814+0000","last_clean":"2024-02-18T11:38:49.853814+0000","last_became_active":"2024-02-18T11:38:47.848092+0000","last_became_peered":"2024-02-18T11:38:47.848092+0000","last_unstale":"2024-02-18T11:38:49.853814+0000","last_undegraded":"2024-02-18T11:38:49.853814+0000","last_fullsized":"2024-02-18T11:38:49.853814+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:23:35.292982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"87'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.856206+0000","last_change":"2024-02-18T11:38:46.531579+0000","last_active":"2024-02-18T11:38:49.856206+0000","last_peered":"2024-02-18T11:38:49.856206+0000","last_clean":"2024-02-18T11:38:49.856206+0000","last_became_active":"2024-02-18T11:38:46.531236+0000","last_became_peered":"2024-02-18T11:38:46.531236+0000","last_unstale":"2024-02-18T11:38:49.856206+0000","last_undegraded":"2024-02-18T11:38:49.856206+0000","last_fullsized":"2024-02-18T11:38:49.856206+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:54:48.886442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:48.828336+0000","last_change":"2024-02-18T11:38:47.840418+0000","last_active":"2024-02-18T11:38:48.828336+0000","last_peered":"2024-02-18T11:38:48.828336+0000","last_clean":"2024-02-18T11:38:48.828336+0000","last_became_active":"2024-02-18T11:38:47.835202+0000","last_became_peered":"2024-02-18T11:38:47.835202+0000","last_unstale":"2024-02-18T11:38:48.828336+0000","last_undegraded":"2024-02-18T11:38:48.828336+0000","last_fullsized":"2024-02-18T11:38:48.828336+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:40:26.874931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'3","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:51.543777+0000","last_change":"2024-02-18T11:38:46.528086+0000","last_active":"2024-02-18T11:38:51.543777+0000","last_peered":"2024-02-18T11:38:51.543777+0000","last_clean":"2024-02-18T11:38:51.543777+0000","last_became_active":"2024-02-18T11:38:46.527753+0000","last_became_peered":"2024-02-18T11:38:46.527753+0000","last_unstale":"2024-02-18T11:38:51.543777+0000","last_undegraded":"2024-02-18T11:38:51.543777+0000","last_fullsized":"2024-02-18T11:38:51.543777+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:48:09.524280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:48.827834+0000","last_change":"2024-02-18T11:38:47.849139+0000","last_active":"2024-02-18T11:38:48.827834+0000","last_peered":"2024-02-18T11:38:48.827834+0000","last_clean":"2024-02-18T11:38:48.827834+0000","last_became_active":"2024-02-18T11:38:47.848752+0000","last_became_peered":"2024-02-18T11:38:47.848752+0000","last_unstale":"2024-02-18T11:38:48.827834+0000","last_undegraded":"2024-02-18T11:38:48.827834+0000","last_fullsized":"2024-02-18T11:38:48.827834+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:00:56.046670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"87'15","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:50.862743+0000","last_change":"2024-02-18T11:38:46.540471+0000","last_active":"2024-02-18T11:38:50.862743+0000","last_peered":"2024-02-18T11:38:50.862743+0000","last_clean":"2024-02-18T11:38:50.862743+0000","last_became_active":"2024-02-18T11:38:46.540169+0000","last_became_peered":"2024-02-18T11:38:46.540169+0000","last_unstale":"2024-02-18T11:38:50.862743+0000","last_undegraded":"2024-02-18T11:38:50.862743+0000","last_fullsized":"2024-02-18T11:38:50.862743+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:29:46.001510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.456087+0000","last_change":"2024-02-18T11:38:47.838077+0000","last_active":"2024-02-18T11:38:49.456087+0000","last_peered":"2024-02-18T11:38:49.456087+0000","last_clean":"2024-02-18T11:38:49.456087+0000","last_became_active":"2024-02-18T11:38:47.837137+0000","last_became_peered":"2024-02-18T11:38:47.837137+0000","last_unstale":"2024-02-18T11:38:49.456087+0000","last_undegraded":"2024-02-18T11:38:49.456087+0000","last_fullsized":"2024-02-18T11:38:49.456087+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:06:15.443851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:48.832894+0000","last_change":"2024-02-18T11:38:46.532828+0000","last_active":"2024-02-18T11:38:48.832894+0000","last_peered":"2024-02-18T11:38:48.832894+0000","last_clean":"2024-02-18T11:38:48.832894+0000","last_became_active":"2024-02-18T11:38:46.532515+0000","last_became_peered":"2024-02-18T11:38:46.532515+0000","last_unstale":"2024-02-18T11:38:48.832894+0000","last_undegraded":"2024-02-18T11:38:48.832894+0000","last_fullsized":"2024-02-18T11:38:48.832894+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:20:23.333916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.854948+0000","last_change":"2024-02-18T11:38:47.835481+0000","last_active":"2024-02-18T11:38:49.854948+0000","last_peered":"2024-02-18T11:38:49.854948+0000","last_clean":"2024-02-18T11:38:49.854948+0000","last_became_active":"2024-02-18T11:38:47.835071+0000","last_became_peered":"2024-02-18T11:38:47.835071+0000","last_unstale":"2024-02-18T11:38:49.854948+0000","last_undegraded":"2024-02-18T11:38:49.854948+0000","last_fullsized":"2024-02-18T11:38:49.854948+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T12:10:42.098646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:48.844409+0000","last_change":"2024-02-18T11:38:46.530294+0000","last_active":"2024-02-18T11:38:48.844409+0000","last_peered":"2024-02-18T11:38:48.844409+0000","last_clean":"2024-02-18T11:38:48.844409+0000","last_became_active":"2024-02-18T11:38:46.529913+0000","last_became_peered":"2024-02-18T11:38:46.529913+0000","last_unstale":"2024-02-18T11:38:48.844409+0000","last_undegraded":"2024-02-18T11:38:48.844409+0000","last_fullsized":"2024-02-18T11:38:48.844409+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:17:54.549659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.854095+0000","last_change":"2024-02-18T11:38:47.842020+0000","last_active":"2024-02-18T11:38:49.854095+0000","last_peered":"2024-02-18T11:38:49.854095+0000","last_clean":"2024-02-18T11:38:49.854095+0000","last_became_active":"2024-02-18T11:38:47.841656+0000","last_became_peered":"2024-02-18T11:38:47.841656+0000","last_unstale":"2024-02-18T11:38:49.854095+0000","last_undegraded":"2024-02-18T11:38:49.854095+0000","last_fullsized":"2024-02-18T11:38:49.854095+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:12:50.508340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"87'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:48.828019+0000","last_change":"2024-02-18T11:38:46.531867+0000","last_active":"2024-02-18T11:38:48.828019+0000","last_peered":"2024-02-18T11:38:48.828019+0000","last_clean":"2024-02-18T11:38:48.828019+0000","last_became_active":"2024-02-18T11:38:46.531051+0000","last_became_peered":"2024-02-18T11:38:46.531051+0000","last_unstale":"2024-02-18T11:38:48.828019+0000","last_undegraded":"2024-02-18T11:38:48.828019+0000","last_fullsized":"2024-02-18T11:38:48.828019+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:15:26.808764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.854486+0000","last_change":"2024-02-18T11:38:47.835160+0000","last_active":"2024-02-18T11:38:49.854486+0000","last_peered":"2024-02-18T11:38:49.854486+0000","last_clean":"2024-02-18T11:38:49.854486+0000","last_became_active":"2024-02-18T11:38:47.834813+0000","last_became_peered":"2024-02-18T11:38:47.834813+0000","last_unstale":"2024-02-18T11:38:49.854486+0000","last_undegraded":"2024-02-18T11:38:49.854486+0000","last_fullsized":"2024-02-18T11:38:49.854486+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:30:59.674796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.456423+0000","last_change":"2024-02-18T11:38:47.838197+0000","last_active":"2024-02-18T11:38:49.456423+0000","last_peered":"2024-02-18T11:38:49.456423+0000","last_clean":"2024-02-18T11:38:49.456423+0000","last_became_active":"2024-02-18T11:38:47.837393+0000","last_became_peered":"2024-02-18T11:38:47.837393+0000","last_unstale":"2024-02-18T11:38:49.456423+0000","last_undegraded":"2024-02-18T11:38:49.456423+0000","last_fullsized":"2024-02-18T11:38:49.456423+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:13:19.095082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'4","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:50.863756+0000","last_change":"2024-02-18T11:38:46.538682+0000","last_active":"2024-02-18T11:38:50.863756+0000","last_peered":"2024-02-18T11:38:50.863756+0000","last_clean":"2024-02-18T11:38:50.863756+0000","last_became_active":"2024-02-18T11:38:46.538356+0000","last_became_peered":"2024-02-18T11:38:46.538356+0000","last_unstale":"2024-02-18T11:38:50.863756+0000","last_undegraded":"2024-02-18T11:38:50.863756+0000","last_fullsized":"2024-02-18T11:38:50.863756+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:41:02.449905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-18T11:38:47.846833+0000","last_change":"2024-02-18T11:38:47.844448+0000","last_active":"2024-02-18T11:38:47.846833+0000","last_peered":"2024-02-18T11:38:47.846833+0000","last_clean":"2024-02-18T11:38:47.846833+0000","last_became_active":"2024-02-18T11:38:47.844048+0000","last_became_peered":"2024-02-18T11:38:47.844048+0000","last_unstale":"2024-02-18T11:38:47.846833+0000","last_undegraded":"2024-02-18T11:38:47.846833+0000","last_fullsized":"2024-02-18T11:38:47.846833+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:33:41.159473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'3","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:51.541339+0000","last_change":"2024-02-18T11:38:46.531504+0000","last_active":"2024-02-18T11:38:51.541339+0000","last_peered":"2024-02-18T11:38:51.541339+0000","last_clean":"2024-02-18T11:38:51.541339+0000","last_became_active":"2024-02-18T11:38:46.531173+0000","last_became_peered":"2024-02-18T11:38:46.531173+0000","last_unstale":"2024-02-18T11:38:51.541339+0000","last_undegraded":"2024-02-18T11:38:51.541339+0000","last_fullsized":"2024-02-18T11:38:51.541339+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:32:40.632054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":341,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:48.827763+0000","last_change":"2024-02-18T11:37:34.534077+0000","last_active":"2024-02-18T11:38:48.827763+0000","last_peered":"2024-02-18T11:38:48.827763+0000","last_clean":"2024-02-18T11:38:48.827763+0000","last_became_active":"2024-02-18T11:37:34.533557+0000","last_became_peered":"2024-02-18T11:37:34.533557+0000","last_unstale":"2024-02-18T11:38:48.827763+0000","last_undegraded":"2024-02-18T11:38:48.827763+0000","last_fullsized":"2024-02-18T11:38:48.827763+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:34:15.604912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:34:15.604912+0000","last_clean_scrub_stamp":"2024-02-18T11:34:15.604912+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T23:01:35.216671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.853984+0000","last_change":"2024-02-18T11:38:47.838123+0000","last_active":"2024-02-18T11:38:49.853984+0000","last_peered":"2024-02-18T11:38:49.853984+0000","last_clean":"2024-02-18T11:38:49.853984+0000","last_became_active":"2024-02-18T11:38:47.837713+0000","last_became_peered":"2024-02-18T11:38:47.837713+0000","last_unstale":"2024-02-18T11:38:49.853984+0000","last_undegraded":"2024-02-18T11:38:49.853984+0000","last_fullsized":"2024-02-18T11:38:49.853984+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T21:47:44.609399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:50.862743+0000","last_change":"2024-02-18T11:38:46.529508+0000","last_active":"2024-02-18T11:38:50.862743+0000","last_peered":"2024-02-18T11:38:50.862743+0000","last_clean":"2024-02-18T11:38:50.862743+0000","last_became_active":"2024-02-18T11:38:46.529165+0000","last_became_peered":"2024-02-18T11:38:46.529165+0000","last_unstale":"2024-02-18T11:38:50.862743+0000","last_undegraded":"2024-02-18T11:38:50.862743+0000","last_fullsized":"2024-02-18T11:38:50.862743+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:10:12.774664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"87'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:48.832804+0000","last_change":"2024-02-18T11:38:46.542236+0000","last_active":"2024-02-18T11:38:48.832804+0000","last_peered":"2024-02-18T11:38:48.832804+0000","last_clean":"2024-02-18T11:38:48.832804+0000","last_became_active":"2024-02-18T11:38:46.541901+0000","last_became_peered":"2024-02-18T11:38:46.541901+0000","last_unstale":"2024-02-18T11:38:48.832804+0000","last_undegraded":"2024-02-18T11:38:48.832804+0000","last_fullsized":"2024-02-18T11:38:48.832804+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:40:28.699330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.854718+0000","last_change":"2024-02-18T11:38:47.841210+0000","last_active":"2024-02-18T11:38:49.854718+0000","last_peered":"2024-02-18T11:38:49.854718+0000","last_clean":"2024-02-18T11:38:49.854718+0000","last_became_active":"2024-02-18T11:38:47.838873+0000","last_became_peered":"2024-02-18T11:38:47.838873+0000","last_unstale":"2024-02-18T11:38:49.854718+0000","last_undegraded":"2024-02-18T11:38:49.854718+0000","last_fullsized":"2024-02-18T11:38:49.854718+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T21:22:11.304237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:50.862850+0000","last_change":"2024-02-18T11:38:46.533820+0000","last_active":"2024-02-18T11:38:50.862850+0000","last_peered":"2024-02-18T11:38:50.862850+0000","last_clean":"2024-02-18T11:38:50.862850+0000","last_became_active":"2024-02-18T11:38:46.533342+0000","last_became_peered":"2024-02-18T11:38:46.533342+0000","last_unstale":"2024-02-18T11:38:50.862850+0000","last_undegraded":"2024-02-18T11:38:50.862850+0000","last_fullsized":"2024-02-18T11:38:50.862850+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:27:34.814511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-18T11:38:47.841517+0000","last_change":"2024-02-18T11:38:47.837034+0000","last_active":"2024-02-18T11:38:47.841517+0000","last_peered":"2024-02-18T11:38:47.841517+0000","last_clean":"2024-02-18T11:38:47.841517+0000","last_became_active":"2024-02-18T11:38:47.836340+0000","last_became_peered":"2024-02-18T11:38:47.836340+0000","last_unstale":"2024-02-18T11:38:47.841517+0000","last_undegraded":"2024-02-18T11:38:47.841517+0000","last_fullsized":"2024-02-18T11:38:47.841517+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T21:28:47.572516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.854297+0000","last_change":"2024-02-18T11:38:46.529389+0000","last_active":"2024-02-18T11:38:49.854297+0000","last_peered":"2024-02-18T11:38:49.854297+0000","last_clean":"2024-02-18T11:38:49.854297+0000","last_became_active":"2024-02-18T11:38:46.529076+0000","last_became_peered":"2024-02-18T11:38:46.529076+0000","last_unstale":"2024-02-18T11:38:49.854297+0000","last_undegraded":"2024-02-18T11:38:49.854297+0000","last_fullsized":"2024-02-18T11:38:49.854297+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T21:01:05.553901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:48.827378+0000","last_change":"2024-02-18T11:38:47.847677+0000","last_active":"2024-02-18T11:38:48.827378+0000","last_peered":"2024-02-18T11:38:48.827378+0000","last_clean":"2024-02-18T11:38:48.827378+0000","last_became_active":"2024-02-18T11:38:47.847051+0000","last_became_peered":"2024-02-18T11:38:47.847051+0000","last_unstale":"2024-02-18T11:38:48.827378+0000","last_undegraded":"2024-02-18T11:38:48.827378+0000","last_fullsized":"2024-02-18T11:38:48.827378+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:25:53.398173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.858228+0000","last_change":"2024-02-18T11:38:46.525030+0000","last_active":"2024-02-18T11:38:49.858228+0000","last_peered":"2024-02-18T11:38:49.858228+0000","last_clean":"2024-02-18T11:38:49.858228+0000","last_became_active":"2024-02-18T11:38:46.524405+0000","last_became_peered":"2024-02-18T11:38:46.524405+0000","last_unstale":"2024-02-18T11:38:49.858228+0000","last_undegraded":"2024-02-18T11:38:49.858228+0000","last_fullsized":"2024-02-18T11:38:49.858228+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T12:33:41.751625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:48.832564+0000","last_change":"2024-02-18T11:38:47.836583+0000","last_active":"2024-02-18T11:38:48.832564+0000","last_peered":"2024-02-18T11:38:48.832564+0000","last_clean":"2024-02-18T11:38:48.832564+0000","last_became_active":"2024-02-18T11:38:47.836138+0000","last_became_peered":"2024-02-18T11:38:47.836138+0000","last_unstale":"2024-02-18T11:38:48.832564+0000","last_undegraded":"2024-02-18T11:38:48.832564+0000","last_fullsized":"2024-02-18T11:38:48.832564+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:34:04.002137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-18T11:38:47.821129+0000","last_change":"2024-02-18T11:38:46.538553+0000","last_active":"2024-02-18T11:38:47.821129+0000","last_peered":"2024-02-18T11:38:47.821129+0000","last_clean":"2024-02-18T11:38:47.821129+0000","last_became_active":"2024-02-18T11:38:46.538216+0000","last_became_peered":"2024-02-18T11:38:46.538216+0000","last_unstale":"2024-02-18T11:38:47.821129+0000","last_undegraded":"2024-02-18T11:38:47.821129+0000","last_fullsized":"2024-02-18T11:38:47.821129+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:31:32.713327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.854428+0000","last_change":"2024-02-18T11:38:47.840452+0000","last_active":"2024-02-18T11:38:49.854428+0000","last_peered":"2024-02-18T11:38:49.854428+0000","last_clean":"2024-02-18T11:38:49.854428+0000","last_became_active":"2024-02-18T11:38:47.840071+0000","last_became_peered":"2024-02-18T11:38:47.840071+0000","last_unstale":"2024-02-18T11:38:49.854428+0000","last_undegraded":"2024-02-18T11:38:49.854428+0000","last_fullsized":"2024-02-18T11:38:49.854428+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:07:22.746854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-18T11:38:47.831525+0000","last_change":"2024-02-18T11:38:46.535460+0000","last_active":"2024-02-18T11:38:47.831525+0000","last_peered":"2024-02-18T11:38:47.831525+0000","last_clean":"2024-02-18T11:38:47.831525+0000","last_became_active":"2024-02-18T11:38:46.534626+0000","last_became_peered":"2024-02-18T11:38:46.534626+0000","last_unstale":"2024-02-18T11:38:47.831525+0000","last_undegraded":"2024-02-18T11:38:47.831525+0000","last_fullsized":"2024-02-18T11:38:47.831525+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:56:18.434798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-18T11:38:47.841881+0000","last_change":"2024-02-18T11:38:47.838616+0000","last_active":"2024-02-18T11:38:47.841881+0000","last_peered":"2024-02-18T11:38:47.841881+0000","last_clean":"2024-02-18T11:38:47.841881+0000","last_became_active":"2024-02-18T11:38:47.838301+0000","last_became_peered":"2024-02-18T11:38:47.838301+0000","last_unstale":"2024-02-18T11:38:47.841881+0000","last_undegraded":"2024-02-18T11:38:47.841881+0000","last_fullsized":"2024-02-18T11:38:47.841881+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:11:25.865726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.856138+0000","last_change":"2024-02-18T11:38:46.536544+0000","last_active":"2024-02-18T11:38:49.856138+0000","last_peered":"2024-02-18T11:38:49.856138+0000","last_clean":"2024-02-18T11:38:49.856138+0000","last_became_active":"2024-02-18T11:38:46.536215+0000","last_became_peered":"2024-02-18T11:38:46.536215+0000","last_unstale":"2024-02-18T11:38:49.856138+0000","last_undegraded":"2024-02-18T11:38:49.856138+0000","last_fullsized":"2024-02-18T11:38:49.856138+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:12:44.496950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:48.828176+0000","last_change":"2024-02-18T11:38:47.847872+0000","last_active":"2024-02-18T11:38:48.828176+0000","last_peered":"2024-02-18T11:38:48.828176+0000","last_clean":"2024-02-18T11:38:48.828176+0000","last_became_active":"2024-02-18T11:38:47.847537+0000","last_became_peered":"2024-02-18T11:38:47.847537+0000","last_unstale":"2024-02-18T11:38:48.828176+0000","last_undegraded":"2024-02-18T11:38:48.828176+0000","last_fullsized":"2024-02-18T11:38:48.828176+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T12:30:46.754937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-18T11:38:47.828657+0000","last_change":"2024-02-18T11:38:46.534677+0000","last_active":"2024-02-18T11:38:47.828657+0000","last_peered":"2024-02-18T11:38:47.828657+0000","last_clean":"2024-02-18T11:38:47.828657+0000","last_became_active":"2024-02-18T11:38:46.527789+0000","last_became_peered":"2024-02-18T11:38:46.527789+0000","last_unstale":"2024-02-18T11:38:47.828657+0000","last_undegraded":"2024-02-18T11:38:47.828657+0000","last_fullsized":"2024-02-18T11:38:47.828657+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:13:15.365557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:48.828621+0000","last_change":"2024-02-18T11:38:47.849874+0000","last_active":"2024-02-18T11:38:48.828621+0000","last_peered":"2024-02-18T11:38:48.828621+0000","last_clean":"2024-02-18T11:38:48.828621+0000","last_became_active":"2024-02-18T11:38:47.849318+0000","last_became_peered":"2024-02-18T11:38:47.849318+0000","last_unstale":"2024-02-18T11:38:48.828621+0000","last_undegraded":"2024-02-18T11:38:48.828621+0000","last_fullsized":"2024-02-18T11:38:48.828621+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:22:31.974649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'1","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.456337+0000","last_change":"2024-02-18T11:38:46.531691+0000","last_active":"2024-02-18T11:38:49.456337+0000","last_peered":"2024-02-18T11:38:49.456337+0000","last_clean":"2024-02-18T11:38:49.456337+0000","last_became_active":"2024-02-18T11:38:46.530987+0000","last_became_peered":"2024-02-18T11:38:46.530987+0000","last_unstale":"2024-02-18T11:38:49.456337+0000","last_undegraded":"2024-02-18T11:38:49.456337+0000","last_fullsized":"2024-02-18T11:38:49.456337+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:15:46.228599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:48.828161+0000","last_change":"2024-02-18T11:38:47.850273+0000","last_active":"2024-02-18T11:38:48.828161+0000","last_peered":"2024-02-18T11:38:48.828161+0000","last_clean":"2024-02-18T11:38:48.828161+0000","last_became_active":"2024-02-18T11:38:47.849110+0000","last_became_peered":"2024-02-18T11:38:47.849110+0000","last_unstale":"2024-02-18T11:38:48.828161+0000","last_undegraded":"2024-02-18T11:38:48.828161+0000","last_fullsized":"2024-02-18T11:38:48.828161+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T23:18:55.096768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.854461+0000","last_change":"2024-02-18T11:38:46.537641+0000","last_active":"2024-02-18T11:38:49.854461+0000","last_peered":"2024-02-18T11:38:49.854461+0000","last_clean":"2024-02-18T11:38:49.854461+0000","last_became_active":"2024-02-18T11:38:46.537311+0000","last_became_peered":"2024-02-18T11:38:46.537311+0000","last_unstale":"2024-02-18T11:38:49.854461+0000","last_undegraded":"2024-02-18T11:38:49.854461+0000","last_fullsized":"2024-02-18T11:38:49.854461+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:13:25.720250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:48.828659+0000","last_change":"2024-02-18T11:38:47.833177+0000","last_active":"2024-02-18T11:38:48.828659+0000","last_peered":"2024-02-18T11:38:48.828659+0000","last_clean":"2024-02-18T11:38:48.828659+0000","last_became_active":"2024-02-18T11:38:47.832839+0000","last_became_peered":"2024-02-18T11:38:47.832839+0000","last_unstale":"2024-02-18T11:38:48.828659+0000","last_undegraded":"2024-02-18T11:38:48.828659+0000","last_fullsized":"2024-02-18T11:38:48.828659+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:12:28.448505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.854545+0000","last_change":"2024-02-18T11:38:47.841062+0000","last_active":"2024-02-18T11:38:49.854545+0000","last_peered":"2024-02-18T11:38:49.854545+0000","last_clean":"2024-02-18T11:38:49.854545+0000","last_became_active":"2024-02-18T11:38:47.840730+0000","last_became_peered":"2024-02-18T11:38:47.840730+0000","last_unstale":"2024-02-18T11:38:49.854545+0000","last_undegraded":"2024-02-18T11:38:49.854545+0000","last_fullsized":"2024-02-18T11:38:49.854545+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:18:13.417676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-18T11:38:47.840327+0000","last_change":"2024-02-18T11:38:47.839488+0000","last_active":"2024-02-18T11:38:47.840327+0000","last_peered":"2024-02-18T11:38:47.840327+0000","last_clean":"2024-02-18T11:38:47.840327+0000","last_became_active":"2024-02-18T11:38:47.839134+0000","last_became_peered":"2024-02-18T11:38:47.839134+0000","last_unstale":"2024-02-18T11:38:47.840327+0000","last_undegraded":"2024-02-18T11:38:47.840327+0000","last_fullsized":"2024-02-18T11:38:47.840327+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T12:08:07.783369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-18T11:38:47.845944+0000","last_change":"2024-02-18T11:38:47.845397+0000","last_active":"2024-02-18T11:38:47.845944+0000","last_peered":"2024-02-18T11:38:47.845944+0000","last_clean":"2024-02-18T11:38:47.845944+0000","last_became_active":"2024-02-18T11:38:47.845027+0000","last_became_peered":"2024-02-18T11:38:47.845027+0000","last_unstale":"2024-02-18T11:38:47.845944+0000","last_undegraded":"2024-02-18T11:38:47.845944+0000","last_fullsized":"2024-02-18T11:38:47.845944+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T23:08:32.725571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:48.827311+0000","last_change":"2024-02-18T11:38:47.839238+0000","last_active":"2024-02-18T11:38:48.827311+0000","last_peered":"2024-02-18T11:38:48.827311+0000","last_clean":"2024-02-18T11:38:48.827311+0000","last_became_active":"2024-02-18T11:38:47.838928+0000","last_became_peered":"2024-02-18T11:38:47.838928+0000","last_unstale":"2024-02-18T11:38:48.827311+0000","last_undegraded":"2024-02-18T11:38:48.827311+0000","last_fullsized":"2024-02-18T11:38:48.827311+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:32:10.867251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.853471+0000","last_change":"2024-02-18T11:38:47.841524+0000","last_active":"2024-02-18T11:38:49.853471+0000","last_peered":"2024-02-18T11:38:49.853471+0000","last_clean":"2024-02-18T11:38:49.853471+0000","last_became_active":"2024-02-18T11:38:47.840807+0000","last_became_peered":"2024-02-18T11:38:47.840807+0000","last_unstale":"2024-02-18T11:38:49.853471+0000","last_undegraded":"2024-02-18T11:38:49.853471+0000","last_fullsized":"2024-02-18T11:38:49.853471+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T12:40:11.488587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.854812+0000","last_change":"2024-02-18T11:38:47.839821+0000","last_active":"2024-02-18T11:38:49.854812+0000","last_peered":"2024-02-18T11:38:49.854812+0000","last_clean":"2024-02-18T11:38:49.854812+0000","last_became_active":"2024-02-18T11:38:47.839431+0000","last_became_peered":"2024-02-18T11:38:47.839431+0000","last_unstale":"2024-02-18T11:38:49.854812+0000","last_undegraded":"2024-02-18T11:38:49.854812+0000","last_fullsized":"2024-02-18T11:38:49.854812+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:43:07.002223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-18T11:38:47.845005+0000","last_change":"2024-02-18T11:38:47.843937+0000","last_active":"2024-02-18T11:38:47.845005+0000","last_peered":"2024-02-18T11:38:47.845005+0000","last_clean":"2024-02-18T11:38:47.845005+0000","last_became_active":"2024-02-18T11:38:47.843603+0000","last_became_peered":"2024-02-18T11:38:47.843603+0000","last_unstale":"2024-02-18T11:38:47.845005+0000","last_undegraded":"2024-02-18T11:38:47.845005+0000","last_fullsized":"2024-02-18T11:38:47.845005+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:35:47.247763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-18T11:38:47.841492+0000","last_change":"2024-02-18T11:38:47.836702+0000","last_active":"2024-02-18T11:38:47.841492+0000","last_peered":"2024-02-18T11:38:47.841492+0000","last_clean":"2024-02-18T11:38:47.841492+0000","last_became_active":"2024-02-18T11:38:47.834843+0000","last_became_peered":"2024-02-18T11:38:47.834843+0000","last_unstale":"2024-02-18T11:38:47.841492+0000","last_undegraded":"2024-02-18T11:38:47.841492+0000","last_fullsized":"2024-02-18T11:38:47.841492+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:14:02.288109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.456116+0000","last_change":"2024-02-18T11:38:47.837429+0000","last_active":"2024-02-18T11:38:49.456116+0000","last_peered":"2024-02-18T11:38:49.456116+0000","last_clean":"2024-02-18T11:38:49.456116+0000","last_became_active":"2024-02-18T11:38:47.832983+0000","last_became_peered":"2024-02-18T11:38:47.832983+0000","last_unstale":"2024-02-18T11:38:49.456116+0000","last_undegraded":"2024-02-18T11:38:49.456116+0000","last_fullsized":"2024-02-18T11:38:49.456116+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:37:07.582626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-18T11:38:47.841620+0000","last_change":"2024-02-18T11:38:47.837041+0000","last_active":"2024-02-18T11:38:47.841620+0000","last_peered":"2024-02-18T11:38:47.841620+0000","last_clean":"2024-02-18T11:38:47.841620+0000","last_became_active":"2024-02-18T11:38:47.836730+0000","last_became_peered":"2024-02-18T11:38:47.836730+0000","last_unstale":"2024-02-18T11:38:47.841620+0000","last_undegraded":"2024-02-18T11:38:47.841620+0000","last_fullsized":"2024-02-18T11:38:47.841620+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:56:59.496016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.856047+0000","last_change":"2024-02-18T11:38:47.849186+0000","last_active":"2024-02-18T11:38:49.856047+0000","last_peered":"2024-02-18T11:38:49.856047+0000","last_clean":"2024-02-18T11:38:49.856047+0000","last_became_active":"2024-02-18T11:38:47.848741+0000","last_became_peered":"2024-02-18T11:38:47.848741+0000","last_unstale":"2024-02-18T11:38:49.856047+0000","last_undegraded":"2024-02-18T11:38:49.856047+0000","last_fullsized":"2024-02-18T11:38:49.856047+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:19:31.971674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.854044+0000","last_change":"2024-02-18T11:38:47.841231+0000","last_active":"2024-02-18T11:38:49.854044+0000","last_peered":"2024-02-18T11:38:49.854044+0000","last_clean":"2024-02-18T11:38:49.854044+0000","last_became_active":"2024-02-18T11:38:47.840860+0000","last_became_peered":"2024-02-18T11:38:47.840860+0000","last_unstale":"2024-02-18T11:38:49.854044+0000","last_undegraded":"2024-02-18T11:38:49.854044+0000","last_fullsized":"2024-02-18T11:38:49.854044+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:29:00.036507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.855113+0000","last_change":"2024-02-18T11:38:47.839901+0000","last_active":"2024-02-18T11:38:49.855113+0000","last_peered":"2024-02-18T11:38:49.855113+0000","last_clean":"2024-02-18T11:38:49.855113+0000","last_became_active":"2024-02-18T11:38:47.839532+0000","last_became_peered":"2024-02-18T11:38:47.839532+0000","last_unstale":"2024-02-18T11:38:49.855113+0000","last_undegraded":"2024-02-18T11:38:49.855113+0000","last_fullsized":"2024-02-18T11:38:49.855113+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T21:32:14.369846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4941,"num_objects":52,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":13,"num_read_kb":11,"num_write":110,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":217088,"data_stored":14561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318286,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":933888,"data_stored":445784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579922,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":41032,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704120,"statfs":{"total":95995035648,"available":95953018880,"internally_reserved":0,"allocated":1568768,"data_stored":1038760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":40434112},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743447,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":950272,"data_stored":446020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005084,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":942080,"data_stored":445828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266722,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":41000,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704152,"statfs":{"total":95995035648,"available":95953051648,"internally_reserved":0,"allocated":1540096,"data_stored":1037362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430247,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":933888,"data_stored":445292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574910,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":942080,"data_stored":446814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738436,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":958464,"data_stored":446680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411386,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":933888,"data_stored":438278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117045,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":925696,"data_stored":436736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":35,"seq":150323855409,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36256,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708896,"statfs":{"total":95995035648,"available":95957909504,"internally_reserved":0,"allocated":942080,"data_stored":445846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691884,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32692,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712460,"statfs":{"total":95995035648,"available":95961559040,"internally_reserved":0,"allocated":1552384,"data_stored":1037437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-18T11:38:54.318 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-18T11:38:54.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:53 smithi188 ceph-mon[36925]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:38:54.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:53 smithi188 ceph-mon[36925]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:38:54.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:53 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/3839470194' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-18T11:38:54.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.462+0000 7fe6e321c640 1 -- 172.21.15.12:0/1133886932 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6dc1000e0 msgr2=0x7fe6dc100500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:54.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.462+0000 7fe6e321c640 1 --2- 172.21.15.12:0/1133886932 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6dc1000e0 0x7fe6dc100500 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0009900 tx=0x7fe6d002bc30 comp rx=0 tx=0).stop 2024-02-18T11:38:54.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.462+0000 7fe6e321c640 1 -- 172.21.15.12:0/1133886932 shutdown_connections 2024-02-18T11:38:54.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.462+0000 7fe6e321c640 1 --2- 172.21.15.12:0/1133886932 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6dc101c90 0x7fe6dc108520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:54.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.462+0000 7fe6e321c640 1 --2- 172.21.15.12:0/1133886932 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe6dc1012d0 0x7fe6dc101750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:54.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.462+0000 7fe6e321c640 1 --2- 172.21.15.12:0/1133886932 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6dc1000e0 0x7fe6dc100500 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:54.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.462+0000 7fe6e321c640 1 -- 172.21.15.12:0/1133886932 >> 172.21.15.12:0/1133886932 conn(0x7fe6dc0fb670 msgr2=0x7fe6dc0fda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:54.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.462+0000 7fe6e321c640 1 -- 172.21.15.12:0/1133886932 shutdown_connections 2024-02-18T11:38:54.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.462+0000 7fe6e321c640 1 -- 172.21.15.12:0/1133886932 wait complete. 2024-02-18T11:38:54.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.462+0000 7fe6e321c640 1 Processor -- start 2024-02-18T11:38:54.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.462+0000 7fe6e321c640 1 -- start start 2024-02-18T11:38:54.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.462+0000 7fe6e321c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6dc1000e0 0x7fe6dc197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:54.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.462+0000 7fe6e321c640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe6dc1012d0 0x7fe6dc197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:54.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.462+0000 7fe6e321c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6dc101c90 0x7fe6dc19e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:54.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.462+0000 7fe6e321c640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6dc19ef20 con 0x7fe6dc1000e0 2024-02-18T11:38:54.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.462+0000 7fe6e321c640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6dc19f090 con 0x7fe6dc1012d0 2024-02-18T11:38:54.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.462+0000 7fe6e321c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6dc19f250 con 0x7fe6dc101c90 2024-02-18T11:38:54.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.463+0000 7fe6e0fb8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6dc1000e0 0x7fe6dc197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:54.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.463+0000 7fe6e0fb8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6dc1000e0 0x7fe6dc197430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43730/0 (socket says 172.21.15.12:43730) 2024-02-18T11:38:54.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.463+0000 7fe6e0fb8640 1 -- 172.21.15.12:0/870648465 learned_addr learned my addr 172.21.15.12:0/870648465 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:38:54.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.463+0000 7fe6e0fb8640 1 -- 172.21.15.12:0/870648465 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6dc101c90 msgr2=0x7fe6dc19e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:54.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.463+0000 7fe6e0fb8640 1 --2- 172.21.15.12:0/870648465 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6dc101c90 0x7fe6dc19e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:54.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.463+0000 7fe6e0fb8640 1 -- 172.21.15.12:0/870648465 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe6dc1012d0 msgr2=0x7fe6dc197970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:54.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.463+0000 7fe6e0fb8640 1 --2- 172.21.15.12:0/870648465 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe6dc1012d0 0x7fe6dc197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:54.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.463+0000 7fe6e0fb8640 1 -- 172.21.15.12:0/870648465 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6d0009560 con 0x7fe6dc1000e0 2024-02-18T11:38:54.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.463+0000 7fe6e0fb8640 1 --2- 172.21.15.12:0/870648465 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6dc1000e0 0x7fe6dc197430 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0036040 tx=0x7fe6d00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:54.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.463+0000 7fe6d9ffb640 1 -- 172.21.15.12:0/870648465 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6d00044e0 con 0x7fe6dc1000e0 2024-02-18T11:38:54.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.463+0000 7fe6e321c640 1 -- 172.21.15.12:0/870648465 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6dc19f4d0 con 0x7fe6dc1000e0 2024-02-18T11:38:54.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.463+0000 7fe6d9ffb640 1 -- 172.21.15.12:0/870648465 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe6d00054e0 con 0x7fe6dc1000e0 2024-02-18T11:38:54.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.463+0000 7fe6d9ffb640 1 -- 172.21.15.12:0/870648465 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6d003e450 con 0x7fe6dc1000e0 2024-02-18T11:38:54.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.463+0000 7fe6e321c640 1 -- 172.21.15.12:0/870648465 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6dc19fa60 con 0x7fe6dc1000e0 2024-02-18T11:38:54.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.464+0000 7fe6e321c640 1 -- 172.21.15.12:0/870648465 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6dc10bbb0 con 0x7fe6dc1000e0 2024-02-18T11:38:54.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.465+0000 7fe6d9ffb640 1 -- 172.21.15.12:0/870648465 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 137296+0+0 (secure 0 0 0) 0x7fe6d0003710 con 0x7fe6dc1000e0 2024-02-18T11:38:54.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.466+0000 7fe6d9ffb640 1 --2- 172.21.15.12:0/870648465 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fe6cc0a27e0 0x7fe6cc0a4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:54.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.466+0000 7fe6d9ffb640 1 -- 172.21.15.12:0/870648465 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fe6d00f0900 con 0x7fe6dc1000e0 2024-02-18T11:38:54.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.466+0000 7fe6dbfff640 1 --2- 172.21.15.12:0/870648465 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fe6cc0a27e0 0x7fe6cc0a4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:54.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.466+0000 7fe6dbfff640 1 --2- 172.21.15.12:0/870648465 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fe6cc0a27e0 0x7fe6cc0a4ca0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc198940 tx=0x7fe6c8006c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:54.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.470+0000 7fe6d9ffb640 1 -- 172.21.15.12:0/870648465 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe6d00b97f0 con 0x7fe6dc1000e0 2024-02-18T11:38:54.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.648+0000 7fe6e321c640 1 -- 172.21.15.12:0/870648465 --> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe6dc069f70 con 0x7fe6cc0a27e0 2024-02-18T11:38:54.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.655+0000 7fe6d9ffb640 1 -- 172.21.15.12:0/870648465 <== mgr.14152 v2:172.21.15.12:6800/1153736685 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+258779 (secure 0 0 0) 0x7fe6dc069f70 con 0x7fe6cc0a27e0 2024-02-18T11:38:54.652 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:38:54.653 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-02-18T11:38:54.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.658+0000 7fe6e321c640 1 -- 172.21.15.12:0/870648465 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fe6cc0a27e0 msgr2=0x7fe6cc0a4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:54.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.658+0000 7fe6e321c640 1 --2- 172.21.15.12:0/870648465 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fe6cc0a27e0 0x7fe6cc0a4ca0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc198940 tx=0x7fe6c8006c90 comp rx=0 tx=0).stop 2024-02-18T11:38:54.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.658+0000 7fe6e321c640 1 -- 172.21.15.12:0/870648465 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6dc1000e0 msgr2=0x7fe6dc197430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:54.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.658+0000 7fe6e321c640 1 --2- 172.21.15.12:0/870648465 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6dc1000e0 0x7fe6dc197430 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0036040 tx=0x7fe6d00042f0 comp rx=0 tx=0).stop 2024-02-18T11:38:54.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.658+0000 7fe6e321c640 1 -- 172.21.15.12:0/870648465 shutdown_connections 2024-02-18T11:38:54.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.658+0000 7fe6e321c640 1 --2- 172.21.15.12:0/870648465 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fe6cc0a27e0 0x7fe6cc0a4ca0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:54.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.658+0000 7fe6e321c640 1 --2- 172.21.15.12:0/870648465 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6dc101c90 0x7fe6dc19e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:54.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.658+0000 7fe6e321c640 1 --2- 172.21.15.12:0/870648465 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe6dc1012d0 0x7fe6dc197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:54.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.658+0000 7fe6e321c640 1 --2- 172.21.15.12:0/870648465 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6dc1000e0 0x7fe6dc197430 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:54.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.658+0000 7fe6e321c640 1 -- 172.21.15.12:0/870648465 >> 172.21.15.12:0/870648465 conn(0x7fe6dc0fb670 msgr2=0x7fe6dc069860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:54.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.658+0000 7fe6e321c640 1 -- 172.21.15.12:0/870648465 shutdown_connections 2024-02-18T11:38:54.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:54.658+0000 7fe6e321c640 1 -- 172.21.15.12:0/870648465 wait complete. 2024-02-18T11:38:54.663 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":288,"stamp":"2024-02-18T11:38:53.016022+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595309,"num_objects":54,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":139,"num_read_kb":120,"num_write":291,"num_write_kb":2975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":234,"ondisk_log_size":234,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":234,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":391128,"kb_used_data":12816,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124550696,"statfs":{"total":1151940427776,"available":1151539912704,"internally_reserved":0,"allocated":13123584,"data_stored":7110837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":387233082},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4941,"num_objects":52,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":13,"num_read_kb":11,"num_write":110,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.501108"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.855808+0000","last_change":"2024-02-18T11:38:47.847524+0000","last_active":"2024-02-18T11:38:49.855808+0000","last_peered":"2024-02-18T11:38:49.855808+0000","last_clean":"2024-02-18T11:38:49.855808+0000","last_became_active":"2024-02-18T11:38:47.846756+0000","last_became_peered":"2024-02-18T11:38:47.846756+0000","last_unstale":"2024-02-18T11:38:49.855808+0000","last_undegraded":"2024-02-18T11:38:49.855808+0000","last_fullsized":"2024-02-18T11:38:49.855808+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:23:02.715418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:48.827338+0000","last_change":"2024-02-18T11:38:47.847391+0000","last_active":"2024-02-18T11:38:48.827338+0000","last_peered":"2024-02-18T11:38:48.827338+0000","last_clean":"2024-02-18T11:38:48.827338+0000","last_became_active":"2024-02-18T11:38:47.846558+0000","last_became_peered":"2024-02-18T11:38:47.846558+0000","last_unstale":"2024-02-18T11:38:48.827338+0000","last_undegraded":"2024-02-18T11:38:48.827338+0000","last_fullsized":"2024-02-18T11:38:48.827338+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:37:34.940328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:48.827305+0000","last_change":"2024-02-18T11:38:47.850143+0000","last_active":"2024-02-18T11:38:48.827305+0000","last_peered":"2024-02-18T11:38:48.827305+0000","last_clean":"2024-02-18T11:38:48.827305+0000","last_became_active":"2024-02-18T11:38:47.845844+0000","last_became_peered":"2024-02-18T11:38:47.845844+0000","last_unstale":"2024-02-18T11:38:48.827305+0000","last_undegraded":"2024-02-18T11:38:48.827305+0000","last_fullsized":"2024-02-18T11:38:48.827305+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T12:45:15.638456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.088979+0000","last_change":"2024-02-18T11:38:47.841317+0000","last_active":"2024-02-18T11:38:49.088979+0000","last_peered":"2024-02-18T11:38:49.088979+0000","last_clean":"2024-02-18T11:38:49.088979+0000","last_became_active":"2024-02-18T11:38:47.840982+0000","last_became_peered":"2024-02-18T11:38:47.840982+0000","last_unstale":"2024-02-18T11:38:49.088979+0000","last_undegraded":"2024-02-18T11:38:49.088979+0000","last_fullsized":"2024-02-18T11:38:49.088979+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:53:40.115908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:48.832632+0000","last_change":"2024-02-18T11:38:47.851422+0000","last_active":"2024-02-18T11:38:48.832632+0000","last_peered":"2024-02-18T11:38:48.832632+0000","last_clean":"2024-02-18T11:38:48.832632+0000","last_became_active":"2024-02-18T11:38:47.850479+0000","last_became_peered":"2024-02-18T11:38:47.850479+0000","last_unstale":"2024-02-18T11:38:48.832632+0000","last_undegraded":"2024-02-18T11:38:48.832632+0000","last_fullsized":"2024-02-18T11:38:48.832632+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:09:21.200282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.853607+0000","last_change":"2024-02-18T11:38:47.837468+0000","last_active":"2024-02-18T11:38:49.853607+0000","last_peered":"2024-02-18T11:38:49.853607+0000","last_clean":"2024-02-18T11:38:49.853607+0000","last_became_active":"2024-02-18T11:38:47.837129+0000","last_became_peered":"2024-02-18T11:38:47.837129+0000","last_unstale":"2024-02-18T11:38:49.853607+0000","last_undegraded":"2024-02-18T11:38:49.853607+0000","last_fullsized":"2024-02-18T11:38:49.853607+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:12:01.954496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:48.827918+0000","last_change":"2024-02-18T11:38:47.849467+0000","last_active":"2024-02-18T11:38:48.827918+0000","last_peered":"2024-02-18T11:38:48.827918+0000","last_clean":"2024-02-18T11:38:48.827918+0000","last_became_active":"2024-02-18T11:38:47.848938+0000","last_became_peered":"2024-02-18T11:38:47.848938+0000","last_unstale":"2024-02-18T11:38:48.827918+0000","last_undegraded":"2024-02-18T11:38:48.827918+0000","last_fullsized":"2024-02-18T11:38:48.827918+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:39:16.680834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-18T11:38:47.841833+0000","last_change":"2024-02-18T11:38:47.830143+0000","last_active":"2024-02-18T11:38:47.841833+0000","last_peered":"2024-02-18T11:38:47.841833+0000","last_clean":"2024-02-18T11:38:47.841833+0000","last_became_active":"2024-02-18T11:38:47.829765+0000","last_became_peered":"2024-02-18T11:38:47.829765+0000","last_unstale":"2024-02-18T11:38:47.841833+0000","last_undegraded":"2024-02-18T11:38:47.841833+0000","last_fullsized":"2024-02-18T11:38:47.841833+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T12:43:49.777465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.857235+0000","last_change":"2024-02-18T11:38:47.840477+0000","last_active":"2024-02-18T11:38:49.857235+0000","last_peered":"2024-02-18T11:38:49.857235+0000","last_clean":"2024-02-18T11:38:49.857235+0000","last_became_active":"2024-02-18T11:38:47.840136+0000","last_became_peered":"2024-02-18T11:38:47.840136+0000","last_unstale":"2024-02-18T11:38:49.857235+0000","last_undegraded":"2024-02-18T11:38:49.857235+0000","last_fullsized":"2024-02-18T11:38:49.857235+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:27:28.272633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.854739+0000","last_change":"2024-02-18T11:38:47.836478+0000","last_active":"2024-02-18T11:38:49.854739+0000","last_peered":"2024-02-18T11:38:49.854739+0000","last_clean":"2024-02-18T11:38:49.854739+0000","last_became_active":"2024-02-18T11:38:47.836161+0000","last_became_peered":"2024-02-18T11:38:47.836161+0000","last_unstale":"2024-02-18T11:38:49.854739+0000","last_undegraded":"2024-02-18T11:38:49.854739+0000","last_fullsized":"2024-02-18T11:38:49.854739+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:28:13.828518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:48.828574+0000","last_change":"2024-02-18T11:38:47.849435+0000","last_active":"2024-02-18T11:38:48.828574+0000","last_peered":"2024-02-18T11:38:48.828574+0000","last_clean":"2024-02-18T11:38:48.828574+0000","last_became_active":"2024-02-18T11:38:47.848947+0000","last_became_peered":"2024-02-18T11:38:47.848947+0000","last_unstale":"2024-02-18T11:38:48.828574+0000","last_undegraded":"2024-02-18T11:38:48.828574+0000","last_fullsized":"2024-02-18T11:38:48.828574+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T11:48:14.101732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.089038+0000","last_change":"2024-02-18T11:38:47.840366+0000","last_active":"2024-02-18T11:38:49.089038+0000","last_peered":"2024-02-18T11:38:49.089038+0000","last_clean":"2024-02-18T11:38:49.089038+0000","last_became_active":"2024-02-18T11:38:47.840044+0000","last_became_peered":"2024-02-18T11:38:47.840044+0000","last_unstale":"2024-02-18T11:38:49.089038+0000","last_undegraded":"2024-02-18T11:38:49.089038+0000","last_fullsized":"2024-02-18T11:38:49.089038+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:31:05.184418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.853951+0000","last_change":"2024-02-18T11:38:47.844433+0000","last_active":"2024-02-18T11:38:49.853951+0000","last_peered":"2024-02-18T11:38:49.853951+0000","last_clean":"2024-02-18T11:38:49.853951+0000","last_became_active":"2024-02-18T11:38:47.844048+0000","last_became_peered":"2024-02-18T11:38:47.844048+0000","last_unstale":"2024-02-18T11:38:49.853951+0000","last_undegraded":"2024-02-18T11:38:49.853951+0000","last_fullsized":"2024-02-18T11:38:49.853951+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:08:27.698272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:48.827089+0000","last_change":"2024-02-18T11:38:47.842819+0000","last_active":"2024-02-18T11:38:48.827089+0000","last_peered":"2024-02-18T11:38:48.827089+0000","last_clean":"2024-02-18T11:38:48.827089+0000","last_became_active":"2024-02-18T11:38:47.842489+0000","last_became_peered":"2024-02-18T11:38:47.842489+0000","last_unstale":"2024-02-18T11:38:48.827089+0000","last_undegraded":"2024-02-18T11:38:48.827089+0000","last_fullsized":"2024-02-18T11:38:48.827089+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:03:21.808910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:48.826981+0000","last_change":"2024-02-18T11:38:47.838399+0000","last_active":"2024-02-18T11:38:48.826981+0000","last_peered":"2024-02-18T11:38:48.826981+0000","last_clean":"2024-02-18T11:38:48.826981+0000","last_became_active":"2024-02-18T11:38:47.838076+0000","last_became_peered":"2024-02-18T11:38:47.838076+0000","last_unstale":"2024-02-18T11:38:48.826981+0000","last_undegraded":"2024-02-18T11:38:48.826981+0000","last_fullsized":"2024-02-18T11:38:48.826981+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:26:58.017359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.855748+0000","last_change":"2024-02-18T11:38:47.838729+0000","last_active":"2024-02-18T11:38:49.855748+0000","last_peered":"2024-02-18T11:38:49.855748+0000","last_clean":"2024-02-18T11:38:49.855748+0000","last_became_active":"2024-02-18T11:38:47.838267+0000","last_became_peered":"2024-02-18T11:38:47.838267+0000","last_unstale":"2024-02-18T11:38:49.855748+0000","last_undegraded":"2024-02-18T11:38:49.855748+0000","last_fullsized":"2024-02-18T11:38:49.855748+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:08:37.797780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.855825+0000","last_change":"2024-02-18T11:38:47.847817+0000","last_active":"2024-02-18T11:38:49.855825+0000","last_peered":"2024-02-18T11:38:49.855825+0000","last_clean":"2024-02-18T11:38:49.855825+0000","last_became_active":"2024-02-18T11:38:47.847200+0000","last_became_peered":"2024-02-18T11:38:47.847200+0000","last_unstale":"2024-02-18T11:38:49.855825+0000","last_undegraded":"2024-02-18T11:38:49.855825+0000","last_fullsized":"2024-02-18T11:38:49.855825+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:22:12.425623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:48.844543+0000","last_change":"2024-02-18T11:38:47.843954+0000","last_active":"2024-02-18T11:38:48.844543+0000","last_peered":"2024-02-18T11:38:48.844543+0000","last_clean":"2024-02-18T11:38:48.844543+0000","last_became_active":"2024-02-18T11:38:47.840084+0000","last_became_peered":"2024-02-18T11:38:47.840084+0000","last_unstale":"2024-02-18T11:38:48.844543+0000","last_undegraded":"2024-02-18T11:38:48.844543+0000","last_fullsized":"2024-02-18T11:38:48.844543+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:21:13.124696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-18T11:38:47.835228+0000","last_change":"2024-02-18T11:38:47.834262+0000","last_active":"2024-02-18T11:38:47.835228+0000","last_peered":"2024-02-18T11:38:47.835228+0000","last_clean":"2024-02-18T11:38:47.835228+0000","last_became_active":"2024-02-18T11:38:47.833948+0000","last_became_peered":"2024-02-18T11:38:47.833948+0000","last_unstale":"2024-02-18T11:38:47.835228+0000","last_undegraded":"2024-02-18T11:38:47.835228+0000","last_fullsized":"2024-02-18T11:38:47.835228+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:34:14.508236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'9","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:50.863816+0000","last_change":"2024-02-18T11:38:46.533805+0000","last_active":"2024-02-18T11:38:50.863816+0000","last_peered":"2024-02-18T11:38:50.863816+0000","last_clean":"2024-02-18T11:38:50.863816+0000","last_became_active":"2024-02-18T11:38:46.533503+0000","last_became_peered":"2024-02-18T11:38:46.533503+0000","last_unstale":"2024-02-18T11:38:50.863816+0000","last_undegraded":"2024-02-18T11:38:50.863816+0000","last_fullsized":"2024-02-18T11:38:50.863816+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:05:28.087740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-18T11:38:47.841388+0000","last_change":"2024-02-18T11:38:47.833802+0000","last_active":"2024-02-18T11:38:47.841388+0000","last_peered":"2024-02-18T11:38:47.841388+0000","last_clean":"2024-02-18T11:38:47.841388+0000","last_became_active":"2024-02-18T11:38:47.833441+0000","last_became_peered":"2024-02-18T11:38:47.833441+0000","last_unstale":"2024-02-18T11:38:47.841388+0000","last_undegraded":"2024-02-18T11:38:47.841388+0000","last_fullsized":"2024-02-18T11:38:47.841388+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T12:10:09.276997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'1","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-18T11:38:47.855770+0000","last_change":"2024-02-18T11:38:46.540718+0000","last_active":"2024-02-18T11:38:47.855770+0000","last_peered":"2024-02-18T11:38:47.855770+0000","last_clean":"2024-02-18T11:38:47.855770+0000","last_became_active":"2024-02-18T11:38:46.540413+0000","last_became_peered":"2024-02-18T11:38:46.540413+0000","last_unstale":"2024-02-18T11:38:47.855770+0000","last_undegraded":"2024-02-18T11:38:47.855770+0000","last_fullsized":"2024-02-18T11:38:47.855770+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:17:53.287376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:48.827293+0000","last_change":"2024-02-18T11:38:47.846400+0000","last_active":"2024-02-18T11:38:48.827293+0000","last_peered":"2024-02-18T11:38:48.827293+0000","last_clean":"2024-02-18T11:38:48.827293+0000","last_became_active":"2024-02-18T11:38:47.845934+0000","last_became_peered":"2024-02-18T11:38:47.845934+0000","last_unstale":"2024-02-18T11:38:48.827293+0000","last_undegraded":"2024-02-18T11:38:48.827293+0000","last_fullsized":"2024-02-18T11:38:48.827293+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:20:12.093697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"87'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:50.860988+0000","last_change":"2024-02-18T11:38:46.537340+0000","last_active":"2024-02-18T11:38:50.860988+0000","last_peered":"2024-02-18T11:38:50.860988+0000","last_clean":"2024-02-18T11:38:50.860988+0000","last_became_active":"2024-02-18T11:38:46.537019+0000","last_became_peered":"2024-02-18T11:38:46.537019+0000","last_unstale":"2024-02-18T11:38:50.860988+0000","last_undegraded":"2024-02-18T11:38:50.860988+0000","last_fullsized":"2024-02-18T11:38:50.860988+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T23:00:50.052654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.856118+0000","last_change":"2024-02-18T11:38:47.852656+0000","last_active":"2024-02-18T11:38:49.856118+0000","last_peered":"2024-02-18T11:38:49.856118+0000","last_clean":"2024-02-18T11:38:49.856118+0000","last_became_active":"2024-02-18T11:38:47.852279+0000","last_became_peered":"2024-02-18T11:38:47.852279+0000","last_unstale":"2024-02-18T11:38:49.856118+0000","last_undegraded":"2024-02-18T11:38:49.856118+0000","last_fullsized":"2024-02-18T11:38:49.856118+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:51:48.911504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:48.832752+0000","last_change":"2024-02-18T11:38:46.533091+0000","last_active":"2024-02-18T11:38:48.832752+0000","last_peered":"2024-02-18T11:38:48.832752+0000","last_clean":"2024-02-18T11:38:48.832752+0000","last_became_active":"2024-02-18T11:38:46.532504+0000","last_became_peered":"2024-02-18T11:38:46.532504+0000","last_unstale":"2024-02-18T11:38:48.832752+0000","last_undegraded":"2024-02-18T11:38:48.832752+0000","last_fullsized":"2024-02-18T11:38:48.832752+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:12:00.529365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.456298+0000","last_change":"2024-02-18T11:38:47.836578+0000","last_active":"2024-02-18T11:38:49.456298+0000","last_peered":"2024-02-18T11:38:49.456298+0000","last_clean":"2024-02-18T11:38:49.456298+0000","last_became_active":"2024-02-18T11:38:47.836217+0000","last_became_peered":"2024-02-18T11:38:47.836217+0000","last_unstale":"2024-02-18T11:38:49.456298+0000","last_undegraded":"2024-02-18T11:38:49.456298+0000","last_fullsized":"2024-02-18T11:38:49.456298+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T21:53:39.011822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"87'6","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:51.545729+0000","last_change":"2024-02-18T11:38:46.532196+0000","last_active":"2024-02-18T11:38:51.545729+0000","last_peered":"2024-02-18T11:38:51.545729+0000","last_clean":"2024-02-18T11:38:51.545729+0000","last_became_active":"2024-02-18T11:38:46.531868+0000","last_became_peered":"2024-02-18T11:38:46.531868+0000","last_unstale":"2024-02-18T11:38:51.545729+0000","last_undegraded":"2024-02-18T11:38:51.545729+0000","last_fullsized":"2024-02-18T11:38:51.545729+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:52:07.396279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:48.832577+0000","last_change":"2024-02-18T11:38:47.837329+0000","last_active":"2024-02-18T11:38:48.832577+0000","last_peered":"2024-02-18T11:38:48.832577+0000","last_clean":"2024-02-18T11:38:48.832577+0000","last_became_active":"2024-02-18T11:38:47.836989+0000","last_became_peered":"2024-02-18T11:38:47.836989+0000","last_unstale":"2024-02-18T11:38:48.832577+0000","last_undegraded":"2024-02-18T11:38:48.832577+0000","last_fullsized":"2024-02-18T11:38:48.832577+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:21:16.801916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"87'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:50.862610+0000","last_change":"2024-02-18T11:38:46.533244+0000","last_active":"2024-02-18T11:38:50.862610+0000","last_peered":"2024-02-18T11:38:50.862610+0000","last_clean":"2024-02-18T11:38:50.862610+0000","last_became_active":"2024-02-18T11:38:46.532326+0000","last_became_peered":"2024-02-18T11:38:46.532326+0000","last_unstale":"2024-02-18T11:38:50.862610+0000","last_undegraded":"2024-02-18T11:38:50.862610+0000","last_fullsized":"2024-02-18T11:38:50.862610+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:32:32.979056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.456426+0000","last_change":"2024-02-18T11:38:47.824560+0000","last_active":"2024-02-18T11:38:49.456426+0000","last_peered":"2024-02-18T11:38:49.456426+0000","last_clean":"2024-02-18T11:38:49.456426+0000","last_became_active":"2024-02-18T11:38:47.824095+0000","last_became_peered":"2024-02-18T11:38:47.824095+0000","last_unstale":"2024-02-18T11:38:49.456426+0000","last_undegraded":"2024-02-18T11:38:49.456426+0000","last_fullsized":"2024-02-18T11:38:49.456426+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:27:09.870585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"87'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:50.862291+0000","last_change":"2024-02-18T11:38:46.526024+0000","last_active":"2024-02-18T11:38:50.862291+0000","last_peered":"2024-02-18T11:38:50.862291+0000","last_clean":"2024-02-18T11:38:50.862291+0000","last_became_active":"2024-02-18T11:38:46.525735+0000","last_became_peered":"2024-02-18T11:38:46.525735+0000","last_unstale":"2024-02-18T11:38:50.862291+0000","last_undegraded":"2024-02-18T11:38:50.862291+0000","last_fullsized":"2024-02-18T11:38:50.862291+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:25:10.659210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-18T11:38:47.843658+0000","last_change":"2024-02-18T11:38:47.841743+0000","last_active":"2024-02-18T11:38:47.843658+0000","last_peered":"2024-02-18T11:38:47.843658+0000","last_clean":"2024-02-18T11:38:47.843658+0000","last_became_active":"2024-02-18T11:38:47.840848+0000","last_became_peered":"2024-02-18T11:38:47.840848+0000","last_unstale":"2024-02-18T11:38:47.843658+0000","last_undegraded":"2024-02-18T11:38:47.843658+0000","last_fullsized":"2024-02-18T11:38:47.843658+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T11:51:35.764831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"87'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.456175+0000","last_change":"2024-02-18T11:38:46.531355+0000","last_active":"2024-02-18T11:38:49.456175+0000","last_peered":"2024-02-18T11:38:49.456175+0000","last_clean":"2024-02-18T11:38:49.456175+0000","last_became_active":"2024-02-18T11:38:46.530601+0000","last_became_peered":"2024-02-18T11:38:46.530601+0000","last_unstale":"2024-02-18T11:38:49.456175+0000","last_undegraded":"2024-02-18T11:38:49.456175+0000","last_fullsized":"2024-02-18T11:38:49.456175+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:55:29.670274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:48.844447+0000","last_change":"2024-02-18T11:38:47.844506+0000","last_active":"2024-02-18T11:38:48.844447+0000","last_peered":"2024-02-18T11:38:48.844447+0000","last_clean":"2024-02-18T11:38:48.844447+0000","last_became_active":"2024-02-18T11:38:47.844158+0000","last_became_peered":"2024-02-18T11:38:47.844158+0000","last_unstale":"2024-02-18T11:38:48.844447+0000","last_undegraded":"2024-02-18T11:38:48.844447+0000","last_fullsized":"2024-02-18T11:38:48.844447+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T23:13:17.360077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"87'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.858291+0000","last_change":"2024-02-18T11:38:46.536313+0000","last_active":"2024-02-18T11:38:49.858291+0000","last_peered":"2024-02-18T11:38:49.858291+0000","last_clean":"2024-02-18T11:38:49.858291+0000","last_became_active":"2024-02-18T11:38:46.535910+0000","last_became_peered":"2024-02-18T11:38:46.535910+0000","last_unstale":"2024-02-18T11:38:49.858291+0000","last_undegraded":"2024-02-18T11:38:49.858291+0000","last_fullsized":"2024-02-18T11:38:49.858291+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:56:16.606415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.856993+0000","last_change":"2024-02-18T11:38:47.836131+0000","last_active":"2024-02-18T11:38:49.856993+0000","last_peered":"2024-02-18T11:38:49.856993+0000","last_clean":"2024-02-18T11:38:49.856993+0000","last_became_active":"2024-02-18T11:38:47.835702+0000","last_became_peered":"2024-02-18T11:38:47.835702+0000","last_unstale":"2024-02-18T11:38:49.856993+0000","last_undegraded":"2024-02-18T11:38:49.856993+0000","last_fullsized":"2024-02-18T11:38:49.856993+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:58:11.622972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"87'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.856996+0000","last_change":"2024-02-18T11:38:46.530031+0000","last_active":"2024-02-18T11:38:49.856996+0000","last_peered":"2024-02-18T11:38:49.856996+0000","last_clean":"2024-02-18T11:38:49.856996+0000","last_became_active":"2024-02-18T11:38:46.529674+0000","last_became_peered":"2024-02-18T11:38:46.529674+0000","last_unstale":"2024-02-18T11:38:49.856996+0000","last_undegraded":"2024-02-18T11:38:49.856996+0000","last_fullsized":"2024-02-18T11:38:49.856996+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T21:13:53.955970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:48.827261+0000","last_change":"2024-02-18T11:38:47.852340+0000","last_active":"2024-02-18T11:38:48.827261+0000","last_peered":"2024-02-18T11:38:48.827261+0000","last_clean":"2024-02-18T11:38:48.827261+0000","last_became_active":"2024-02-18T11:38:47.847978+0000","last_became_peered":"2024-02-18T11:38:47.847978+0000","last_unstale":"2024-02-18T11:38:48.827261+0000","last_undegraded":"2024-02-18T11:38:48.827261+0000","last_fullsized":"2024-02-18T11:38:48.827261+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:47:09.667619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'4","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.857723+0000","last_change":"2024-02-18T11:38:46.533511+0000","last_active":"2024-02-18T11:38:49.857723+0000","last_peered":"2024-02-18T11:38:49.857723+0000","last_clean":"2024-02-18T11:38:49.857723+0000","last_became_active":"2024-02-18T11:38:46.533162+0000","last_became_peered":"2024-02-18T11:38:46.533162+0000","last_unstale":"2024-02-18T11:38:49.857723+0000","last_undegraded":"2024-02-18T11:38:49.857723+0000","last_fullsized":"2024-02-18T11:38:49.857723+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:00:49.307489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.856288+0000","last_change":"2024-02-18T11:38:47.843835+0000","last_active":"2024-02-18T11:38:49.856288+0000","last_peered":"2024-02-18T11:38:49.856288+0000","last_clean":"2024-02-18T11:38:49.856288+0000","last_became_active":"2024-02-18T11:38:47.843517+0000","last_became_peered":"2024-02-18T11:38:47.843517+0000","last_unstale":"2024-02-18T11:38:49.856288+0000","last_undegraded":"2024-02-18T11:38:49.856288+0000","last_fullsized":"2024-02-18T11:38:49.856288+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:54:02.259854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'8","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.858239+0000","last_change":"2024-02-18T11:38:46.536900+0000","last_active":"2024-02-18T11:38:49.858239+0000","last_peered":"2024-02-18T11:38:49.858239+0000","last_clean":"2024-02-18T11:38:49.858239+0000","last_became_active":"2024-02-18T11:38:46.536563+0000","last_became_peered":"2024-02-18T11:38:46.536563+0000","last_unstale":"2024-02-18T11:38:49.858239+0000","last_undegraded":"2024-02-18T11:38:49.858239+0000","last_fullsized":"2024-02-18T11:38:49.858239+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T12:56:57.959482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:48.832845+0000","last_change":"2024-02-18T11:38:47.842452+0000","last_active":"2024-02-18T11:38:48.832845+0000","last_peered":"2024-02-18T11:38:48.832845+0000","last_clean":"2024-02-18T11:38:48.832845+0000","last_became_active":"2024-02-18T11:38:47.842105+0000","last_became_peered":"2024-02-18T11:38:47.842105+0000","last_unstale":"2024-02-18T11:38:48.832845+0000","last_undegraded":"2024-02-18T11:38:48.832845+0000","last_fullsized":"2024-02-18T11:38:48.832845+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:10:05.131880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'2","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-18T11:38:47.856142+0000","last_change":"2024-02-18T11:38:46.534220+0000","last_active":"2024-02-18T11:38:47.856142+0000","last_peered":"2024-02-18T11:38:47.856142+0000","last_clean":"2024-02-18T11:38:47.856142+0000","last_became_active":"2024-02-18T11:38:46.532929+0000","last_became_peered":"2024-02-18T11:38:46.532929+0000","last_unstale":"2024-02-18T11:38:47.856142+0000","last_undegraded":"2024-02-18T11:38:47.856142+0000","last_fullsized":"2024-02-18T11:38:47.856142+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:41:42.674196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.853814+0000","last_change":"2024-02-18T11:38:47.848434+0000","last_active":"2024-02-18T11:38:49.853814+0000","last_peered":"2024-02-18T11:38:49.853814+0000","last_clean":"2024-02-18T11:38:49.853814+0000","last_became_active":"2024-02-18T11:38:47.848092+0000","last_became_peered":"2024-02-18T11:38:47.848092+0000","last_unstale":"2024-02-18T11:38:49.853814+0000","last_undegraded":"2024-02-18T11:38:49.853814+0000","last_fullsized":"2024-02-18T11:38:49.853814+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:23:35.292982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"87'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.856206+0000","last_change":"2024-02-18T11:38:46.531579+0000","last_active":"2024-02-18T11:38:49.856206+0000","last_peered":"2024-02-18T11:38:49.856206+0000","last_clean":"2024-02-18T11:38:49.856206+0000","last_became_active":"2024-02-18T11:38:46.531236+0000","last_became_peered":"2024-02-18T11:38:46.531236+0000","last_unstale":"2024-02-18T11:38:49.856206+0000","last_undegraded":"2024-02-18T11:38:49.856206+0000","last_fullsized":"2024-02-18T11:38:49.856206+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:54:48.886442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:48.828336+0000","last_change":"2024-02-18T11:38:47.840418+0000","last_active":"2024-02-18T11:38:48.828336+0000","last_peered":"2024-02-18T11:38:48.828336+0000","last_clean":"2024-02-18T11:38:48.828336+0000","last_became_active":"2024-02-18T11:38:47.835202+0000","last_became_peered":"2024-02-18T11:38:47.835202+0000","last_unstale":"2024-02-18T11:38:48.828336+0000","last_undegraded":"2024-02-18T11:38:48.828336+0000","last_fullsized":"2024-02-18T11:38:48.828336+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:40:26.874931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'3","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:51.543777+0000","last_change":"2024-02-18T11:38:46.528086+0000","last_active":"2024-02-18T11:38:51.543777+0000","last_peered":"2024-02-18T11:38:51.543777+0000","last_clean":"2024-02-18T11:38:51.543777+0000","last_became_active":"2024-02-18T11:38:46.527753+0000","last_became_peered":"2024-02-18T11:38:46.527753+0000","last_unstale":"2024-02-18T11:38:51.543777+0000","last_undegraded":"2024-02-18T11:38:51.543777+0000","last_fullsized":"2024-02-18T11:38:51.543777+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:48:09.524280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:48.827834+0000","last_change":"2024-02-18T11:38:47.849139+0000","last_active":"2024-02-18T11:38:48.827834+0000","last_peered":"2024-02-18T11:38:48.827834+0000","last_clean":"2024-02-18T11:38:48.827834+0000","last_became_active":"2024-02-18T11:38:47.848752+0000","last_became_peered":"2024-02-18T11:38:47.848752+0000","last_unstale":"2024-02-18T11:38:48.827834+0000","last_undegraded":"2024-02-18T11:38:48.827834+0000","last_fullsized":"2024-02-18T11:38:48.827834+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:00:56.046670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"87'15","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:50.862743+0000","last_change":"2024-02-18T11:38:46.540471+0000","last_active":"2024-02-18T11:38:50.862743+0000","last_peered":"2024-02-18T11:38:50.862743+0000","last_clean":"2024-02-18T11:38:50.862743+0000","last_became_active":"2024-02-18T11:38:46.540169+0000","last_became_peered":"2024-02-18T11:38:46.540169+0000","last_unstale":"2024-02-18T11:38:50.862743+0000","last_undegraded":"2024-02-18T11:38:50.862743+0000","last_fullsized":"2024-02-18T11:38:50.862743+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:29:46.001510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.456087+0000","last_change":"2024-02-18T11:38:47.838077+0000","last_active":"2024-02-18T11:38:49.456087+0000","last_peered":"2024-02-18T11:38:49.456087+0000","last_clean":"2024-02-18T11:38:49.456087+0000","last_became_active":"2024-02-18T11:38:47.837137+0000","last_became_peered":"2024-02-18T11:38:47.837137+0000","last_unstale":"2024-02-18T11:38:49.456087+0000","last_undegraded":"2024-02-18T11:38:49.456087+0000","last_fullsized":"2024-02-18T11:38:49.456087+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:06:15.443851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:48.832894+0000","last_change":"2024-02-18T11:38:46.532828+0000","last_active":"2024-02-18T11:38:48.832894+0000","last_peered":"2024-02-18T11:38:48.832894+0000","last_clean":"2024-02-18T11:38:48.832894+0000","last_became_active":"2024-02-18T11:38:46.532515+0000","last_became_peered":"2024-02-18T11:38:46.532515+0000","last_unstale":"2024-02-18T11:38:48.832894+0000","last_undegraded":"2024-02-18T11:38:48.832894+0000","last_fullsized":"2024-02-18T11:38:48.832894+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:20:23.333916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.854948+0000","last_change":"2024-02-18T11:38:47.835481+0000","last_active":"2024-02-18T11:38:49.854948+0000","last_peered":"2024-02-18T11:38:49.854948+0000","last_clean":"2024-02-18T11:38:49.854948+0000","last_became_active":"2024-02-18T11:38:47.835071+0000","last_became_peered":"2024-02-18T11:38:47.835071+0000","last_unstale":"2024-02-18T11:38:49.854948+0000","last_undegraded":"2024-02-18T11:38:49.854948+0000","last_fullsized":"2024-02-18T11:38:49.854948+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T12:10:42.098646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:48.844409+0000","last_change":"2024-02-18T11:38:46.530294+0000","last_active":"2024-02-18T11:38:48.844409+0000","last_peered":"2024-02-18T11:38:48.844409+0000","last_clean":"2024-02-18T11:38:48.844409+0000","last_became_active":"2024-02-18T11:38:46.529913+0000","last_became_peered":"2024-02-18T11:38:46.529913+0000","last_unstale":"2024-02-18T11:38:48.844409+0000","last_undegraded":"2024-02-18T11:38:48.844409+0000","last_fullsized":"2024-02-18T11:38:48.844409+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:17:54.549659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.854095+0000","last_change":"2024-02-18T11:38:47.842020+0000","last_active":"2024-02-18T11:38:49.854095+0000","last_peered":"2024-02-18T11:38:49.854095+0000","last_clean":"2024-02-18T11:38:49.854095+0000","last_became_active":"2024-02-18T11:38:47.841656+0000","last_became_peered":"2024-02-18T11:38:47.841656+0000","last_unstale":"2024-02-18T11:38:49.854095+0000","last_undegraded":"2024-02-18T11:38:49.854095+0000","last_fullsized":"2024-02-18T11:38:49.854095+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:12:50.508340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"87'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:48.828019+0000","last_change":"2024-02-18T11:38:46.531867+0000","last_active":"2024-02-18T11:38:48.828019+0000","last_peered":"2024-02-18T11:38:48.828019+0000","last_clean":"2024-02-18T11:38:48.828019+0000","last_became_active":"2024-02-18T11:38:46.531051+0000","last_became_peered":"2024-02-18T11:38:46.531051+0000","last_unstale":"2024-02-18T11:38:48.828019+0000","last_undegraded":"2024-02-18T11:38:48.828019+0000","last_fullsized":"2024-02-18T11:38:48.828019+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:15:26.808764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.854486+0000","last_change":"2024-02-18T11:38:47.835160+0000","last_active":"2024-02-18T11:38:49.854486+0000","last_peered":"2024-02-18T11:38:49.854486+0000","last_clean":"2024-02-18T11:38:49.854486+0000","last_became_active":"2024-02-18T11:38:47.834813+0000","last_became_peered":"2024-02-18T11:38:47.834813+0000","last_unstale":"2024-02-18T11:38:49.854486+0000","last_undegraded":"2024-02-18T11:38:49.854486+0000","last_fullsized":"2024-02-18T11:38:49.854486+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:30:59.674796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.456423+0000","last_change":"2024-02-18T11:38:47.838197+0000","last_active":"2024-02-18T11:38:49.456423+0000","last_peered":"2024-02-18T11:38:49.456423+0000","last_clean":"2024-02-18T11:38:49.456423+0000","last_became_active":"2024-02-18T11:38:47.837393+0000","last_became_peered":"2024-02-18T11:38:47.837393+0000","last_unstale":"2024-02-18T11:38:49.456423+0000","last_undegraded":"2024-02-18T11:38:49.456423+0000","last_fullsized":"2024-02-18T11:38:49.456423+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:13:19.095082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'4","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:50.863756+0000","last_change":"2024-02-18T11:38:46.538682+0000","last_active":"2024-02-18T11:38:50.863756+0000","last_peered":"2024-02-18T11:38:50.863756+0000","last_clean":"2024-02-18T11:38:50.863756+0000","last_became_active":"2024-02-18T11:38:46.538356+0000","last_became_peered":"2024-02-18T11:38:46.538356+0000","last_unstale":"2024-02-18T11:38:50.863756+0000","last_undegraded":"2024-02-18T11:38:50.863756+0000","last_fullsized":"2024-02-18T11:38:50.863756+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:41:02.449905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-18T11:38:47.846833+0000","last_change":"2024-02-18T11:38:47.844448+0000","last_active":"2024-02-18T11:38:47.846833+0000","last_peered":"2024-02-18T11:38:47.846833+0000","last_clean":"2024-02-18T11:38:47.846833+0000","last_became_active":"2024-02-18T11:38:47.844048+0000","last_became_peered":"2024-02-18T11:38:47.844048+0000","last_unstale":"2024-02-18T11:38:47.846833+0000","last_undegraded":"2024-02-18T11:38:47.846833+0000","last_fullsized":"2024-02-18T11:38:47.846833+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:33:41.159473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'3","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:51.541339+0000","last_change":"2024-02-18T11:38:46.531504+0000","last_active":"2024-02-18T11:38:51.541339+0000","last_peered":"2024-02-18T11:38:51.541339+0000","last_clean":"2024-02-18T11:38:51.541339+0000","last_became_active":"2024-02-18T11:38:46.531173+0000","last_became_peered":"2024-02-18T11:38:46.531173+0000","last_unstale":"2024-02-18T11:38:51.541339+0000","last_undegraded":"2024-02-18T11:38:51.541339+0000","last_fullsized":"2024-02-18T11:38:51.541339+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:32:40.632054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":341,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:48.827763+0000","last_change":"2024-02-18T11:37:34.534077+0000","last_active":"2024-02-18T11:38:48.827763+0000","last_peered":"2024-02-18T11:38:48.827763+0000","last_clean":"2024-02-18T11:38:48.827763+0000","last_became_active":"2024-02-18T11:37:34.533557+0000","last_became_peered":"2024-02-18T11:37:34.533557+0000","last_unstale":"2024-02-18T11:38:48.827763+0000","last_undegraded":"2024-02-18T11:38:48.827763+0000","last_fullsized":"2024-02-18T11:38:48.827763+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:34:15.604912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:34:15.604912+0000","last_clean_scrub_stamp":"2024-02-18T11:34:15.604912+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T23:01:35.216671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.853984+0000","last_change":"2024-02-18T11:38:47.838123+0000","last_active":"2024-02-18T11:38:49.853984+0000","last_peered":"2024-02-18T11:38:49.853984+0000","last_clean":"2024-02-18T11:38:49.853984+0000","last_became_active":"2024-02-18T11:38:47.837713+0000","last_became_peered":"2024-02-18T11:38:47.837713+0000","last_unstale":"2024-02-18T11:38:49.853984+0000","last_undegraded":"2024-02-18T11:38:49.853984+0000","last_fullsized":"2024-02-18T11:38:49.853984+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T21:47:44.609399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:50.862743+0000","last_change":"2024-02-18T11:38:46.529508+0000","last_active":"2024-02-18T11:38:50.862743+0000","last_peered":"2024-02-18T11:38:50.862743+0000","last_clean":"2024-02-18T11:38:50.862743+0000","last_became_active":"2024-02-18T11:38:46.529165+0000","last_became_peered":"2024-02-18T11:38:46.529165+0000","last_unstale":"2024-02-18T11:38:50.862743+0000","last_undegraded":"2024-02-18T11:38:50.862743+0000","last_fullsized":"2024-02-18T11:38:50.862743+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:10:12.774664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"87'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:48.832804+0000","last_change":"2024-02-18T11:38:46.542236+0000","last_active":"2024-02-18T11:38:48.832804+0000","last_peered":"2024-02-18T11:38:48.832804+0000","last_clean":"2024-02-18T11:38:48.832804+0000","last_became_active":"2024-02-18T11:38:46.541901+0000","last_became_peered":"2024-02-18T11:38:46.541901+0000","last_unstale":"2024-02-18T11:38:48.832804+0000","last_undegraded":"2024-02-18T11:38:48.832804+0000","last_fullsized":"2024-02-18T11:38:48.832804+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:40:28.699330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.854718+0000","last_change":"2024-02-18T11:38:47.841210+0000","last_active":"2024-02-18T11:38:49.854718+0000","last_peered":"2024-02-18T11:38:49.854718+0000","last_clean":"2024-02-18T11:38:49.854718+0000","last_became_active":"2024-02-18T11:38:47.838873+0000","last_became_peered":"2024-02-18T11:38:47.838873+0000","last_unstale":"2024-02-18T11:38:49.854718+0000","last_undegraded":"2024-02-18T11:38:49.854718+0000","last_fullsized":"2024-02-18T11:38:49.854718+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T21:22:11.304237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:50.862850+0000","last_change":"2024-02-18T11:38:46.533820+0000","last_active":"2024-02-18T11:38:50.862850+0000","last_peered":"2024-02-18T11:38:50.862850+0000","last_clean":"2024-02-18T11:38:50.862850+0000","last_became_active":"2024-02-18T11:38:46.533342+0000","last_became_peered":"2024-02-18T11:38:46.533342+0000","last_unstale":"2024-02-18T11:38:50.862850+0000","last_undegraded":"2024-02-18T11:38:50.862850+0000","last_fullsized":"2024-02-18T11:38:50.862850+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:27:34.814511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-18T11:38:47.841517+0000","last_change":"2024-02-18T11:38:47.837034+0000","last_active":"2024-02-18T11:38:47.841517+0000","last_peered":"2024-02-18T11:38:47.841517+0000","last_clean":"2024-02-18T11:38:47.841517+0000","last_became_active":"2024-02-18T11:38:47.836340+0000","last_became_peered":"2024-02-18T11:38:47.836340+0000","last_unstale":"2024-02-18T11:38:47.841517+0000","last_undegraded":"2024-02-18T11:38:47.841517+0000","last_fullsized":"2024-02-18T11:38:47.841517+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T21:28:47.572516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.854297+0000","last_change":"2024-02-18T11:38:46.529389+0000","last_active":"2024-02-18T11:38:49.854297+0000","last_peered":"2024-02-18T11:38:49.854297+0000","last_clean":"2024-02-18T11:38:49.854297+0000","last_became_active":"2024-02-18T11:38:46.529076+0000","last_became_peered":"2024-02-18T11:38:46.529076+0000","last_unstale":"2024-02-18T11:38:49.854297+0000","last_undegraded":"2024-02-18T11:38:49.854297+0000","last_fullsized":"2024-02-18T11:38:49.854297+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T21:01:05.553901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:48.827378+0000","last_change":"2024-02-18T11:38:47.847677+0000","last_active":"2024-02-18T11:38:48.827378+0000","last_peered":"2024-02-18T11:38:48.827378+0000","last_clean":"2024-02-18T11:38:48.827378+0000","last_became_active":"2024-02-18T11:38:47.847051+0000","last_became_peered":"2024-02-18T11:38:47.847051+0000","last_unstale":"2024-02-18T11:38:48.827378+0000","last_undegraded":"2024-02-18T11:38:48.827378+0000","last_fullsized":"2024-02-18T11:38:48.827378+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:25:53.398173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.858228+0000","last_change":"2024-02-18T11:38:46.525030+0000","last_active":"2024-02-18T11:38:49.858228+0000","last_peered":"2024-02-18T11:38:49.858228+0000","last_clean":"2024-02-18T11:38:49.858228+0000","last_became_active":"2024-02-18T11:38:46.524405+0000","last_became_peered":"2024-02-18T11:38:46.524405+0000","last_unstale":"2024-02-18T11:38:49.858228+0000","last_undegraded":"2024-02-18T11:38:49.858228+0000","last_fullsized":"2024-02-18T11:38:49.858228+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T12:33:41.751625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:48.832564+0000","last_change":"2024-02-18T11:38:47.836583+0000","last_active":"2024-02-18T11:38:48.832564+0000","last_peered":"2024-02-18T11:38:48.832564+0000","last_clean":"2024-02-18T11:38:48.832564+0000","last_became_active":"2024-02-18T11:38:47.836138+0000","last_became_peered":"2024-02-18T11:38:47.836138+0000","last_unstale":"2024-02-18T11:38:48.832564+0000","last_undegraded":"2024-02-18T11:38:48.832564+0000","last_fullsized":"2024-02-18T11:38:48.832564+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:34:04.002137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-18T11:38:47.821129+0000","last_change":"2024-02-18T11:38:46.538553+0000","last_active":"2024-02-18T11:38:47.821129+0000","last_peered":"2024-02-18T11:38:47.821129+0000","last_clean":"2024-02-18T11:38:47.821129+0000","last_became_active":"2024-02-18T11:38:46.538216+0000","last_became_peered":"2024-02-18T11:38:46.538216+0000","last_unstale":"2024-02-18T11:38:47.821129+0000","last_undegraded":"2024-02-18T11:38:47.821129+0000","last_fullsized":"2024-02-18T11:38:47.821129+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:31:32.713327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.854428+0000","last_change":"2024-02-18T11:38:47.840452+0000","last_active":"2024-02-18T11:38:49.854428+0000","last_peered":"2024-02-18T11:38:49.854428+0000","last_clean":"2024-02-18T11:38:49.854428+0000","last_became_active":"2024-02-18T11:38:47.840071+0000","last_became_peered":"2024-02-18T11:38:47.840071+0000","last_unstale":"2024-02-18T11:38:49.854428+0000","last_undegraded":"2024-02-18T11:38:49.854428+0000","last_fullsized":"2024-02-18T11:38:49.854428+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:07:22.746854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-18T11:38:47.831525+0000","last_change":"2024-02-18T11:38:46.535460+0000","last_active":"2024-02-18T11:38:47.831525+0000","last_peered":"2024-02-18T11:38:47.831525+0000","last_clean":"2024-02-18T11:38:47.831525+0000","last_became_active":"2024-02-18T11:38:46.534626+0000","last_became_peered":"2024-02-18T11:38:46.534626+0000","last_unstale":"2024-02-18T11:38:47.831525+0000","last_undegraded":"2024-02-18T11:38:47.831525+0000","last_fullsized":"2024-02-18T11:38:47.831525+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:56:18.434798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-18T11:38:47.841881+0000","last_change":"2024-02-18T11:38:47.838616+0000","last_active":"2024-02-18T11:38:47.841881+0000","last_peered":"2024-02-18T11:38:47.841881+0000","last_clean":"2024-02-18T11:38:47.841881+0000","last_became_active":"2024-02-18T11:38:47.838301+0000","last_became_peered":"2024-02-18T11:38:47.838301+0000","last_unstale":"2024-02-18T11:38:47.841881+0000","last_undegraded":"2024-02-18T11:38:47.841881+0000","last_fullsized":"2024-02-18T11:38:47.841881+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:11:25.865726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.856138+0000","last_change":"2024-02-18T11:38:46.536544+0000","last_active":"2024-02-18T11:38:49.856138+0000","last_peered":"2024-02-18T11:38:49.856138+0000","last_clean":"2024-02-18T11:38:49.856138+0000","last_became_active":"2024-02-18T11:38:46.536215+0000","last_became_peered":"2024-02-18T11:38:46.536215+0000","last_unstale":"2024-02-18T11:38:49.856138+0000","last_undegraded":"2024-02-18T11:38:49.856138+0000","last_fullsized":"2024-02-18T11:38:49.856138+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:12:44.496950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:48.828176+0000","last_change":"2024-02-18T11:38:47.847872+0000","last_active":"2024-02-18T11:38:48.828176+0000","last_peered":"2024-02-18T11:38:48.828176+0000","last_clean":"2024-02-18T11:38:48.828176+0000","last_became_active":"2024-02-18T11:38:47.847537+0000","last_became_peered":"2024-02-18T11:38:47.847537+0000","last_unstale":"2024-02-18T11:38:48.828176+0000","last_undegraded":"2024-02-18T11:38:48.828176+0000","last_fullsized":"2024-02-18T11:38:48.828176+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T12:30:46.754937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-18T11:38:47.828657+0000","last_change":"2024-02-18T11:38:46.534677+0000","last_active":"2024-02-18T11:38:47.828657+0000","last_peered":"2024-02-18T11:38:47.828657+0000","last_clean":"2024-02-18T11:38:47.828657+0000","last_became_active":"2024-02-18T11:38:46.527789+0000","last_became_peered":"2024-02-18T11:38:46.527789+0000","last_unstale":"2024-02-18T11:38:47.828657+0000","last_undegraded":"2024-02-18T11:38:47.828657+0000","last_fullsized":"2024-02-18T11:38:47.828657+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:13:15.365557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:48.828621+0000","last_change":"2024-02-18T11:38:47.849874+0000","last_active":"2024-02-18T11:38:48.828621+0000","last_peered":"2024-02-18T11:38:48.828621+0000","last_clean":"2024-02-18T11:38:48.828621+0000","last_became_active":"2024-02-18T11:38:47.849318+0000","last_became_peered":"2024-02-18T11:38:47.849318+0000","last_unstale":"2024-02-18T11:38:48.828621+0000","last_undegraded":"2024-02-18T11:38:48.828621+0000","last_fullsized":"2024-02-18T11:38:48.828621+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:22:31.974649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'1","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.456337+0000","last_change":"2024-02-18T11:38:46.531691+0000","last_active":"2024-02-18T11:38:49.456337+0000","last_peered":"2024-02-18T11:38:49.456337+0000","last_clean":"2024-02-18T11:38:49.456337+0000","last_became_active":"2024-02-18T11:38:46.530987+0000","last_became_peered":"2024-02-18T11:38:46.530987+0000","last_unstale":"2024-02-18T11:38:49.456337+0000","last_undegraded":"2024-02-18T11:38:49.456337+0000","last_fullsized":"2024-02-18T11:38:49.456337+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:15:46.228599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:48.828161+0000","last_change":"2024-02-18T11:38:47.850273+0000","last_active":"2024-02-18T11:38:48.828161+0000","last_peered":"2024-02-18T11:38:48.828161+0000","last_clean":"2024-02-18T11:38:48.828161+0000","last_became_active":"2024-02-18T11:38:47.849110+0000","last_became_peered":"2024-02-18T11:38:47.849110+0000","last_unstale":"2024-02-18T11:38:48.828161+0000","last_undegraded":"2024-02-18T11:38:48.828161+0000","last_fullsized":"2024-02-18T11:38:48.828161+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T23:18:55.096768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.854461+0000","last_change":"2024-02-18T11:38:46.537641+0000","last_active":"2024-02-18T11:38:49.854461+0000","last_peered":"2024-02-18T11:38:49.854461+0000","last_clean":"2024-02-18T11:38:49.854461+0000","last_became_active":"2024-02-18T11:38:46.537311+0000","last_became_peered":"2024-02-18T11:38:46.537311+0000","last_unstale":"2024-02-18T11:38:49.854461+0000","last_undegraded":"2024-02-18T11:38:49.854461+0000","last_fullsized":"2024-02-18T11:38:49.854461+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:13:25.720250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:48.828659+0000","last_change":"2024-02-18T11:38:47.833177+0000","last_active":"2024-02-18T11:38:48.828659+0000","last_peered":"2024-02-18T11:38:48.828659+0000","last_clean":"2024-02-18T11:38:48.828659+0000","last_became_active":"2024-02-18T11:38:47.832839+0000","last_became_peered":"2024-02-18T11:38:47.832839+0000","last_unstale":"2024-02-18T11:38:48.828659+0000","last_undegraded":"2024-02-18T11:38:48.828659+0000","last_fullsized":"2024-02-18T11:38:48.828659+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:12:28.448505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.854545+0000","last_change":"2024-02-18T11:38:47.841062+0000","last_active":"2024-02-18T11:38:49.854545+0000","last_peered":"2024-02-18T11:38:49.854545+0000","last_clean":"2024-02-18T11:38:49.854545+0000","last_became_active":"2024-02-18T11:38:47.840730+0000","last_became_peered":"2024-02-18T11:38:47.840730+0000","last_unstale":"2024-02-18T11:38:49.854545+0000","last_undegraded":"2024-02-18T11:38:49.854545+0000","last_fullsized":"2024-02-18T11:38:49.854545+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:18:13.417676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-18T11:38:47.840327+0000","last_change":"2024-02-18T11:38:47.839488+0000","last_active":"2024-02-18T11:38:47.840327+0000","last_peered":"2024-02-18T11:38:47.840327+0000","last_clean":"2024-02-18T11:38:47.840327+0000","last_became_active":"2024-02-18T11:38:47.839134+0000","last_became_peered":"2024-02-18T11:38:47.839134+0000","last_unstale":"2024-02-18T11:38:47.840327+0000","last_undegraded":"2024-02-18T11:38:47.840327+0000","last_fullsized":"2024-02-18T11:38:47.840327+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T12:08:07.783369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-18T11:38:47.845944+0000","last_change":"2024-02-18T11:38:47.845397+0000","last_active":"2024-02-18T11:38:47.845944+0000","last_peered":"2024-02-18T11:38:47.845944+0000","last_clean":"2024-02-18T11:38:47.845944+0000","last_became_active":"2024-02-18T11:38:47.845027+0000","last_became_peered":"2024-02-18T11:38:47.845027+0000","last_unstale":"2024-02-18T11:38:47.845944+0000","last_undegraded":"2024-02-18T11:38:47.845944+0000","last_fullsized":"2024-02-18T11:38:47.845944+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T23:08:32.725571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:48.827311+0000","last_change":"2024-02-18T11:38:47.839238+0000","last_active":"2024-02-18T11:38:48.827311+0000","last_peered":"2024-02-18T11:38:48.827311+0000","last_clean":"2024-02-18T11:38:48.827311+0000","last_became_active":"2024-02-18T11:38:47.838928+0000","last_became_peered":"2024-02-18T11:38:47.838928+0000","last_unstale":"2024-02-18T11:38:48.827311+0000","last_undegraded":"2024-02-18T11:38:48.827311+0000","last_fullsized":"2024-02-18T11:38:48.827311+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:32:10.867251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.853471+0000","last_change":"2024-02-18T11:38:47.841524+0000","last_active":"2024-02-18T11:38:49.853471+0000","last_peered":"2024-02-18T11:38:49.853471+0000","last_clean":"2024-02-18T11:38:49.853471+0000","last_became_active":"2024-02-18T11:38:47.840807+0000","last_became_peered":"2024-02-18T11:38:47.840807+0000","last_unstale":"2024-02-18T11:38:49.853471+0000","last_undegraded":"2024-02-18T11:38:49.853471+0000","last_fullsized":"2024-02-18T11:38:49.853471+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T12:40:11.488587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.854812+0000","last_change":"2024-02-18T11:38:47.839821+0000","last_active":"2024-02-18T11:38:49.854812+0000","last_peered":"2024-02-18T11:38:49.854812+0000","last_clean":"2024-02-18T11:38:49.854812+0000","last_became_active":"2024-02-18T11:38:47.839431+0000","last_became_peered":"2024-02-18T11:38:47.839431+0000","last_unstale":"2024-02-18T11:38:49.854812+0000","last_undegraded":"2024-02-18T11:38:49.854812+0000","last_fullsized":"2024-02-18T11:38:49.854812+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:43:07.002223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-18T11:38:47.845005+0000","last_change":"2024-02-18T11:38:47.843937+0000","last_active":"2024-02-18T11:38:47.845005+0000","last_peered":"2024-02-18T11:38:47.845005+0000","last_clean":"2024-02-18T11:38:47.845005+0000","last_became_active":"2024-02-18T11:38:47.843603+0000","last_became_peered":"2024-02-18T11:38:47.843603+0000","last_unstale":"2024-02-18T11:38:47.845005+0000","last_undegraded":"2024-02-18T11:38:47.845005+0000","last_fullsized":"2024-02-18T11:38:47.845005+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:35:47.247763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-18T11:38:47.841492+0000","last_change":"2024-02-18T11:38:47.836702+0000","last_active":"2024-02-18T11:38:47.841492+0000","last_peered":"2024-02-18T11:38:47.841492+0000","last_clean":"2024-02-18T11:38:47.841492+0000","last_became_active":"2024-02-18T11:38:47.834843+0000","last_became_peered":"2024-02-18T11:38:47.834843+0000","last_unstale":"2024-02-18T11:38:47.841492+0000","last_undegraded":"2024-02-18T11:38:47.841492+0000","last_fullsized":"2024-02-18T11:38:47.841492+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:14:02.288109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.456116+0000","last_change":"2024-02-18T11:38:47.837429+0000","last_active":"2024-02-18T11:38:49.456116+0000","last_peered":"2024-02-18T11:38:49.456116+0000","last_clean":"2024-02-18T11:38:49.456116+0000","last_became_active":"2024-02-18T11:38:47.832983+0000","last_became_peered":"2024-02-18T11:38:47.832983+0000","last_unstale":"2024-02-18T11:38:49.456116+0000","last_undegraded":"2024-02-18T11:38:49.456116+0000","last_fullsized":"2024-02-18T11:38:49.456116+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:37:07.582626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-18T11:38:47.841620+0000","last_change":"2024-02-18T11:38:47.837041+0000","last_active":"2024-02-18T11:38:47.841620+0000","last_peered":"2024-02-18T11:38:47.841620+0000","last_clean":"2024-02-18T11:38:47.841620+0000","last_became_active":"2024-02-18T11:38:47.836730+0000","last_became_peered":"2024-02-18T11:38:47.836730+0000","last_unstale":"2024-02-18T11:38:47.841620+0000","last_undegraded":"2024-02-18T11:38:47.841620+0000","last_fullsized":"2024-02-18T11:38:47.841620+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:56:59.496016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.856047+0000","last_change":"2024-02-18T11:38:47.849186+0000","last_active":"2024-02-18T11:38:49.856047+0000","last_peered":"2024-02-18T11:38:49.856047+0000","last_clean":"2024-02-18T11:38:49.856047+0000","last_became_active":"2024-02-18T11:38:47.848741+0000","last_became_peered":"2024-02-18T11:38:47.848741+0000","last_unstale":"2024-02-18T11:38:49.856047+0000","last_undegraded":"2024-02-18T11:38:49.856047+0000","last_fullsized":"2024-02-18T11:38:49.856047+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:19:31.971674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.854044+0000","last_change":"2024-02-18T11:38:47.841231+0000","last_active":"2024-02-18T11:38:49.854044+0000","last_peered":"2024-02-18T11:38:49.854044+0000","last_clean":"2024-02-18T11:38:49.854044+0000","last_became_active":"2024-02-18T11:38:47.840860+0000","last_became_peered":"2024-02-18T11:38:47.840860+0000","last_unstale":"2024-02-18T11:38:49.854044+0000","last_undegraded":"2024-02-18T11:38:49.854044+0000","last_fullsized":"2024-02-18T11:38:49.854044+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:29:00.036507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T11:38:49.855113+0000","last_change":"2024-02-18T11:38:47.839901+0000","last_active":"2024-02-18T11:38:49.855113+0000","last_peered":"2024-02-18T11:38:49.855113+0000","last_clean":"2024-02-18T11:38:49.855113+0000","last_became_active":"2024-02-18T11:38:47.839532+0000","last_became_peered":"2024-02-18T11:38:47.839532+0000","last_unstale":"2024-02-18T11:38:49.855113+0000","last_undegraded":"2024-02-18T11:38:49.855113+0000","last_fullsized":"2024-02-18T11:38:49.855113+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T21:32:14.369846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4941,"num_objects":52,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":13,"num_read_kb":11,"num_write":110,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":217088,"data_stored":14561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318286,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":933888,"data_stored":445784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579922,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":41032,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704120,"statfs":{"total":95995035648,"available":95953018880,"internally_reserved":0,"allocated":1568768,"data_stored":1038760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":40434112},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743447,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":950272,"data_stored":446020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005084,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":942080,"data_stored":445828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266722,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":41000,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704152,"statfs":{"total":95995035648,"available":95953051648,"internally_reserved":0,"allocated":1540096,"data_stored":1037362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430247,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":933888,"data_stored":445292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574910,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":942080,"data_stored":446814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738436,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":958464,"data_stored":446680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411386,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":933888,"data_stored":438278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117045,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":925696,"data_stored":436736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":35,"seq":150323855409,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36256,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708896,"statfs":{"total":95995035648,"available":95957909504,"internally_reserved":0,"allocated":942080,"data_stored":445846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691884,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32692,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712460,"statfs":{"total":95995035648,"available":95961559040,"internally_reserved":0,"allocated":1552384,"data_stored":1037437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-18T11:38:54.668 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-02-18T11:38:54.668 INFO:tasks.cephadm:Setting up client nodes... 2024-02-18T11:38:54.668 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-02-18T11:38:54.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:54 smithi012 ceph-mon[32460]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:38:54.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:54 smithi012 ceph-mon[32460]: pgmap v288: 97 pgs: 97 active+clean; 581 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 5.7 KiB/s wr, 18 op/s 2024-02-18T11:38:54.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:54 smithi012 ceph-mon[32460]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:38:54.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:54 smithi012 ceph-mon[32460]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-02-18T11:38:54.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:54 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/3275697730' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-18T11:38:54.947 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T11:38:55.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:54 smithi158 ceph-mon[36626]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:38:55.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:54 smithi158 ceph-mon[36626]: pgmap v288: 97 pgs: 97 active+clean; 581 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 5.7 KiB/s wr, 18 op/s 2024-02-18T11:38:55.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:54 smithi158 ceph-mon[36626]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:38:55.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:54 smithi158 ceph-mon[36626]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-02-18T11:38:55.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:54 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/3275697730' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-18T11:38:55.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:54 smithi188 ceph-mon[36925]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:38:55.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:54 smithi188 ceph-mon[36925]: pgmap v288: 97 pgs: 97 active+clean; 581 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 5.7 KiB/s wr, 18 op/s 2024-02-18T11:38:55.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:54 smithi188 ceph-mon[36925]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:38:55.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:54 smithi188 ceph-mon[36925]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-02-18T11:38:55.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:54 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/3275697730' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-18T11:38:56.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:55 smithi012 ceph-mon[32460]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T11:38:56.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:55 smithi012 ceph-mon[32460]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T11:38:56.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:55 smithi158 ceph-mon[36626]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T11:38:56.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:55 smithi158 ceph-mon[36626]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T11:38:56.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:55 smithi188 ceph-mon[36925]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T11:38:56.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:55 smithi188 ceph-mon[36925]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T11:38:56.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:56.619+0000 7f8742327700 1 -- 172.21.15.12:0/4155374502 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f873c102f00 msgr2=0x7f873c103360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:56.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:56.619+0000 7f8742327700 1 --2- 172.21.15.12:0/4155374502 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f873c102f00 0x7f873c103360 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f87300099c0 tx=0x7f8730009cd0 comp rx=0 tx=0).stop 2024-02-18T11:38:56.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:56.619+0000 7f8742327700 1 -- 172.21.15.12:0/4155374502 shutdown_connections 2024-02-18T11:38:56.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:56.619+0000 7f8742327700 1 --2- 172.21.15.12:0/4155374502 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f873c1038a0 0x7f873c10a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:56.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:56.619+0000 7f8742327700 1 --2- 172.21.15.12:0/4155374502 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f873c101d00 0x7f873c102120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:56.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:56.619+0000 7f8742327700 1 --2- 172.21.15.12:0/4155374502 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f873c102f00 0x7f873c103360 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:56.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:56.619+0000 7f8742327700 1 -- 172.21.15.12:0/4155374502 >> 172.21.15.12:0/4155374502 conn(0x7f873c0fd2c0 msgr2=0x7f873c0ff6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:56.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:56.619+0000 7f8742327700 1 -- 172.21.15.12:0/4155374502 shutdown_connections 2024-02-18T11:38:56.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:56.619+0000 7f8742327700 1 -- 172.21.15.12:0/4155374502 wait complete. 2024-02-18T11:38:56.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:56.620+0000 7f8742327700 1 Processor -- start 2024-02-18T11:38:56.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:56.620+0000 7f8742327700 1 -- start start 2024-02-18T11:38:56.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:56.620+0000 7f8742327700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f873c101d00 0x7f873c197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:56.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:56.620+0000 7f8742327700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f873c102f00 0x7f873c198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:56.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:56.620+0000 7f8742327700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f873c1038a0 0x7f873c19f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:56.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:56.620+0000 7f8742327700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f873c19f770 con 0x7f873c101d00 2024-02-18T11:38:56.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:56.620+0000 7f8742327700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f873c19f8b0 con 0x7f873c102f00 2024-02-18T11:38:56.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:56.620+0000 7f8742327700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f873c19fa40 con 0x7f873c1038a0 2024-02-18T11:38:56.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:56.620+0000 7f873bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f873c101d00 0x7f873c197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:56.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:56.620+0000 7f873bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f873c101d00 0x7f873c197be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43752/0 (socket says 172.21.15.12:43752) 2024-02-18T11:38:56.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:56.620+0000 7f873bfff700 1 -- 172.21.15.12:0/863963714 learned_addr learned my addr 172.21.15.12:0/863963714 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:38:56.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:56.621+0000 7f87408c4700 1 --2- 172.21.15.12:0/863963714 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f873c1038a0 0x7f873c19f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:56.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:56.621+0000 7f873b7fe700 1 --2- 172.21.15.12:0/863963714 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f873c102f00 0x7f873c198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:56.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:56.621+0000 7f873bfff700 1 -- 172.21.15.12:0/863963714 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f873c1038a0 msgr2=0x7f873c19f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:56.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:56.621+0000 7f873bfff700 1 --2- 172.21.15.12:0/863963714 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f873c1038a0 0x7f873c19f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:56.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:56.621+0000 7f873bfff700 1 -- 172.21.15.12:0/863963714 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f873c102f00 msgr2=0x7f873c198120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:56.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:56.621+0000 7f873bfff700 1 --2- 172.21.15.12:0/863963714 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f873c102f00 0x7f873c198120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:56.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:56.621+0000 7f873bfff700 1 -- 172.21.15.12:0/863963714 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87300095f0 con 0x7f873c101d00 2024-02-18T11:38:56.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:56.621+0000 7f873bfff700 1 --2- 172.21.15.12:0/863963714 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f873c101d00 0x7f873c197be0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f872c00d9b0 tx=0x7f872c00dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:56.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:56.621+0000 7f87397fa700 1 -- 172.21.15.12:0/863963714 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872c00be80 con 0x7f873c101d00 2024-02-18T11:38:56.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:56.621+0000 7f8742327700 1 -- 172.21.15.12:0/863963714 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f873c19fd20 con 0x7f873c101d00 2024-02-18T11:38:56.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:56.621+0000 7f87397fa700 1 -- 172.21.15.12:0/863963714 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f872c004500 con 0x7f873c101d00 2024-02-18T11:38:56.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:56.621+0000 7f87397fa700 1 -- 172.21.15.12:0/863963714 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872c003be0 con 0x7f873c101d00 2024-02-18T11:38:56.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:56.621+0000 7f8742327700 1 -- 172.21.15.12:0/863963714 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f873c1a01c0 con 0x7f873c101d00 2024-02-18T11:38:56.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:56.622+0000 7f8742327700 1 -- 172.21.15.12:0/863963714 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f873c109060 con 0x7f873c101d00 2024-02-18T11:38:56.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:56.623+0000 7f87397fa700 1 -- 172.21.15.12:0/863963714 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 137296+0+0 (secure 0 0 0) 0x7f872c009c00 con 0x7f873c101d00 2024-02-18T11:38:56.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:56.624+0000 7f87397fa700 1 --2- 172.21.15.12:0/863963714 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f87240a2920 0x7f87240a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:56.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:56.624+0000 7f873b7fe700 1 --2- 172.21.15.12:0/863963714 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f87240a2920 0x7f87240a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:56.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:56.624+0000 7f87397fa700 1 -- 172.21.15.12:0/863963714 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f872c0ce320 con 0x7f873c101d00 2024-02-18T11:38:56.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:56.624+0000 7f873b7fe700 1 --2- 172.21.15.12:0/863963714 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f87240a2920 0x7f87240a4de0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f87300038c0 tx=0x7f8730000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:56.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:56.628+0000 7f87397fa700 1 -- 172.21.15.12:0/863963714 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f872c097000 con 0x7f873c101d00 2024-02-18T11:38:56.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:56.916+0000 7f8742327700 1 -- 172.21.15.12:0/863963714 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f873c10d6e0 con 0x7f873c101d00 2024-02-18T11:38:56.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:56.918+0000 7f87397fa700 1 -- 172.21.15.12:0/863963714 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) v1 ==== 170+0+59 (secure 0 0 0) 0x7f872c09a6c0 con 0x7f873c101d00 2024-02-18T11:38:56.912 INFO:teuthology.orchestra.run.smithi012.stdout:[client.0] 2024-02-18T11:38:56.912 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQBQ7NFliBqtNhAA5eAiA92vSRBX29hZpQvR1Q== 2024-02-18T11:38:56.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:56.920+0000 7f8742327700 1 -- 172.21.15.12:0/863963714 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f87240a2920 msgr2=0x7f87240a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:56.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:56.920+0000 7f8742327700 1 --2- 172.21.15.12:0/863963714 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f87240a2920 0x7f87240a4de0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f87300038c0 tx=0x7f8730000bc0 comp rx=0 tx=0).stop 2024-02-18T11:38:56.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:56.920+0000 7f8742327700 1 -- 172.21.15.12:0/863963714 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f873c101d00 msgr2=0x7f873c197be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:56.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:56.920+0000 7f8742327700 1 --2- 172.21.15.12:0/863963714 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f873c101d00 0x7f873c197be0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f872c00d9b0 tx=0x7f872c00dd70 comp rx=0 tx=0).stop 2024-02-18T11:38:56.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:56.920+0000 7f8742327700 1 -- 172.21.15.12:0/863963714 shutdown_connections 2024-02-18T11:38:56.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:56.920+0000 7f8742327700 1 --2- 172.21.15.12:0/863963714 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f873c1038a0 0x7f873c19f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:56.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:56.920+0000 7f8742327700 1 --2- 172.21.15.12:0/863963714 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f87240a2920 0x7f87240a4de0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:56.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:56.920+0000 7f8742327700 1 --2- 172.21.15.12:0/863963714 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f873c101d00 0x7f873c197be0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:56.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:56.920+0000 7f8742327700 1 --2- 172.21.15.12:0/863963714 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f873c102f00 0x7f873c198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:56.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:56.920+0000 7f8742327700 1 -- 172.21.15.12:0/863963714 >> 172.21.15.12:0/863963714 conn(0x7f873c0fd2c0 msgr2=0x7f873c1061a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:56.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:56.920+0000 7f8742327700 1 -- 172.21.15.12:0/863963714 shutdown_connections 2024-02-18T11:38:56.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:56.920+0000 7f8742327700 1 -- 172.21.15.12:0/863963714 wait complete. 2024-02-18T11:38:57.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:56 smithi012 ceph-mon[32460]: pgmap v289: 97 pgs: 97 active+clean; 582 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 4.9 KiB/s wr, 17 op/s 2024-02-18T11:38:57.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:56 smithi158 ceph-mon[36626]: pgmap v289: 97 pgs: 97 active+clean; 582 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 4.9 KiB/s wr, 17 op/s 2024-02-18T11:38:57.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:56 smithi188 ceph-mon[36925]: pgmap v289: 97 pgs: 97 active+clean; 582 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 4.9 KiB/s wr, 17 op/s 2024-02-18T11:38:57.530 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-02-18T11:38:57.530 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-02-18T11:38:57.530 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-02-18T11:38:57.574 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-02-18T11:38:57.574 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-02-18T11:38:57.574 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph mgr dump --format=json 2024-02-18T11:38:57.854 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T11:38:58.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:57 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/863963714' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-18T11:38:58.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:57 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/863963714' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-18T11:38:58.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:57 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/863963714' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-18T11:38:58.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:57 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/863963714' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-18T11:38:58.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:57 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/863963714' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-18T11:38:58.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:57 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/863963714' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-18T11:38:59.127 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:58 smithi012 ceph-mon[32460]: pgmap v290: 97 pgs: 97 active+clean; 618 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 8.9 KiB/s wr, 20 op/s 2024-02-18T11:38:59.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:58 smithi158 ceph-mon[36626]: pgmap v290: 97 pgs: 97 active+clean; 618 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 8.9 KiB/s wr, 20 op/s 2024-02-18T11:38:59.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:59.353+0000 7f1043b51700 1 -- 172.21.15.12:0/2902817534 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f103c100bb0 msgr2=0x7f103c101010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:59.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:59.353+0000 7f1043b51700 1 --2- 172.21.15.12:0/2902817534 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f103c100bb0 0x7f103c101010 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f102c009990 tx=0x7f102c009ca0 comp rx=0 tx=0).stop 2024-02-18T11:38:59.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:59.353+0000 7f1043b51700 1 -- 172.21.15.12:0/2902817534 shutdown_connections 2024-02-18T11:38:59.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:59.353+0000 7f1043b51700 1 --2- 172.21.15.12:0/2902817534 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f103c101550 0x7f103c10da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:59.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:59.353+0000 7f1043b51700 1 --2- 172.21.15.12:0/2902817534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f103c106bf0 0x7f103c106fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:59.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:59.353+0000 7f1043b51700 1 --2- 172.21.15.12:0/2902817534 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f103c100bb0 0x7f103c101010 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:59.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:59.353+0000 7f1043b51700 1 -- 172.21.15.12:0/2902817534 >> 172.21.15.12:0/2902817534 conn(0x7f103c077ae0 msgr2=0x7f103c077ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:59.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:59.353+0000 7f1043b51700 1 -- 172.21.15.12:0/2902817534 shutdown_connections 2024-02-18T11:38:59.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:59.353+0000 7f1043b51700 1 -- 172.21.15.12:0/2902817534 wait complete. 2024-02-18T11:38:59.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:59.354+0000 7f1043b51700 1 Processor -- start 2024-02-18T11:38:59.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:59.354+0000 7f1043b51700 1 -- start start 2024-02-18T11:38:59.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:59.354+0000 7f1043b51700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f103c100bb0 0x7f103c196e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:59.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:59.354+0000 7f1043b51700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f103c101550 0x7f103c197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:59.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:59.354+0000 7f1043b51700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f103c106bf0 0x7f103c197890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:59.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:59.354+0000 7f1043b51700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f103c1a6550 con 0x7f103c106bf0 2024-02-18T11:38:59.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:59.354+0000 7f1043b51700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f103c1a66c0 con 0x7f103c101550 2024-02-18T11:38:59.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:59.354+0000 7f1043b51700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f103c1a6880 con 0x7f103c100bb0 2024-02-18T11:38:59.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:59.354+0000 7f10420ee700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f103c106bf0 0x7f103c197890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:59.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:59.354+0000 7f10420ee700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f103c106bf0 0x7f103c197890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43778/0 (socket says 172.21.15.12:43778) 2024-02-18T11:38:59.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:59.354+0000 7f10418ed700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f103c100bb0 0x7f103c196e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:59.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:59.354+0000 7f10420ee700 1 -- 172.21.15.12:0/3211365520 learned_addr learned my addr 172.21.15.12:0/3211365520 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:38:59.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:59.354+0000 7f10410ec700 1 --2- 172.21.15.12:0/3211365520 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f103c101550 0x7f103c197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:59.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:59.354+0000 7f10420ee700 1 -- 172.21.15.12:0/3211365520 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f103c100bb0 msgr2=0x7f103c196e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:59.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:59.354+0000 7f10420ee700 1 --2- 172.21.15.12:0/3211365520 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f103c100bb0 0x7f103c196e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:59.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:59.354+0000 7f10420ee700 1 -- 172.21.15.12:0/3211365520 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f103c101550 msgr2=0x7f103c197350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:59.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:59.355+0000 7f10420ee700 1 --2- 172.21.15.12:0/3211365520 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f103c101550 0x7f103c197350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:59.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:59.355+0000 7f10420ee700 1 -- 172.21.15.12:0/3211365520 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f102c0095f0 con 0x7f103c106bf0 2024-02-18T11:38:59.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:59.355+0000 7f10410ec700 1 --2- 172.21.15.12:0/3211365520 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f103c101550 0x7f103c197350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:38:59.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:59.355+0000 7f10420ee700 1 --2- 172.21.15.12:0/3211365520 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f103c106bf0 0x7f103c197890 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1034007d70 tx=0x7f103400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:59.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:59.355+0000 7f1032ffd700 1 -- 172.21.15.12:0/3211365520 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f103400ad40 con 0x7f103c106bf0 2024-02-18T11:38:59.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:59.355+0000 7f1043b51700 1 -- 172.21.15.12:0/3211365520 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f103c1a6b60 con 0x7f103c106bf0 2024-02-18T11:38:59.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:59.355+0000 7f1032ffd700 1 -- 172.21.15.12:0/3211365520 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1034012900 con 0x7f103c106bf0 2024-02-18T11:38:59.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:59.355+0000 7f1043b51700 1 -- 172.21.15.12:0/3211365520 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f103c1a7060 con 0x7f103c106bf0 2024-02-18T11:38:59.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:59.355+0000 7f1032ffd700 1 -- 172.21.15.12:0/3211365520 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1034011780 con 0x7f103c106bf0 2024-02-18T11:38:59.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:59.356+0000 7f1043b51700 1 -- 172.21.15.12:0/3211365520 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f103c109110 con 0x7f103c106bf0 2024-02-18T11:38:59.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:59.357+0000 7f1032ffd700 1 -- 172.21.15.12:0/3211365520 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 137296+0+0 (secure 0 0 0) 0x7f1034012a70 con 0x7f103c106bf0 2024-02-18T11:38:59.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:59.357+0000 7f1032ffd700 1 --2- 172.21.15.12:0/3211365520 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f10280a2870 0x7f10280a4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:38:59.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:59.358+0000 7f10418ed700 1 --2- 172.21.15.12:0/3211365520 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f10280a2870 0x7f10280a4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:38:59.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:59.358+0000 7f1032ffd700 1 -- 172.21.15.12:0/3211365520 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f10340ce780 con 0x7f103c106bf0 2024-02-18T11:38:59.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:59.358+0000 7f10418ed700 1 --2- 172.21.15.12:0/3211365520 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f10280a2870 0x7f10280a4d30 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f10380045b0 tx=0x7f103800b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:38:59.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:58 smithi188 ceph-mon[36925]: pgmap v290: 97 pgs: 97 active+clean; 618 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 8.9 KiB/s wr, 20 op/s 2024-02-18T11:38:59.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:59.362+0000 7f1032ffd700 1 -- 172.21.15.12:0/3211365520 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1034097570 con 0x7f103c106bf0 2024-02-18T11:38:59.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:59.618+0000 7f1043b51700 1 -- 172.21.15.12:0/3211365520 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f103c110c40 con 0x7f103c106bf0 2024-02-18T11:38:59.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:59.623+0000 7f1032ffd700 1 -- 172.21.15.12:0/3211365520 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+262291 (secure 0 0 0) 0x7f103409ac30 con 0x7f103c106bf0 2024-02-18T11:38:59.622 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:38:59.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:59.626+0000 7f1043b51700 1 -- 172.21.15.12:0/3211365520 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f10280a2870 msgr2=0x7f10280a4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:59.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:59.626+0000 7f1043b51700 1 --2- 172.21.15.12:0/3211365520 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f10280a2870 0x7f10280a4d30 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f10380045b0 tx=0x7f103800b3c0 comp rx=0 tx=0).stop 2024-02-18T11:38:59.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:59.626+0000 7f1043b51700 1 -- 172.21.15.12:0/3211365520 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f103c106bf0 msgr2=0x7f103c197890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:38:59.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:59.626+0000 7f1043b51700 1 --2- 172.21.15.12:0/3211365520 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f103c106bf0 0x7f103c197890 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1034007d70 tx=0x7f103400a340 comp rx=0 tx=0).stop 2024-02-18T11:38:59.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:59.626+0000 7f1043b51700 1 -- 172.21.15.12:0/3211365520 shutdown_connections 2024-02-18T11:38:59.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:59.626+0000 7f1043b51700 1 --2- 172.21.15.12:0/3211365520 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f10280a2870 0x7f10280a4d30 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:59.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:59.626+0000 7f1043b51700 1 --2- 172.21.15.12:0/3211365520 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f103c106bf0 0x7f103c197890 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:59.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:59.626+0000 7f1043b51700 1 --2- 172.21.15.12:0/3211365520 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f103c100bb0 0x7f103c196e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:59.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:59.626+0000 7f1043b51700 1 --2- 172.21.15.12:0/3211365520 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f103c101550 0x7f103c197350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:38:59.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:59.626+0000 7f1043b51700 1 -- 172.21.15.12:0/3211365520 >> 172.21.15.12:0/3211365520 conn(0x7f103c077ae0 msgr2=0x7f103c0ff7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:38:59.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:59.626+0000 7f1043b51700 1 -- 172.21.15.12:0/3211365520 shutdown_connections 2024-02-18T11:38:59.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:38:59.626+0000 7f1043b51700 1 -- 172.21.15.12:0/3211365520 wait complete. 2024-02-18T11:39:00.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:38:59 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/3211365520' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-18T11:39:00.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:38:59 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/3211365520' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-18T11:39:00.280 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":1153736685},{"type":"v1","addr":"172.21.15.12:6801","nonce":1153736685}]},"active_addr":"172.21.15.12:6801/1153736685","active_change":"2024-02-18T11:31:50.937042+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24106,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24112,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.12:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.12:0","nonce":3660507355}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.12:0","nonce":3219103450}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.12:0","nonce":3670903020}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.12:0","nonce":3429222323}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.12:0","nonce":457739848}]}]} 2024-02-18T11:39:00.286 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-02-18T11:39:00.344 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-02-18T11:39:00.345 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph osd dump --format=json 2024-02-18T11:39:00.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:38:59 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/3211365520' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-18T11:39:00.593 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T11:39:01.053 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:39:00 smithi012 ceph-mon[32460]: pgmap v291: 97 pgs: 97 active+clean; 618 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 8.0 KiB/s wr, 19 op/s 2024-02-18T11:39:01.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:39:00 smithi158 ceph-mon[36626]: pgmap v291: 97 pgs: 97 active+clean; 618 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 8.0 KiB/s wr, 19 op/s 2024-02-18T11:39:01.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:39:00 smithi188 ceph-mon[36925]: pgmap v291: 97 pgs: 97 active+clean; 618 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 8.0 KiB/s wr, 19 op/s 2024-02-18T11:39:02.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:02.236+0000 7f7005373700 1 -- 172.21.15.12:0/646467724 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7000107ce0 msgr2=0x7f70001080c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:02.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:02.236+0000 7f7005373700 1 --2- 172.21.15.12:0/646467724 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7000107ce0 0x7f70001080c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6ff800b0d0 tx=0x7f6ff800b3e0 comp rx=0 tx=0).stop 2024-02-18T11:39:02.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:02.236+0000 7f7005373700 1 -- 172.21.15.12:0/646467724 shutdown_connections 2024-02-18T11:39:02.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:02.236+0000 7f7005373700 1 --2- 172.21.15.12:0/646467724 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70001026c0 0x7f700010eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:02.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:02.236+0000 7f7005373700 1 --2- 172.21.15.12:0/646467724 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7000107ce0 0x7f70001080c0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:02.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:02.236+0000 7f7005373700 1 --2- 172.21.15.12:0/646467724 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7000101d00 0x7f7000102180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:02.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:02.236+0000 7f7005373700 1 -- 172.21.15.12:0/646467724 >> 172.21.15.12:0/646467724 conn(0x7f7000078de0 msgr2=0x7f70000791f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:39:02.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:02.236+0000 7f7005373700 1 -- 172.21.15.12:0/646467724 shutdown_connections 2024-02-18T11:39:02.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:02.236+0000 7f7005373700 1 -- 172.21.15.12:0/646467724 wait complete. 2024-02-18T11:39:02.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:02.237+0000 7f7005373700 1 Processor -- start 2024-02-18T11:39:02.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:02.237+0000 7f7005373700 1 -- start start 2024-02-18T11:39:02.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:02.237+0000 7f7005373700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7000101d00 0x7f7000198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:02.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:02.237+0000 7f7005373700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70001026c0 0x7f70001987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:02.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:02.237+0000 7f7005373700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7000107ce0 0x7f700019cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:02.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:02.237+0000 7f7005373700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700019d170 con 0x7f7000101d00 2024-02-18T11:39:02.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:02.237+0000 7f7005373700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700019d2e0 con 0x7f7000107ce0 2024-02-18T11:39:02.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:02.237+0000 7f7005373700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700019d4a0 con 0x7f70001026c0 2024-02-18T11:39:02.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:02.238+0000 7f6ffeffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7000101d00 0x7f7000198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:02.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:02.238+0000 7f6ffeffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7000101d00 0x7f7000198290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50304/0 (socket says 172.21.15.12:50304) 2024-02-18T11:39:02.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:02.238+0000 7f6ffeffd700 1 -- 172.21.15.12:0/700247632 learned_addr learned my addr 172.21.15.12:0/700247632 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:39:02.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:02.238+0000 7f6ffe7fc700 1 --2- 172.21.15.12:0/700247632 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70001026c0 0x7f70001987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:02.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:02.238+0000 7f6fff7fe700 1 --2- 172.21.15.12:0/700247632 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7000107ce0 0x7f700019cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:02.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:02.238+0000 7f6ffeffd700 1 -- 172.21.15.12:0/700247632 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70001026c0 msgr2=0x7f70001987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:02.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:02.238+0000 7f6ffeffd700 1 --2- 172.21.15.12:0/700247632 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70001026c0 0x7f70001987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:02.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:02.238+0000 7f6ffeffd700 1 -- 172.21.15.12:0/700247632 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7000107ce0 msgr2=0x7f700019cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:02.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:02.238+0000 7f6ffeffd700 1 --2- 172.21.15.12:0/700247632 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7000107ce0 0x7f700019cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:02.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:02.238+0000 7f6ffeffd700 1 -- 172.21.15.12:0/700247632 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ff8009d00 con 0x7f7000101d00 2024-02-18T11:39:02.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:02.238+0000 7f6ffe7fc700 1 --2- 172.21.15.12:0/700247632 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70001026c0 0x7f70001987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:39:02.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:02.238+0000 7f6ffeffd700 1 --2- 172.21.15.12:0/700247632 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7000101d00 0x7f7000198290 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6ff8007870 tx=0x7f6ff8007950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:39:02.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:02.238+0000 7f6ff7fff700 1 -- 172.21.15.12:0/700247632 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff800e040 con 0x7f7000101d00 2024-02-18T11:39:02.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:02.238+0000 7f7005373700 1 -- 172.21.15.12:0/700247632 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f700019d720 con 0x7f7000101d00 2024-02-18T11:39:02.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:02.238+0000 7f6ff7fff700 1 -- 172.21.15.12:0/700247632 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6ff8003e10 con 0x7f7000101d00 2024-02-18T11:39:02.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:02.238+0000 7f6ff7fff700 1 -- 172.21.15.12:0/700247632 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff801bb40 con 0x7f7000101d00 2024-02-18T11:39:02.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:02.238+0000 7f7005373700 1 -- 172.21.15.12:0/700247632 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70001a4fe0 con 0x7f7000101d00 2024-02-18T11:39:02.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:02.239+0000 7f7005373700 1 -- 172.21.15.12:0/700247632 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f700010a1c0 con 0x7f7000101d00 2024-02-18T11:39:02.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:02.240+0000 7f6ff7fff700 1 -- 172.21.15.12:0/700247632 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 137296+0+0 (secure 0 0 0) 0x7f6ff8012470 con 0x7f7000101d00 2024-02-18T11:39:02.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:02.241+0000 7f6ff7fff700 1 --2- 172.21.15.12:0/700247632 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f6fec0a2870 0x7f6fec0a4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:02.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:02.241+0000 7f6ff7fff700 1 -- 172.21.15.12:0/700247632 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f6ff80cfb10 con 0x7f7000101d00 2024-02-18T11:39:02.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:02.241+0000 7f6ffe7fc700 1 --2- 172.21.15.12:0/700247632 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f6fec0a2870 0x7f6fec0a4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:02.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:02.242+0000 7f6ffe7fc700 1 --2- 172.21.15.12:0/700247632 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f6fec0a2870 0x7f6fec0a4d30 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6fe8005950 tx=0x7f6fe80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:39:02.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:02.246+0000 7f6ff7fff700 1 -- 172.21.15.12:0/700247632 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6ff8098980 con 0x7f7000101d00 2024-02-18T11:39:02.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:02.450+0000 7f7005373700 1 -- 172.21.15.12:0/700247632 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7000103390 con 0x7f7000101d00 2024-02-18T11:39:02.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:02.451+0000 7f6ff7fff700 1 -- 172.21.15.12:0/700247632 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22229 (secure 0 0 0) 0x7f6ff809c040 con 0x7f7000101d00 2024-02-18T11:39:02.446 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:39:02.446 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":87,"fsid":"0c8bb0de-ce51-11ee-95ba-87774f69a715","created":"2024-02-18T11:30:49.110059+0000","modified":"2024-02-18T11:38:48.817117+0000","last_up_change":"2024-02-18T11:37:57.512337+0000","last_in_change":"2024-02-18T11:37:36.567969+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-18T11:34:15.281138+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-18T11:38:44.802134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-18T11:38:46.081188+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"35f15ea6-deb8-48a1-a7dd-ab649ec13533","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":536522426},{"type":"v1","addr":"172.21.15.12:6803","nonce":536522426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":536522426},{"type":"v1","addr":"172.21.15.12:6805","nonce":536522426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":536522426},{"type":"v1","addr":"172.21.15.12:6809","nonce":536522426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":536522426},{"type":"v1","addr":"172.21.15.12:6807","nonce":536522426}]},"public_addr":"172.21.15.12:6803/536522426","cluster_addr":"172.21.15.12:6805/536522426","heartbeat_back_addr":"172.21.15.12:6809/536522426","heartbeat_front_addr":"172.21.15.12:6807/536522426","state":["exists","up"]},{"osd":1,"uuid":"2af937ef-d827-44e6-b805-70cffbfe1126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":4249111488},{"type":"v1","addr":"172.21.15.158:6801","nonce":4249111488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":4249111488},{"type":"v1","addr":"172.21.15.158:6803","nonce":4249111488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":4249111488},{"type":"v1","addr":"172.21.15.158:6807","nonce":4249111488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":4249111488},{"type":"v1","addr":"172.21.15.158:6805","nonce":4249111488}]},"public_addr":"172.21.15.158:6801/4249111488","cluster_addr":"172.21.15.158:6803/4249111488","heartbeat_back_addr":"172.21.15.158:6807/4249111488","heartbeat_front_addr":"172.21.15.158:6805/4249111488","state":["exists","up"]},{"osd":2,"uuid":"17b804dd-31df-45ae-a637-4a5f65bbaf26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":627976150},{"type":"v1","addr":"172.21.15.188:6801","nonce":627976150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":627976150},{"type":"v1","addr":"172.21.15.188:6803","nonce":627976150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":627976150},{"type":"v1","addr":"172.21.15.188:6807","nonce":627976150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":627976150},{"type":"v1","addr":"172.21.15.188:6805","nonce":627976150}]},"public_addr":"172.21.15.188:6801/627976150","cluster_addr":"172.21.15.188:6803/627976150","heartbeat_back_addr":"172.21.15.188:6807/627976150","heartbeat_front_addr":"172.21.15.188:6805/627976150","state":["exists","up"]},{"osd":3,"uuid":"3f9cc423-cdfa-4acf-aaa2-b58b32ccb9cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":3758512503},{"type":"v1","addr":"172.21.15.12:6811","nonce":3758512503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":3758512503},{"type":"v1","addr":"172.21.15.12:6813","nonce":3758512503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3758512503},{"type":"v1","addr":"172.21.15.12:6817","nonce":3758512503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":3758512503},{"type":"v1","addr":"172.21.15.12:6815","nonce":3758512503}]},"public_addr":"172.21.15.12:6811/3758512503","cluster_addr":"172.21.15.12:6813/3758512503","heartbeat_back_addr":"172.21.15.12:6817/3758512503","heartbeat_front_addr":"172.21.15.12:6815/3758512503","state":["exists","up"]},{"osd":4,"uuid":"a92e20ec-af1c-4ef4-a0cb-caba5fcc71aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":3892820997},{"type":"v1","addr":"172.21.15.158:6809","nonce":3892820997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":3892820997},{"type":"v1","addr":"172.21.15.158:6811","nonce":3892820997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":3892820997},{"type":"v1","addr":"172.21.15.158:6815","nonce":3892820997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":3892820997},{"type":"v1","addr":"172.21.15.158:6813","nonce":3892820997}]},"public_addr":"172.21.15.158:6809/3892820997","cluster_addr":"172.21.15.158:6811/3892820997","heartbeat_back_addr":"172.21.15.158:6815/3892820997","heartbeat_front_addr":"172.21.15.158:6813/3892820997","state":["exists","up"]},{"osd":5,"uuid":"d9c2e0c8-85f5-4ae2-bf3e-125c1be3a2db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3939495896},{"type":"v1","addr":"172.21.15.188:6809","nonce":3939495896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3939495896},{"type":"v1","addr":"172.21.15.188:6811","nonce":3939495896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3939495896},{"type":"v1","addr":"172.21.15.188:6815","nonce":3939495896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3939495896},{"type":"v1","addr":"172.21.15.188:6813","nonce":3939495896}]},"public_addr":"172.21.15.188:6809/3939495896","cluster_addr":"172.21.15.188:6811/3939495896","heartbeat_back_addr":"172.21.15.188:6815/3939495896","heartbeat_front_addr":"172.21.15.188:6813/3939495896","state":["exists","up"]},{"osd":6,"uuid":"4a523512-de6b-49f0-be81-00f2f00c1c10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":1206337767},{"type":"v1","addr":"172.21.15.12:6819","nonce":1206337767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":1206337767},{"type":"v1","addr":"172.21.15.12:6821","nonce":1206337767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":1206337767},{"type":"v1","addr":"172.21.15.12:6825","nonce":1206337767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":1206337767},{"type":"v1","addr":"172.21.15.12:6823","nonce":1206337767}]},"public_addr":"172.21.15.12:6819/1206337767","cluster_addr":"172.21.15.12:6821/1206337767","heartbeat_back_addr":"172.21.15.12:6825/1206337767","heartbeat_front_addr":"172.21.15.12:6823/1206337767","state":["exists","up"]},{"osd":7,"uuid":"363271a6-df93-44c9-b0e5-3d29768aa96c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":3283400032},{"type":"v1","addr":"172.21.15.158:6817","nonce":3283400032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":3283400032},{"type":"v1","addr":"172.21.15.158:6819","nonce":3283400032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":3283400032},{"type":"v1","addr":"172.21.15.158:6823","nonce":3283400032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":3283400032},{"type":"v1","addr":"172.21.15.158:6821","nonce":3283400032}]},"public_addr":"172.21.15.158:6817/3283400032","cluster_addr":"172.21.15.158:6819/3283400032","heartbeat_back_addr":"172.21.15.158:6823/3283400032","heartbeat_front_addr":"172.21.15.158:6821/3283400032","state":["exists","up"]},{"osd":8,"uuid":"9a089528-3e8e-45d3-9279-ddb54088b96a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2110170682},{"type":"v1","addr":"172.21.15.188:6817","nonce":2110170682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2110170682},{"type":"v1","addr":"172.21.15.188:6819","nonce":2110170682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":2110170682},{"type":"v1","addr":"172.21.15.188:6823","nonce":2110170682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2110170682},{"type":"v1","addr":"172.21.15.188:6821","nonce":2110170682}]},"public_addr":"172.21.15.188:6817/2110170682","cluster_addr":"172.21.15.188:6819/2110170682","heartbeat_back_addr":"172.21.15.188:6823/2110170682","heartbeat_front_addr":"172.21.15.188:6821/2110170682","state":["exists","up"]},{"osd":9,"uuid":"b12e2727-d649-4464-90e1-fafbc1754832","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1825928986},{"type":"v1","addr":"172.21.15.12:6827","nonce":1825928986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1825928986},{"type":"v1","addr":"172.21.15.12:6829","nonce":1825928986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":1825928986},{"type":"v1","addr":"172.21.15.12:6833","nonce":1825928986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1825928986},{"type":"v1","addr":"172.21.15.12:6831","nonce":1825928986}]},"public_addr":"172.21.15.12:6827/1825928986","cluster_addr":"172.21.15.12:6829/1825928986","heartbeat_back_addr":"172.21.15.12:6833/1825928986","heartbeat_front_addr":"172.21.15.12:6831/1825928986","state":["exists","up"]},{"osd":10,"uuid":"518b4cbf-9d90-4e9a-8ac9-b5cb9b3dee4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":3239052167},{"type":"v1","addr":"172.21.15.158:6825","nonce":3239052167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":3239052167},{"type":"v1","addr":"172.21.15.158:6827","nonce":3239052167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":3239052167},{"type":"v1","addr":"172.21.15.158:6831","nonce":3239052167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":3239052167},{"type":"v1","addr":"172.21.15.158:6829","nonce":3239052167}]},"public_addr":"172.21.15.158:6825/3239052167","cluster_addr":"172.21.15.158:6827/3239052167","heartbeat_back_addr":"172.21.15.158:6831/3239052167","heartbeat_front_addr":"172.21.15.158:6829/3239052167","state":["exists","up"]},{"osd":11,"uuid":"98aa3a0d-4b2f-46c8-9341-375d13794c3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1135773350},{"type":"v1","addr":"172.21.15.188:6825","nonce":1135773350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1135773350},{"type":"v1","addr":"172.21.15.188:6827","nonce":1135773350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1135773350},{"type":"v1","addr":"172.21.15.188:6831","nonce":1135773350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1135773350},{"type":"v1","addr":"172.21.15.188:6829","nonce":1135773350}]},"public_addr":"172.21.15.188:6825/1135773350","cluster_addr":"172.21.15.188:6827/1135773350","heartbeat_back_addr":"172.21.15.188:6831/1135773350","heartbeat_front_addr":"172.21.15.188:6829/1135773350","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:33:24.280879+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:33:47.859461+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:34:11.725283+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:34:35.707629+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:35:00.024197+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:35:24.499803+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:35:48.953870+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:36:13.642456+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:36:38.936457+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:37:04.021466+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:37:29.073178+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:37:54.863919+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:6801/4135274411":"2024-02-19T11:31:50.936928+0000","172.21.15.12:6800/4135274411":"2024-02-19T11:31:50.936928+0000","172.21.15.12:0/118428584":"2024-02-19T11:31:50.936928+0000","172.21.15.12:0/4276810915":"2024-02-19T11:31:50.936928+0000","172.21.15.12:0/3777545262":"2024-02-19T11:31:24.000830+0000","172.21.15.12:0/114707743":"2024-02-19T11:31:24.000830+0000","172.21.15.12:6800/3925204723":"2024-02-19T11:31:24.000830+0000","172.21.15.12:6801/3925204723":"2024-02-19T11:31:24.000830+0000","172.21.15.12:0/74881618":"2024-02-19T11:31:24.000830+0000","172.21.15.12:0/1614805375":"2024-02-19T11:31:50.936928+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-18T11:39:02.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:02.453+0000 7f7005373700 1 -- 172.21.15.12:0/700247632 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f6fec0a2870 msgr2=0x7f6fec0a4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:02.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:02.453+0000 7f7005373700 1 --2- 172.21.15.12:0/700247632 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f6fec0a2870 0x7f6fec0a4d30 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6fe8005950 tx=0x7f6fe80058e0 comp rx=0 tx=0).stop 2024-02-18T11:39:02.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:02.453+0000 7f7005373700 1 -- 172.21.15.12:0/700247632 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7000101d00 msgr2=0x7f7000198290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:02.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:02.453+0000 7f7005373700 1 --2- 172.21.15.12:0/700247632 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7000101d00 0x7f7000198290 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6ff8007870 tx=0x7f6ff8007950 comp rx=0 tx=0).stop 2024-02-18T11:39:02.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:02.453+0000 7f7005373700 1 -- 172.21.15.12:0/700247632 shutdown_connections 2024-02-18T11:39:02.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:02.453+0000 7f7005373700 1 --2- 172.21.15.12:0/700247632 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7000107ce0 0x7f700019cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:02.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:02.453+0000 7f7005373700 1 --2- 172.21.15.12:0/700247632 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f6fec0a2870 0x7f6fec0a4d30 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:02.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:02.453+0000 7f7005373700 1 --2- 172.21.15.12:0/700247632 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7000101d00 0x7f7000198290 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:02.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:02.453+0000 7f7005373700 1 --2- 172.21.15.12:0/700247632 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70001026c0 0x7f70001987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:02.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:02.453+0000 7f7005373700 1 -- 172.21.15.12:0/700247632 >> 172.21.15.12:0/700247632 conn(0x7f7000078de0 msgr2=0x7f70001008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:39:02.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:02.454+0000 7f7005373700 1 -- 172.21.15.12:0/700247632 shutdown_connections 2024-02-18T11:39:02.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:02.454+0000 7f7005373700 1 -- 172.21.15.12:0/700247632 wait complete. 2024-02-18T11:39:02.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:39:02 smithi012 ceph-mon[32460]: pgmap v292: 97 pgs: 97 active+clean; 627 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.8 KiB/s wr, 18 op/s 2024-02-18T11:39:02.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:39:02 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/700247632' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-18T11:39:03.073 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-02-18T11:39:03.074 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph osd dump --format=json 2024-02-18T11:39:03.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:39:02 smithi158 ceph-mon[36626]: pgmap v292: 97 pgs: 97 active+clean; 627 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.8 KiB/s wr, 18 op/s 2024-02-18T11:39:03.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:39:02 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/700247632' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-18T11:39:03.318 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T11:39:03.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:39:02 smithi188 ceph-mon[36925]: pgmap v292: 97 pgs: 97 active+clean; 627 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.8 KiB/s wr, 18 op/s 2024-02-18T11:39:03.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:39:02 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/700247632' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-18T11:39:04.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:04.925+0000 7fce18241700 1 -- 172.21.15.12:0/3811238574 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fce10106c20 msgr2=0x7fce10107000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:04.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:04.925+0000 7fce18241700 1 --2- 172.21.15.12:0/3811238574 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fce10106c20 0x7fce10107000 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fce0000b0d0 tx=0x7fce0000b3e0 comp rx=0 tx=0).stop 2024-02-18T11:39:04.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:04.925+0000 7fce18241700 1 -- 172.21.15.12:0/3811238574 shutdown_connections 2024-02-18T11:39:04.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:04.925+0000 7fce18241700 1 --2- 172.21.15.12:0/3811238574 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fce101015a0 0x7fce1010d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:04.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:04.925+0000 7fce18241700 1 --2- 172.21.15.12:0/3811238574 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fce10106c20 0x7fce10107000 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:04.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:04.925+0000 7fce18241700 1 --2- 172.21.15.12:0/3811238574 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce10100be0 0x7fce10101060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:04.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:04.925+0000 7fce18241700 1 -- 172.21.15.12:0/3811238574 >> 172.21.15.12:0/3811238574 conn(0x7fce10076c00 msgr2=0x7fce10077010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:39:04.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:04.925+0000 7fce18241700 1 -- 172.21.15.12:0/3811238574 shutdown_connections 2024-02-18T11:39:04.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:04.925+0000 7fce18241700 1 -- 172.21.15.12:0/3811238574 wait complete. 2024-02-18T11:39:04.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:04.926+0000 7fce18241700 1 Processor -- start 2024-02-18T11:39:04.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:04.926+0000 7fce18241700 1 -- start start 2024-02-18T11:39:04.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:04.926+0000 7fce18241700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce10100be0 0x7fce10198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:04.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:04.926+0000 7fce18241700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fce101015a0 0x7fce101989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:04.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:04.926+0000 7fce18241700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fce10106c20 0x7fce1019cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:04.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:04.926+0000 7fce18241700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce1019d2a0 con 0x7fce10106c20 2024-02-18T11:39:04.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:04.926+0000 7fce18241700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce1019d410 con 0x7fce101015a0 2024-02-18T11:39:04.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:04.926+0000 7fce18241700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce1019d5d0 con 0x7fce10100be0 2024-02-18T11:39:04.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:04.926+0000 7fce167de700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fce10106c20 0x7fce1019cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:04.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:04.926+0000 7fce167de700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fce10106c20 0x7fce1019cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50320/0 (socket says 172.21.15.12:50320) 2024-02-18T11:39:04.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:04.926+0000 7fce167de700 1 -- 172.21.15.12:0/3012045585 learned_addr learned my addr 172.21.15.12:0/3012045585 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:39:04.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:04.926+0000 7fce157dc700 1 --2- 172.21.15.12:0/3012045585 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fce101015a0 0x7fce101989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:04.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:04.926+0000 7fce15fdd700 1 --2- 172.21.15.12:0/3012045585 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce10100be0 0x7fce10198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:04.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:04.926+0000 7fce167de700 1 -- 172.21.15.12:0/3012045585 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce10100be0 msgr2=0x7fce10198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:04.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:04.926+0000 7fce167de700 1 --2- 172.21.15.12:0/3012045585 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce10100be0 0x7fce10198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:04.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:04.926+0000 7fce167de700 1 -- 172.21.15.12:0/3012045585 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fce101015a0 msgr2=0x7fce101989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:04.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:04.926+0000 7fce167de700 1 --2- 172.21.15.12:0/3012045585 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fce101015a0 0x7fce101989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:04.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:04.926+0000 7fce167de700 1 -- 172.21.15.12:0/3012045585 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce00009d00 con 0x7fce10106c20 2024-02-18T11:39:04.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:04.927+0000 7fce167de700 1 --2- 172.21.15.12:0/3012045585 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fce10106c20 0x7fce1019cc90 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fce0800b550 tx=0x7fce0800b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:39:04.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:04.927+0000 7fce0effd700 1 -- 172.21.15.12:0/3012045585 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce08011830 con 0x7fce10106c20 2024-02-18T11:39:04.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:04.927+0000 7fce0effd700 1 -- 172.21.15.12:0/3012045585 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fce08011e70 con 0x7fce10106c20 2024-02-18T11:39:04.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:04.927+0000 7fce18241700 1 -- 172.21.15.12:0/3012045585 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce1019d8b0 con 0x7fce10106c20 2024-02-18T11:39:04.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:04.927+0000 7fce0effd700 1 -- 172.21.15.12:0/3012045585 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce08010e70 con 0x7fce10106c20 2024-02-18T11:39:04.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:04.927+0000 7fce18241700 1 -- 172.21.15.12:0/3012045585 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce101a5110 con 0x7fce10106c20 2024-02-18T11:39:04.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:04.928+0000 7fce18241700 1 -- 172.21.15.12:0/3012045585 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce10068970 con 0x7fce10106c20 2024-02-18T11:39:04.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:04.929+0000 7fce0effd700 1 -- 172.21.15.12:0/3012045585 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 137296+0+0 (secure 0 0 0) 0x7fce08010510 con 0x7fce10106c20 2024-02-18T11:39:04.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:04.930+0000 7fce0effd700 1 --2- 172.21.15.12:0/3012045585 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fcdf80a28f0 0x7fcdf80a4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:04.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:04.930+0000 7fce15fdd700 1 --2- 172.21.15.12:0/3012045585 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fcdf80a28f0 0x7fcdf80a4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:04.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:04.930+0000 7fce0effd700 1 -- 172.21.15.12:0/3012045585 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fce080ce2d0 con 0x7fce10106c20 2024-02-18T11:39:04.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:04.930+0000 7fce15fdd700 1 --2- 172.21.15.12:0/3012045585 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fcdf80a28f0 0x7fcdf80a4db0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fce00009fd0 tx=0x7fce0000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:39:04.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:04.933+0000 7fce0effd700 1 -- 172.21.15.12:0/3012045585 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fce080970c0 con 0x7fce10106c20 2024-02-18T11:39:05.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:05.135+0000 7fce18241700 1 -- 172.21.15.12:0/3012045585 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fce10199720 con 0x7fce10106c20 2024-02-18T11:39:05.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:05.137+0000 7fce0effd700 1 -- 172.21.15.12:0/3012045585 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22229 (secure 0 0 0) 0x7fce08015020 con 0x7fce10106c20 2024-02-18T11:39:05.131 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:39:05.131 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":87,"fsid":"0c8bb0de-ce51-11ee-95ba-87774f69a715","created":"2024-02-18T11:30:49.110059+0000","modified":"2024-02-18T11:38:48.817117+0000","last_up_change":"2024-02-18T11:37:57.512337+0000","last_in_change":"2024-02-18T11:37:36.567969+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-18T11:34:15.281138+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-18T11:38:44.802134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-18T11:38:46.081188+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"35f15ea6-deb8-48a1-a7dd-ab649ec13533","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":536522426},{"type":"v1","addr":"172.21.15.12:6803","nonce":536522426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":536522426},{"type":"v1","addr":"172.21.15.12:6805","nonce":536522426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":536522426},{"type":"v1","addr":"172.21.15.12:6809","nonce":536522426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":536522426},{"type":"v1","addr":"172.21.15.12:6807","nonce":536522426}]},"public_addr":"172.21.15.12:6803/536522426","cluster_addr":"172.21.15.12:6805/536522426","heartbeat_back_addr":"172.21.15.12:6809/536522426","heartbeat_front_addr":"172.21.15.12:6807/536522426","state":["exists","up"]},{"osd":1,"uuid":"2af937ef-d827-44e6-b805-70cffbfe1126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":4249111488},{"type":"v1","addr":"172.21.15.158:6801","nonce":4249111488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":4249111488},{"type":"v1","addr":"172.21.15.158:6803","nonce":4249111488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":4249111488},{"type":"v1","addr":"172.21.15.158:6807","nonce":4249111488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":4249111488},{"type":"v1","addr":"172.21.15.158:6805","nonce":4249111488}]},"public_addr":"172.21.15.158:6801/4249111488","cluster_addr":"172.21.15.158:6803/4249111488","heartbeat_back_addr":"172.21.15.158:6807/4249111488","heartbeat_front_addr":"172.21.15.158:6805/4249111488","state":["exists","up"]},{"osd":2,"uuid":"17b804dd-31df-45ae-a637-4a5f65bbaf26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":627976150},{"type":"v1","addr":"172.21.15.188:6801","nonce":627976150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":627976150},{"type":"v1","addr":"172.21.15.188:6803","nonce":627976150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":627976150},{"type":"v1","addr":"172.21.15.188:6807","nonce":627976150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":627976150},{"type":"v1","addr":"172.21.15.188:6805","nonce":627976150}]},"public_addr":"172.21.15.188:6801/627976150","cluster_addr":"172.21.15.188:6803/627976150","heartbeat_back_addr":"172.21.15.188:6807/627976150","heartbeat_front_addr":"172.21.15.188:6805/627976150","state":["exists","up"]},{"osd":3,"uuid":"3f9cc423-cdfa-4acf-aaa2-b58b32ccb9cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":3758512503},{"type":"v1","addr":"172.21.15.12:6811","nonce":3758512503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":3758512503},{"type":"v1","addr":"172.21.15.12:6813","nonce":3758512503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3758512503},{"type":"v1","addr":"172.21.15.12:6817","nonce":3758512503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":3758512503},{"type":"v1","addr":"172.21.15.12:6815","nonce":3758512503}]},"public_addr":"172.21.15.12:6811/3758512503","cluster_addr":"172.21.15.12:6813/3758512503","heartbeat_back_addr":"172.21.15.12:6817/3758512503","heartbeat_front_addr":"172.21.15.12:6815/3758512503","state":["exists","up"]},{"osd":4,"uuid":"a92e20ec-af1c-4ef4-a0cb-caba5fcc71aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":3892820997},{"type":"v1","addr":"172.21.15.158:6809","nonce":3892820997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":3892820997},{"type":"v1","addr":"172.21.15.158:6811","nonce":3892820997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":3892820997},{"type":"v1","addr":"172.21.15.158:6815","nonce":3892820997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":3892820997},{"type":"v1","addr":"172.21.15.158:6813","nonce":3892820997}]},"public_addr":"172.21.15.158:6809/3892820997","cluster_addr":"172.21.15.158:6811/3892820997","heartbeat_back_addr":"172.21.15.158:6815/3892820997","heartbeat_front_addr":"172.21.15.158:6813/3892820997","state":["exists","up"]},{"osd":5,"uuid":"d9c2e0c8-85f5-4ae2-bf3e-125c1be3a2db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3939495896},{"type":"v1","addr":"172.21.15.188:6809","nonce":3939495896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3939495896},{"type":"v1","addr":"172.21.15.188:6811","nonce":3939495896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3939495896},{"type":"v1","addr":"172.21.15.188:6815","nonce":3939495896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3939495896},{"type":"v1","addr":"172.21.15.188:6813","nonce":3939495896}]},"public_addr":"172.21.15.188:6809/3939495896","cluster_addr":"172.21.15.188:6811/3939495896","heartbeat_back_addr":"172.21.15.188:6815/3939495896","heartbeat_front_addr":"172.21.15.188:6813/3939495896","state":["exists","up"]},{"osd":6,"uuid":"4a523512-de6b-49f0-be81-00f2f00c1c10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":1206337767},{"type":"v1","addr":"172.21.15.12:6819","nonce":1206337767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":1206337767},{"type":"v1","addr":"172.21.15.12:6821","nonce":1206337767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":1206337767},{"type":"v1","addr":"172.21.15.12:6825","nonce":1206337767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":1206337767},{"type":"v1","addr":"172.21.15.12:6823","nonce":1206337767}]},"public_addr":"172.21.15.12:6819/1206337767","cluster_addr":"172.21.15.12:6821/1206337767","heartbeat_back_addr":"172.21.15.12:6825/1206337767","heartbeat_front_addr":"172.21.15.12:6823/1206337767","state":["exists","up"]},{"osd":7,"uuid":"363271a6-df93-44c9-b0e5-3d29768aa96c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":3283400032},{"type":"v1","addr":"172.21.15.158:6817","nonce":3283400032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":3283400032},{"type":"v1","addr":"172.21.15.158:6819","nonce":3283400032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":3283400032},{"type":"v1","addr":"172.21.15.158:6823","nonce":3283400032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":3283400032},{"type":"v1","addr":"172.21.15.158:6821","nonce":3283400032}]},"public_addr":"172.21.15.158:6817/3283400032","cluster_addr":"172.21.15.158:6819/3283400032","heartbeat_back_addr":"172.21.15.158:6823/3283400032","heartbeat_front_addr":"172.21.15.158:6821/3283400032","state":["exists","up"]},{"osd":8,"uuid":"9a089528-3e8e-45d3-9279-ddb54088b96a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2110170682},{"type":"v1","addr":"172.21.15.188:6817","nonce":2110170682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2110170682},{"type":"v1","addr":"172.21.15.188:6819","nonce":2110170682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":2110170682},{"type":"v1","addr":"172.21.15.188:6823","nonce":2110170682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2110170682},{"type":"v1","addr":"172.21.15.188:6821","nonce":2110170682}]},"public_addr":"172.21.15.188:6817/2110170682","cluster_addr":"172.21.15.188:6819/2110170682","heartbeat_back_addr":"172.21.15.188:6823/2110170682","heartbeat_front_addr":"172.21.15.188:6821/2110170682","state":["exists","up"]},{"osd":9,"uuid":"b12e2727-d649-4464-90e1-fafbc1754832","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1825928986},{"type":"v1","addr":"172.21.15.12:6827","nonce":1825928986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1825928986},{"type":"v1","addr":"172.21.15.12:6829","nonce":1825928986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":1825928986},{"type":"v1","addr":"172.21.15.12:6833","nonce":1825928986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1825928986},{"type":"v1","addr":"172.21.15.12:6831","nonce":1825928986}]},"public_addr":"172.21.15.12:6827/1825928986","cluster_addr":"172.21.15.12:6829/1825928986","heartbeat_back_addr":"172.21.15.12:6833/1825928986","heartbeat_front_addr":"172.21.15.12:6831/1825928986","state":["exists","up"]},{"osd":10,"uuid":"518b4cbf-9d90-4e9a-8ac9-b5cb9b3dee4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":3239052167},{"type":"v1","addr":"172.21.15.158:6825","nonce":3239052167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":3239052167},{"type":"v1","addr":"172.21.15.158:6827","nonce":3239052167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":3239052167},{"type":"v1","addr":"172.21.15.158:6831","nonce":3239052167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":3239052167},{"type":"v1","addr":"172.21.15.158:6829","nonce":3239052167}]},"public_addr":"172.21.15.158:6825/3239052167","cluster_addr":"172.21.15.158:6827/3239052167","heartbeat_back_addr":"172.21.15.158:6831/3239052167","heartbeat_front_addr":"172.21.15.158:6829/3239052167","state":["exists","up"]},{"osd":11,"uuid":"98aa3a0d-4b2f-46c8-9341-375d13794c3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1135773350},{"type":"v1","addr":"172.21.15.188:6825","nonce":1135773350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1135773350},{"type":"v1","addr":"172.21.15.188:6827","nonce":1135773350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1135773350},{"type":"v1","addr":"172.21.15.188:6831","nonce":1135773350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1135773350},{"type":"v1","addr":"172.21.15.188:6829","nonce":1135773350}]},"public_addr":"172.21.15.188:6825/1135773350","cluster_addr":"172.21.15.188:6827/1135773350","heartbeat_back_addr":"172.21.15.188:6831/1135773350","heartbeat_front_addr":"172.21.15.188:6829/1135773350","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:33:24.280879+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:33:47.859461+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:34:11.725283+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:34:35.707629+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:35:00.024197+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:35:24.499803+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:35:48.953870+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:36:13.642456+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:36:38.936457+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:37:04.021466+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:37:29.073178+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:37:54.863919+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:6801/4135274411":"2024-02-19T11:31:50.936928+0000","172.21.15.12:6800/4135274411":"2024-02-19T11:31:50.936928+0000","172.21.15.12:0/118428584":"2024-02-19T11:31:50.936928+0000","172.21.15.12:0/4276810915":"2024-02-19T11:31:50.936928+0000","172.21.15.12:0/3777545262":"2024-02-19T11:31:24.000830+0000","172.21.15.12:0/114707743":"2024-02-19T11:31:24.000830+0000","172.21.15.12:6800/3925204723":"2024-02-19T11:31:24.000830+0000","172.21.15.12:6801/3925204723":"2024-02-19T11:31:24.000830+0000","172.21.15.12:0/74881618":"2024-02-19T11:31:24.000830+0000","172.21.15.12:0/1614805375":"2024-02-19T11:31:50.936928+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-18T11:39:05.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:05.139+0000 7fce18241700 1 -- 172.21.15.12:0/3012045585 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fcdf80a28f0 msgr2=0x7fcdf80a4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:05.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:05.139+0000 7fce18241700 1 --2- 172.21.15.12:0/3012045585 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fcdf80a28f0 0x7fcdf80a4db0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fce00009fd0 tx=0x7fce0000b040 comp rx=0 tx=0).stop 2024-02-18T11:39:05.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:05.139+0000 7fce18241700 1 -- 172.21.15.12:0/3012045585 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fce10106c20 msgr2=0x7fce1019cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:05.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:05.139+0000 7fce18241700 1 --2- 172.21.15.12:0/3012045585 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fce10106c20 0x7fce1019cc90 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fce0800b550 tx=0x7fce0800b860 comp rx=0 tx=0).stop 2024-02-18T11:39:05.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:05.139+0000 7fce18241700 1 -- 172.21.15.12:0/3012045585 shutdown_connections 2024-02-18T11:39:05.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:05.139+0000 7fce18241700 1 --2- 172.21.15.12:0/3012045585 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fcdf80a28f0 0x7fcdf80a4db0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:05.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:05.139+0000 7fce18241700 1 --2- 172.21.15.12:0/3012045585 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fce10106c20 0x7fce1019cc90 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:05.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:05.139+0000 7fce18241700 1 --2- 172.21.15.12:0/3012045585 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce10100be0 0x7fce10198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:05.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:05.139+0000 7fce18241700 1 --2- 172.21.15.12:0/3012045585 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fce101015a0 0x7fce101989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:05.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:05.139+0000 7fce18241700 1 -- 172.21.15.12:0/3012045585 >> 172.21.15.12:0/3012045585 conn(0x7fce10076c00 msgr2=0x7fce1006a330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:39:05.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:05.139+0000 7fce18241700 1 -- 172.21.15.12:0/3012045585 shutdown_connections 2024-02-18T11:39:05.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:05.139+0000 7fce18241700 1 -- 172.21.15.12:0/3012045585 wait complete. 2024-02-18T11:39:05.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:39:04 smithi012 ceph-mon[32460]: pgmap v293: 97 pgs: 97 active+clean; 627 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.2 KiB/s wr, 15 op/s 2024-02-18T11:39:05.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:39:04 smithi158 ceph-mon[36626]: pgmap v293: 97 pgs: 97 active+clean; 627 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.2 KiB/s wr, 15 op/s 2024-02-18T11:39:05.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:39:04 smithi188 ceph-mon[36925]: pgmap v293: 97 pgs: 97 active+clean; 627 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.2 KiB/s wr, 15 op/s 2024-02-18T11:39:05.766 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-02-18T11:39:05.766 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-02-18T11:39:05.766 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-02-18T11:39:05.767 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-02-18T11:39:05.767 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-02-18T11:39:05.768 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-02-18T11:39:05.768 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-02-18T11:39:05.768 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-02-18T11:39:05.769 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph tell osd.8 flush_pg_stats 2024-02-18T11:39:05.769 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph tell osd.9 flush_pg_stats 2024-02-18T11:39:05.770 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph tell osd.10 flush_pg_stats 2024-02-18T11:39:05.770 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph tell osd.11 flush_pg_stats 2024-02-18T11:39:06.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:39:05 smithi012 ceph-mon[32460]: pgmap v294: 97 pgs: 97 active+clean; 627 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.8 KiB/s wr, 9 op/s 2024-02-18T11:39:06.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:39:05 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/3012045585' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-18T11:39:06.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:39:05 smithi158 ceph-mon[36626]: pgmap v294: 97 pgs: 97 active+clean; 627 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.8 KiB/s wr, 9 op/s 2024-02-18T11:39:06.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:39:05 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/3012045585' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-18T11:39:06.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:39:05 smithi188 ceph-mon[36925]: pgmap v294: 97 pgs: 97 active+clean; 627 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.8 KiB/s wr, 9 op/s 2024-02-18T11:39:06.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:39:05 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/3012045585' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-18T11:39:06.489 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T11:39:06.491 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T11:39:06.495 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T11:39:06.539 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T11:39:06.545 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T11:39:06.551 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T11:39:06.729 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T11:39:07.521 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T11:39:07.521 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T11:39:07.522 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T11:39:07.522 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T11:39:07.522 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T11:39:08.206 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:39:08 smithi012 ceph-mon[32460]: pgmap v295: 97 pgs: 97 active+clean; 635 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 5.4 KiB/s wr, 10 op/s 2024-02-18T11:39:08.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:39:08 smithi188 ceph-mon[36925]: pgmap v295: 97 pgs: 97 active+clean; 635 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 5.4 KiB/s wr, 10 op/s 2024-02-18T11:39:08.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:39:08 smithi158 ceph-mon[36626]: pgmap v295: 97 pgs: 97 active+clean; 635 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 5.4 KiB/s wr, 10 op/s 2024-02-18T11:39:10.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:39:10 smithi188 ceph-mon[36925]: pgmap v296: 97 pgs: 97 active+clean; 635 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2024-02-18T11:39:10.423 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:39:10 smithi012 ceph-mon[32460]: pgmap v296: 97 pgs: 97 active+clean; 635 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2024-02-18T11:39:10.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:39:10 smithi158 ceph-mon[36626]: pgmap v296: 97 pgs: 97 active+clean; 635 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2024-02-18T11:39:12.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:39:12 smithi188 ceph-mon[36925]: pgmap v297: 97 pgs: 97 active+clean; 638 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-02-18T11:39:12.372 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:39:12 smithi012 ceph-mon[32460]: pgmap v297: 97 pgs: 97 active+clean; 638 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-02-18T11:39:12.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:39:12 smithi158 ceph-mon[36626]: pgmap v297: 97 pgs: 97 active+clean; 638 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-02-18T11:39:14.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:39:14 smithi188 ceph-mon[36925]: pgmap v298: 97 pgs: 97 active+clean; 638 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-02-18T11:39:14.423 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:39:14 smithi012 ceph-mon[32460]: pgmap v298: 97 pgs: 97 active+clean; 638 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-02-18T11:39:14.496 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:39:14 smithi158 ceph-mon[36626]: pgmap v298: 97 pgs: 97 active+clean; 638 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-02-18T11:39:16.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:39:16 smithi012 ceph-mon[32460]: pgmap v299: 97 pgs: 97 active+clean; 638 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-02-18T11:39:16.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:39:16 smithi188 ceph-mon[36925]: pgmap v299: 97 pgs: 97 active+clean; 638 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-02-18T11:39:16.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:39:16 smithi158 ceph-mon[36626]: pgmap v299: 97 pgs: 97 active+clean; 638 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-02-18T11:39:16.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:13.553+0000 7f8dcc726700 1 -- 172.21.15.12:0/1961440930 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8dc4106d20 msgr2=0x7f8dc4107100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:16.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:13.553+0000 7f8dcc726700 1 --2- 172.21.15.12:0/1961440930 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8dc4106d20 0x7f8dc4107100 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8dc0009990 tx=0x7f8dc0009ca0 comp rx=0 tx=0).stop 2024-02-18T11:39:16.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:13.554+0000 7f8dcc726700 1 -- 172.21.15.12:0/1961440930 shutdown_connections 2024-02-18T11:39:16.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:13.554+0000 7f8dcc726700 1 --2- 172.21.15.12:0/1961440930 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8dc4100c60 0x7f8dc410ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:16.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:13.554+0000 7f8dcc726700 1 --2- 172.21.15.12:0/1961440930 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8dc4106d20 0x7f8dc4107100 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:16.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:13.554+0000 7f8dcc726700 1 --2- 172.21.15.12:0/1961440930 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8dc41002a0 0x7f8dc4100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:16.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:13.554+0000 7f8dcc726700 1 -- 172.21.15.12:0/1961440930 >> 172.21.15.12:0/1961440930 conn(0x7f8dc40771b0 msgr2=0x7f8dc40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:39:16.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:13.554+0000 7f8dcc726700 1 -- 172.21.15.12:0/1961440930 shutdown_connections 2024-02-18T11:39:16.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:13.554+0000 7f8dcc726700 1 -- 172.21.15.12:0/1961440930 wait complete. 2024-02-18T11:39:16.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:13.554+0000 7f8dcc726700 1 Processor -- start 2024-02-18T11:39:16.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:13.554+0000 7f8dcc726700 1 -- start start 2024-02-18T11:39:16.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:13.554+0000 7f8dcc726700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8dc41002a0 0x7f8dc4101480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:16.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:13.555+0000 7f8dcc726700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8dc4100c60 0x7f8dc41019c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:16.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:13.555+0000 7f8dcc726700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8dc4106d20 0x7f8dc4101f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:16.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:13.555+0000 7f8dcc726700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc41a66b0 con 0x7f8dc4100c60 2024-02-18T11:39:16.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:13.555+0000 7f8dcc726700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc41a6820 con 0x7f8dc41002a0 2024-02-18T11:39:16.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:13.555+0000 7f8dcc726700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc41a69e0 con 0x7f8dc4106d20 2024-02-18T11:39:16.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:13.555+0000 7f8dc9cc1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8dc4100c60 0x7f8dc41019c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:16.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:13.555+0000 7f8dc9cc1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8dc4100c60 0x7f8dc41019c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42060/0 (socket says 172.21.15.12:42060) 2024-02-18T11:39:16.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:13.555+0000 7f8dc9cc1700 1 -- 172.21.15.12:0/1266882982 learned_addr learned my addr 172.21.15.12:0/1266882982 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:39:16.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:13.555+0000 7f8dca4c2700 1 --2- 172.21.15.12:0/1266882982 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8dc41002a0 0x7f8dc4101480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:16.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:13.555+0000 7f8dcacc3700 1 --2- 172.21.15.12:0/1266882982 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8dc4106d20 0x7f8dc4101f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:16.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:13.555+0000 7f8dc9cc1700 1 -- 172.21.15.12:0/1266882982 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8dc4106d20 msgr2=0x7f8dc4101f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:16.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:13.555+0000 7f8dc9cc1700 1 --2- 172.21.15.12:0/1266882982 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8dc4106d20 0x7f8dc4101f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:16.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:13.555+0000 7f8dc9cc1700 1 -- 172.21.15.12:0/1266882982 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8dc41002a0 msgr2=0x7f8dc4101480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:16.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:13.555+0000 7f8dc9cc1700 1 --2- 172.21.15.12:0/1266882982 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8dc41002a0 0x7f8dc4101480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:16.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:13.555+0000 7f8dc9cc1700 1 -- 172.21.15.12:0/1266882982 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dc00095f0 con 0x7f8dc4100c60 2024-02-18T11:39:16.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:13.555+0000 7f8dc9cc1700 1 --2- 172.21.15.12:0/1266882982 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8dc4100c60 0x7f8dc41019c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8db400d980 tx=0x7f8db400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:39:16.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:13.556+0000 7f8dbb7fe700 1 -- 172.21.15.12:0/1266882982 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db4004220 con 0x7f8dc4100c60 2024-02-18T11:39:16.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:13.556+0000 7f8dbb7fe700 1 -- 172.21.15.12:0/1266882982 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8db4007640 con 0x7f8dc4100c60 2024-02-18T11:39:16.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:13.556+0000 7f8dcc726700 1 -- 172.21.15.12:0/1266882982 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dc41a6cc0 con 0x7f8dc4100c60 2024-02-18T11:39:16.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:13.556+0000 7f8dbb7fe700 1 -- 172.21.15.12:0/1266882982 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db4005720 con 0x7f8dc4100c60 2024-02-18T11:39:16.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:13.556+0000 7f8dcc726700 1 -- 172.21.15.12:0/1266882982 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dc41a7150 con 0x7f8dc4100c60 2024-02-18T11:39:16.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:13.557+0000 7f8dcc726700 1 -- 172.21.15.12:0/1266882982 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8dc41120f0 con 0x7f8dc4100c60 2024-02-18T11:39:16.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:13.559+0000 7f8dbb7fe700 1 -- 172.21.15.12:0/1266882982 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 137296+0+0 (secure 0 0 0) 0x7f8db4004410 con 0x7f8dc4100c60 2024-02-18T11:39:16.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:13.559+0000 7f8dbb7fe700 1 --2- 172.21.15.12:0/1266882982 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f8db00a28f0 0x7f8db00a4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:16.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:13.559+0000 7f8dca4c2700 1 --2- 172.21.15.12:0/1266882982 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f8db00a28f0 0x7f8db00a4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:16.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:13.559+0000 7f8dbb7fe700 1 -- 172.21.15.12:0/1266882982 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f8db40ce8d0 con 0x7f8dc4100c60 2024-02-18T11:39:16.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:13.559+0000 7f8dbb7fe700 1 --2- 172.21.15.12:0/1266882982 >> [v2:172.21.15.158:6800/4249111488,v1:172.21.15.158:6801/4249111488] conn(0x7f8db00ac0d0 0x7f8db00ae4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:16.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:13.559+0000 7f8dbb7fe700 1 -- 172.21.15.12:0/1266882982 --> [v2:172.21.15.158:6800/4249111488,v1:172.21.15.158:6801/4249111488] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8db00aeba0 con 0x7f8db00ac0d0 2024-02-18T11:39:16.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:13.559+0000 7f8dbb7fe700 1 -- 172.21.15.12:0/1266882982 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8db40ced70 con 0x7f8dc4100c60 2024-02-18T11:39:16.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:13.560+0000 7f8dca4c2700 1 --2- 172.21.15.12:0/1266882982 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f8db00a28f0 0x7f8db00a4db0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8dc000b5c0 tx=0x7f8dc0005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:39:16.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:13.560+0000 7f8dcacc3700 1 --2- 172.21.15.12:0/1266882982 >> [v2:172.21.15.158:6800/4249111488,v1:172.21.15.158:6801/4249111488] conn(0x7f8db00ac0d0 0x7f8db00ae4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:16.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:13.560+0000 7f8dcacc3700 1 --2- 172.21.15.12:0/1266882982 >> [v2:172.21.15.158:6800/4249111488,v1:172.21.15.158:6801/4249111488] conn(0x7f8db00ac0d0 0x7f8db00ae4f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:39:16.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:13.561+0000 7f8dbb7fe700 1 -- 172.21.15.12:0/1266882982 <== osd.1 v2:172.21.15.158:6800/4249111488 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f8db00aeba0 con 0x7f8db00ac0d0 2024-02-18T11:39:16.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:13.578+0000 7f8dcc726700 1 -- 172.21.15.12:0/1266882982 --> [v2:172.21.15.158:6800/4249111488,v1:172.21.15.158:6801/4249111488] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8dc41056a0 con 0x7f8db00ac0d0 2024-02-18T11:39:16.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:13.578+0000 7f8dbb7fe700 1 -- 172.21.15.12:0/1266882982 <== osd.1 v2:172.21.15.158:6800/4249111488 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8dc41056a0 con 0x7f8db00ac0d0 2024-02-18T11:39:16.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:13.578+0000 7f8dcc726700 1 -- 172.21.15.12:0/1266882982 >> [v2:172.21.15.158:6800/4249111488,v1:172.21.15.158:6801/4249111488] conn(0x7f8db00ac0d0 msgr2=0x7f8db00ae4f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:16.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:13.578+0000 7f8dcc726700 1 --2- 172.21.15.12:0/1266882982 >> [v2:172.21.15.158:6800/4249111488,v1:172.21.15.158:6801/4249111488] conn(0x7f8db00ac0d0 0x7f8db00ae4f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:16.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:13.579+0000 7f8dcc726700 1 -- 172.21.15.12:0/1266882982 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f8db00a28f0 msgr2=0x7f8db00a4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:16.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:13.579+0000 7f8dcc726700 1 --2- 172.21.15.12:0/1266882982 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f8db00a28f0 0x7f8db00a4db0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8dc000b5c0 tx=0x7f8dc0005960 comp rx=0 tx=0).stop 2024-02-18T11:39:16.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:13.579+0000 7f8dcc726700 1 -- 172.21.15.12:0/1266882982 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8dc4100c60 msgr2=0x7f8dc41019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:16.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:13.579+0000 7f8dcc726700 1 --2- 172.21.15.12:0/1266882982 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8dc4100c60 0x7f8dc41019c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8db400d980 tx=0x7f8db400dc90 comp rx=0 tx=0).stop 2024-02-18T11:39:16.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:13.579+0000 7f8dcacc3700 1 -- 172.21.15.12:0/1266882982 reap_dead start 2024-02-18T11:39:16.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:13.579+0000 7f8dcc726700 1 -- 172.21.15.12:0/1266882982 shutdown_connections 2024-02-18T11:39:16.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:13.579+0000 7f8dcc726700 1 -- 172.21.15.12:0/1266882982 >> 172.21.15.12:0/1266882982 conn(0x7f8dc40771b0 msgr2=0x7f8dc4104cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:39:16.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:13.579+0000 7f8dcc726700 1 -- 172.21.15.12:0/1266882982 shutdown_connections 2024-02-18T11:39:16.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:13.579+0000 7f8dcc726700 1 -- 172.21.15.12:0/1266882982 wait complete. 2024-02-18T11:39:17.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:17.134+0000 7f543cba0700 1 -- 172.21.15.12:0/437521686 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f54381017e0 msgr2=0x7f5438105830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:17.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:17.134+0000 7f543cba0700 1 --2- 172.21.15.12:0/437521686 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f54381017e0 0x7f5438105830 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5420009990 tx=0x7f5420009ca0 comp rx=0 tx=0).stop 2024-02-18T11:39:17.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:17.134+0000 7f543cba0700 1 -- 172.21.15.12:0/437521686 shutdown_connections 2024-02-18T11:39:17.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:17.134+0000 7f543cba0700 1 --2- 172.21.15.12:0/437521686 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5438105f10 0x7f543810dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:17.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:17.134+0000 7f543cba0700 1 --2- 172.21.15.12:0/437521686 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5438100e30 0x7f5438101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:17.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:17.134+0000 7f543cba0700 1 --2- 172.21.15.12:0/437521686 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f54381017e0 0x7f5438105830 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:17.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:17.134+0000 7f543cba0700 1 -- 172.21.15.12:0/437521686 >> 172.21.15.12:0/437521686 conn(0x7f5438077c10 msgr2=0x7f5438078020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:39:17.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:17.134+0000 7f543cba0700 1 -- 172.21.15.12:0/437521686 shutdown_connections 2024-02-18T11:39:17.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:17.134+0000 7f543cba0700 1 -- 172.21.15.12:0/437521686 wait complete. 2024-02-18T11:39:17.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:17.135+0000 7f543cba0700 1 Processor -- start 2024-02-18T11:39:17.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:17.135+0000 7f543cba0700 1 -- start start 2024-02-18T11:39:17.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:17.135+0000 7f543cba0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5438100e30 0x7f5438196e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:17.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:17.135+0000 7f543cba0700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f54381017e0 0x7f5438197360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:17.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:17.135+0000 7f543cba0700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5438105f10 0x7f54381978a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:17.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:17.135+0000 7f543cba0700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54381a6560 con 0x7f5438100e30 2024-02-18T11:39:17.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:17.135+0000 7f543cba0700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54381a66d0 con 0x7f54381017e0 2024-02-18T11:39:17.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:17.135+0000 7f543cba0700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54381a6890 con 0x7f5438105f10 2024-02-18T11:39:17.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:17.135+0000 7f543659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5438100e30 0x7f5438196e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:17.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:17.135+0000 7f543659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5438100e30 0x7f5438196e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42072/0 (socket says 172.21.15.12:42072) 2024-02-18T11:39:17.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:17.135+0000 7f543659c700 1 -- 172.21.15.12:0/3638857303 learned_addr learned my addr 172.21.15.12:0/3638857303 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:39:17.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:17.135+0000 7f5436d9d700 1 --2- 172.21.15.12:0/3638857303 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5438105f10 0x7f54381978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:17.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:17.135+0000 7f5435d9b700 1 --2- 172.21.15.12:0/3638857303 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f54381017e0 0x7f5438197360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:17.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:17.136+0000 7f543659c700 1 -- 172.21.15.12:0/3638857303 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5438105f10 msgr2=0x7f54381978a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:17.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:17.136+0000 7f543659c700 1 --2- 172.21.15.12:0/3638857303 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5438105f10 0x7f54381978a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:17.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:17.136+0000 7f543659c700 1 -- 172.21.15.12:0/3638857303 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f54381017e0 msgr2=0x7f5438197360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:17.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:17.136+0000 7f543659c700 1 --2- 172.21.15.12:0/3638857303 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f54381017e0 0x7f5438197360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:17.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:17.136+0000 7f543659c700 1 -- 172.21.15.12:0/3638857303 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54200095f0 con 0x7f5438100e30 2024-02-18T11:39:17.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:17.136+0000 7f543659c700 1 --2- 172.21.15.12:0/3638857303 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5438100e30 0x7f5438196e20 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5428007e20 tx=0x7f542800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:39:17.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:17.136+0000 7f542f7fe700 1 -- 172.21.15.12:0/3638857303 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542800ad40 con 0x7f5438100e30 2024-02-18T11:39:17.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:17.136+0000 7f543cba0700 1 -- 172.21.15.12:0/3638857303 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54381a6b70 con 0x7f5438100e30 2024-02-18T11:39:17.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:17.136+0000 7f542f7fe700 1 -- 172.21.15.12:0/3638857303 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5428012900 con 0x7f5438100e30 2024-02-18T11:39:17.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:17.136+0000 7f542f7fe700 1 -- 172.21.15.12:0/3638857303 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5428011840 con 0x7f5438100e30 2024-02-18T11:39:17.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:17.136+0000 7f543cba0700 1 -- 172.21.15.12:0/3638857303 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54381a7070 con 0x7f5438100e30 2024-02-18T11:39:17.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:17.137+0000 7f543cba0700 1 -- 172.21.15.12:0/3638857303 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5438110f20 con 0x7f5438100e30 2024-02-18T11:39:17.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:17.138+0000 7f542f7fe700 1 -- 172.21.15.12:0/3638857303 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 137296+0+0 (secure 0 0 0) 0x7f5428012a70 con 0x7f5438100e30 2024-02-18T11:39:17.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:17.138+0000 7f542f7fe700 1 --2- 172.21.15.12:0/3638857303 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f54240a2870 0x7f54240a4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:17.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:17.139+0000 7f5435d9b700 1 --2- 172.21.15.12:0/3638857303 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f54240a2870 0x7f54240a4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:17.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:17.139+0000 7f542f7fe700 1 -- 172.21.15.12:0/3638857303 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f54280cecd0 con 0x7f5438100e30 2024-02-18T11:39:17.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:17.139+0000 7f542f7fe700 1 --2- 172.21.15.12:0/3638857303 >> [v2:172.21.15.158:6816/3283400032,v1:172.21.15.158:6817/3283400032] conn(0x7f54240ac050 0x7f54240ae470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:17.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:17.139+0000 7f542f7fe700 1 -- 172.21.15.12:0/3638857303 --> [v2:172.21.15.158:6816/3283400032,v1:172.21.15.158:6817/3283400032] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f54240aeb20 con 0x7f54240ac050 2024-02-18T11:39:17.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:17.139+0000 7f542f7fe700 1 -- 172.21.15.12:0/3638857303 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f54280cf040 con 0x7f5438100e30 2024-02-18T11:39:17.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:17.139+0000 7f5435d9b700 1 --2- 172.21.15.12:0/3638857303 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f54240a2870 0x7f54240a4d30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f542000b5c0 tx=0x7f542001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:39:17.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:17.139+0000 7f5436d9d700 1 --2- 172.21.15.12:0/3638857303 >> [v2:172.21.15.158:6816/3283400032,v1:172.21.15.158:6817/3283400032] conn(0x7f54240ac050 0x7f54240ae470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:17.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:17.140+0000 7f5436d9d700 1 --2- 172.21.15.12:0/3638857303 >> [v2:172.21.15.158:6816/3283400032,v1:172.21.15.158:6817/3283400032] conn(0x7f54240ac050 0x7f54240ae470 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:39:17.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:17.141+0000 7f542f7fe700 1 -- 172.21.15.12:0/3638857303 <== osd.7 v2:172.21.15.158:6816/3283400032 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f54240aeb20 con 0x7f54240ac050 2024-02-18T11:39:17.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:17.157+0000 7f543cba0700 1 -- 172.21.15.12:0/3638857303 --> [v2:172.21.15.158:6816/3283400032,v1:172.21.15.158:6817/3283400032] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f54380623c0 con 0x7f54240ac050 2024-02-18T11:39:17.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:17.158+0000 7f542f7fe700 1 -- 172.21.15.12:0/3638857303 <== osd.7 v2:172.21.15.158:6816/3283400032 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f54380623c0 con 0x7f54240ac050 2024-02-18T11:39:17.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:17.158+0000 7f543cba0700 1 -- 172.21.15.12:0/3638857303 >> [v2:172.21.15.158:6816/3283400032,v1:172.21.15.158:6817/3283400032] conn(0x7f54240ac050 msgr2=0x7f54240ae470 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:17.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:17.158+0000 7f543cba0700 1 --2- 172.21.15.12:0/3638857303 >> [v2:172.21.15.158:6816/3283400032,v1:172.21.15.158:6817/3283400032] conn(0x7f54240ac050 0x7f54240ae470 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:17.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:17.158+0000 7f543cba0700 1 -- 172.21.15.12:0/3638857303 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f54240a2870 msgr2=0x7f54240a4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:17.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:17.158+0000 7f543cba0700 1 --2- 172.21.15.12:0/3638857303 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f54240a2870 0x7f54240a4d30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f542000b5c0 tx=0x7f542001a040 comp rx=0 tx=0).stop 2024-02-18T11:39:17.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:17.158+0000 7f543cba0700 1 -- 172.21.15.12:0/3638857303 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5438100e30 msgr2=0x7f5438196e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:17.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:17.158+0000 7f543cba0700 1 --2- 172.21.15.12:0/3638857303 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5438100e30 0x7f5438196e20 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5428007e20 tx=0x7f542800a340 comp rx=0 tx=0).stop 2024-02-18T11:39:17.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:17.158+0000 7f5436d9d700 1 -- 172.21.15.12:0/3638857303 reap_dead start 2024-02-18T11:39:17.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:17.158+0000 7f543cba0700 1 -- 172.21.15.12:0/3638857303 shutdown_connections 2024-02-18T11:39:17.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:17.158+0000 7f543cba0700 1 -- 172.21.15.12:0/3638857303 >> 172.21.15.12:0/3638857303 conn(0x7f5438077c10 msgr2=0x7f54380feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:39:17.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:17.158+0000 7f543cba0700 1 -- 172.21.15.12:0/3638857303 shutdown_connections 2024-02-18T11:39:17.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:17.158+0000 7f543cba0700 1 -- 172.21.15.12:0/3638857303 wait complete. 2024-02-18T11:39:18.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:39:18 smithi188 ceph-mon[36925]: pgmap v300: 97 pgs: 97 active+clean; 638 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-02-18T11:39:18.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:39:18 smithi012 ceph-mon[32460]: pgmap v300: 97 pgs: 97 active+clean; 638 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-02-18T11:39:18.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:39:18 smithi158 ceph-mon[36626]: pgmap v300: 97 pgs: 97 active+clean; 638 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-02-18T11:39:20.304 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:39:20 smithi012 ceph-mon[32460]: pgmap v301: 97 pgs: 97 active+clean; 638 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-02-18T11:39:20.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:39:20 smithi188 ceph-mon[36925]: pgmap v301: 97 pgs: 97 active+clean; 638 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-02-18T11:39:20.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:20.496+0000 7ff8cbddb700 1 -- 172.21.15.12:0/3482071605 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8c4105720 msgr2=0x7ff8c410d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:20.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:20.496+0000 7ff8cbddb700 1 --2- 172.21.15.12:0/3482071605 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8c4105720 0x7ff8c410d260 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc00b0d0 tx=0x7ff8bc00b3e0 comp rx=0 tx=0).stop 2024-02-18T11:39:20.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:20.496+0000 7ff8cbddb700 1 -- 172.21.15.12:0/3482071605 shutdown_connections 2024-02-18T11:39:20.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:20.496+0000 7ff8cbddb700 1 --2- 172.21.15.12:0/3482071605 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8c4105720 0x7ff8c410d260 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:20.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:20.496+0000 7ff8cbddb700 1 --2- 172.21.15.12:0/3482071605 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8c4100850 0x7ff8c4100c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:20.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:20.496+0000 7ff8cbddb700 1 --2- 172.21.15.12:0/3482071605 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff8c4101200 0x7ff8c4104ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:20.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:20.496+0000 7ff8cbddb700 1 -- 172.21.15.12:0/3482071605 >> 172.21.15.12:0/3482071605 conn(0x7ff8c4075ef0 msgr2=0x7ff8c4076300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:39:20.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:20.496+0000 7ff8cbddb700 1 -- 172.21.15.12:0/3482071605 shutdown_connections 2024-02-18T11:39:20.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:20.496+0000 7ff8cbddb700 1 -- 172.21.15.12:0/3482071605 wait complete. 2024-02-18T11:39:20.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:20.496+0000 7ff8cbddb700 1 Processor -- start 2024-02-18T11:39:20.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:20.496+0000 7ff8cbddb700 1 -- start start 2024-02-18T11:39:20.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:20.497+0000 7ff8cbddb700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8c4100850 0x7ff8c4198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:20.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:20.497+0000 7ff8cbddb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8c4101200 0x7ff8c4198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:20.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:20.497+0000 7ff8cbddb700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff8c4105720 0x7ff8c419cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:20.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:20.497+0000 7ff8cbddb700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c419d0f0 con 0x7ff8c4101200 2024-02-18T11:39:20.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:20.497+0000 7ff8cbddb700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c419d260 con 0x7ff8c4105720 2024-02-18T11:39:20.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:20.497+0000 7ff8cbddb700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c419d420 con 0x7ff8c4100850 2024-02-18T11:39:20.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:20.497+0000 7ff8c9376700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8c4101200 0x7ff8c4198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:20.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:20.497+0000 7ff8c9376700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8c4101200 0x7ff8c4198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42108/0 (socket says 172.21.15.12:42108) 2024-02-18T11:39:20.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:20.497+0000 7ff8c9376700 1 -- 172.21.15.12:0/1482929652 learned_addr learned my addr 172.21.15.12:0/1482929652 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:39:20.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:20.497+0000 7ff8ca378700 1 --2- 172.21.15.12:0/1482929652 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff8c4105720 0x7ff8c419cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:20.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:20.497+0000 7ff8c9b77700 1 --2- 172.21.15.12:0/1482929652 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8c4100850 0x7ff8c4198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:20.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:20.497+0000 7ff8c9376700 1 -- 172.21.15.12:0/1482929652 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8c4100850 msgr2=0x7ff8c4198210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:20.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:20.497+0000 7ff8c9376700 1 --2- 172.21.15.12:0/1482929652 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8c4100850 0x7ff8c4198210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:20.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:20.497+0000 7ff8c9376700 1 -- 172.21.15.12:0/1482929652 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff8c4105720 msgr2=0x7ff8c419cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:20.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:20.497+0000 7ff8c9376700 1 --2- 172.21.15.12:0/1482929652 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff8c4105720 0x7ff8c419cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:20.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:20.497+0000 7ff8c9376700 1 -- 172.21.15.12:0/1482929652 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8bc009d00 con 0x7ff8c4101200 2024-02-18T11:39:20.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:20.497+0000 7ff8c9376700 1 --2- 172.21.15.12:0/1482929652 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8c4101200 0x7ff8c4198750 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff8c000c9b0 tx=0x7ff8c000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:39:20.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:20.497+0000 7ff8baffd700 1 -- 172.21.15.12:0/1482929652 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c0007aa0 con 0x7ff8c4101200 2024-02-18T11:39:20.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:20.497+0000 7ff8baffd700 1 -- 172.21.15.12:0/1482929652 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff8c000ce50 con 0x7ff8c4101200 2024-02-18T11:39:20.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:20.497+0000 7ff8baffd700 1 -- 172.21.15.12:0/1482929652 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c0018660 con 0x7ff8c4101200 2024-02-18T11:39:20.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:20.498+0000 7ff8cbddb700 1 -- 172.21.15.12:0/1482929652 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8c419d700 con 0x7ff8c4101200 2024-02-18T11:39:20.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:20.498+0000 7ff8cbddb700 1 -- 172.21.15.12:0/1482929652 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8c4102120 con 0x7ff8c4101200 2024-02-18T11:39:20.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:20.498+0000 7ff8cbddb700 1 -- 172.21.15.12:0/1482929652 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff8c4101a60 con 0x7ff8c4101200 2024-02-18T11:39:20.496 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:39:20 smithi158 ceph-mon[36626]: pgmap v301: 97 pgs: 97 active+clean; 638 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-02-18T11:39:20.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:20.499+0000 7ff8baffd700 1 -- 172.21.15.12:0/1482929652 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 137296+0+0 (secure 0 0 0) 0x7ff8c000f450 con 0x7ff8c4101200 2024-02-18T11:39:20.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:20.500+0000 7ff8baffd700 1 --2- 172.21.15.12:0/1482929652 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7ff8b00a2870 0x7ff8b00a4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:20.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:20.500+0000 7ff8baffd700 1 -- 172.21.15.12:0/1482929652 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff8c00ce640 con 0x7ff8c4101200 2024-02-18T11:39:20.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:20.500+0000 7ff8c9b77700 1 --2- 172.21.15.12:0/1482929652 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7ff8b00a2870 0x7ff8b00a4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:20.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:20.500+0000 7ff8baffd700 1 --2- 172.21.15.12:0/1482929652 >> [v2:172.21.15.188:6800/627976150,v1:172.21.15.188:6801/627976150] conn(0x7ff8b00ac050 0x7ff8b00ae470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:20.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:20.500+0000 7ff8baffd700 1 -- 172.21.15.12:0/1482929652 --> [v2:172.21.15.188:6800/627976150,v1:172.21.15.188:6801/627976150] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff8b00aeb20 con 0x7ff8b00ac050 2024-02-18T11:39:20.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:20.500+0000 7ff8baffd700 1 -- 172.21.15.12:0/1482929652 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff8c00ceae0 con 0x7ff8c4101200 2024-02-18T11:39:20.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:20.501+0000 7ff8c9b77700 1 --2- 172.21.15.12:0/1482929652 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7ff8b00a2870 0x7ff8b00a4d30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4005950 tx=0x7ff8b400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:39:20.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:20.501+0000 7ff8ca378700 1 --2- 172.21.15.12:0/1482929652 >> [v2:172.21.15.188:6800/627976150,v1:172.21.15.188:6801/627976150] conn(0x7ff8b00ac050 0x7ff8b00ae470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:20.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:20.501+0000 7ff8ca378700 1 --2- 172.21.15.12:0/1482929652 >> [v2:172.21.15.188:6800/627976150,v1:172.21.15.188:6801/627976150] conn(0x7ff8b00ac050 0x7ff8b00ae470 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:39:20.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:20.502+0000 7ff8baffd700 1 -- 172.21.15.12:0/1482929652 <== osd.2 v2:172.21.15.188:6800/627976150 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7ff8b00aeb20 con 0x7ff8b00ac050 2024-02-18T11:39:20.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:20.518+0000 7ff8cbddb700 1 -- 172.21.15.12:0/1482929652 --> [v2:172.21.15.188:6800/627976150,v1:172.21.15.188:6801/627976150] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff8c4110710 con 0x7ff8b00ac050 2024-02-18T11:39:20.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:20.519+0000 7ff8baffd700 1 -- 172.21.15.12:0/1482929652 <== osd.2 v2:172.21.15.188:6800/627976150 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff8c4110710 con 0x7ff8b00ac050 2024-02-18T11:39:20.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:20.519+0000 7ff8cbddb700 1 -- 172.21.15.12:0/1482929652 >> [v2:172.21.15.188:6800/627976150,v1:172.21.15.188:6801/627976150] conn(0x7ff8b00ac050 msgr2=0x7ff8b00ae470 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:20.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:20.519+0000 7ff8cbddb700 1 --2- 172.21.15.12:0/1482929652 >> [v2:172.21.15.188:6800/627976150,v1:172.21.15.188:6801/627976150] conn(0x7ff8b00ac050 0x7ff8b00ae470 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:20.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:20.519+0000 7ff8cbddb700 1 -- 172.21.15.12:0/1482929652 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7ff8b00a2870 msgr2=0x7ff8b00a4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:20.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:20.519+0000 7ff8cbddb700 1 --2- 172.21.15.12:0/1482929652 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7ff8b00a2870 0x7ff8b00a4d30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4005950 tx=0x7ff8b400b410 comp rx=0 tx=0).stop 2024-02-18T11:39:20.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:20.519+0000 7ff8cbddb700 1 -- 172.21.15.12:0/1482929652 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8c4101200 msgr2=0x7ff8c4198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:20.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:20.519+0000 7ff8cbddb700 1 --2- 172.21.15.12:0/1482929652 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8c4101200 0x7ff8c4198750 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff8c000c9b0 tx=0x7ff8c000ccc0 comp rx=0 tx=0).stop 2024-02-18T11:39:20.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:20.519+0000 7ff8ca378700 1 -- 172.21.15.12:0/1482929652 reap_dead start 2024-02-18T11:39:20.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:20.519+0000 7ff8cbddb700 1 -- 172.21.15.12:0/1482929652 shutdown_connections 2024-02-18T11:39:20.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:20.519+0000 7ff8cbddb700 1 -- 172.21.15.12:0/1482929652 >> 172.21.15.12:0/1482929652 conn(0x7ff8c4075ef0 msgr2=0x7ff8c4107180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:39:20.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:20.519+0000 7ff8cbddb700 1 -- 172.21.15.12:0/1482929652 shutdown_connections 2024-02-18T11:39:20.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:20.519+0000 7ff8cbddb700 1 -- 172.21.15.12:0/1482929652 wait complete. 2024-02-18T11:39:21.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:18.210+0000 7fb75a9d5700 1 -- 172.21.15.12:0/2162065349 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7541015a0 msgr2=0x7fb75410d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:21.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:18.210+0000 7fb75a9d5700 1 --2- 172.21.15.12:0/2162065349 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7541015a0 0x7fb75410d9f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb74c00b0d0 tx=0x7fb74c00b3e0 comp rx=0 tx=0).stop 2024-02-18T11:39:21.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:18.210+0000 7fb75a9d5700 1 -- 172.21.15.12:0/2162065349 shutdown_connections 2024-02-18T11:39:21.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:18.210+0000 7fb75a9d5700 1 --2- 172.21.15.12:0/2162065349 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7541015a0 0x7fb75410d9f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:21.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:18.210+0000 7fb75a9d5700 1 --2- 172.21.15.12:0/2162065349 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb754106c20 0x7fb754107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:21.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:18.210+0000 7fb75a9d5700 1 --2- 172.21.15.12:0/2162065349 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb754100be0 0x7fb754101060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:21.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:18.210+0000 7fb75a9d5700 1 -- 172.21.15.12:0/2162065349 >> 172.21.15.12:0/2162065349 conn(0x7fb754076c00 msgr2=0x7fb754077010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:39:21.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:18.210+0000 7fb75a9d5700 1 -- 172.21.15.12:0/2162065349 shutdown_connections 2024-02-18T11:39:21.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:18.210+0000 7fb75a9d5700 1 -- 172.21.15.12:0/2162065349 wait complete. 2024-02-18T11:39:21.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:18.211+0000 7fb75a9d5700 1 Processor -- start 2024-02-18T11:39:21.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:18.211+0000 7fb75a9d5700 1 -- start start 2024-02-18T11:39:21.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:18.211+0000 7fb75a9d5700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb754100be0 0x7fb754198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:21.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:18.212+0000 7fb75a9d5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7541015a0 0x7fb754198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:21.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:18.212+0000 7fb75a9d5700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb754106c20 0x7fb75419cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:21.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:18.212+0000 7fb75a9d5700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb75419d130 con 0x7fb7541015a0 2024-02-18T11:39:21.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:18.212+0000 7fb75a9d5700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb75419d270 con 0x7fb754100be0 2024-02-18T11:39:21.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:18.212+0000 7fb75a9d5700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb75419d430 con 0x7fb754106c20 2024-02-18T11:39:21.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:18.212+0000 7fb7537fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7541015a0 0x7fb754198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:21.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:18.212+0000 7fb753fff700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb754100be0 0x7fb754198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:21.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:18.212+0000 7fb7537fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7541015a0 0x7fb754198ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42084/0 (socket says 172.21.15.12:42084) 2024-02-18T11:39:21.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:18.212+0000 7fb753fff700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb754100be0 0x7fb754198660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.158:3300/0 says I am v2:172.21.15.12:36328/0 (socket says 172.21.15.12:36328) 2024-02-18T11:39:21.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:18.212+0000 7fb7537fe700 1 -- 172.21.15.12:0/2323166578 learned_addr learned my addr 172.21.15.12:0/2323166578 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:39:21.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:18.212+0000 7fb758f72700 1 --2- 172.21.15.12:0/2323166578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb754106c20 0x7fb75419cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:21.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:18.212+0000 7fb7537fe700 1 -- 172.21.15.12:0/2323166578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb754106c20 msgr2=0x7fb75419cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:21.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:18.212+0000 7fb7537fe700 1 --2- 172.21.15.12:0/2323166578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb754106c20 0x7fb75419cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:21.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:18.212+0000 7fb7537fe700 1 -- 172.21.15.12:0/2323166578 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb754100be0 msgr2=0x7fb754198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:21.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:18.212+0000 7fb7537fe700 1 --2- 172.21.15.12:0/2323166578 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb754100be0 0x7fb754198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:21.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:18.212+0000 7fb7537fe700 1 -- 172.21.15.12:0/2323166578 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb74c009d00 con 0x7fb7541015a0 2024-02-18T11:39:21.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:18.212+0000 7fb753fff700 1 --2- 172.21.15.12:0/2323166578 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb754100be0 0x7fb754198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-18T11:39:21.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:18.212+0000 7fb7537fe700 1 --2- 172.21.15.12:0/2323166578 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7541015a0 0x7fb754198ba0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb74800e9a0 tx=0x7fb74800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:39:21.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:18.213+0000 7fb7517fa700 1 -- 172.21.15.12:0/2323166578 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb748009790 con 0x7fb7541015a0 2024-02-18T11:39:21.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:18.213+0000 7fb7517fa700 1 -- 172.21.15.12:0/2323166578 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb748010430 con 0x7fb7541015a0 2024-02-18T11:39:21.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:18.213+0000 7fb7517fa700 1 -- 172.21.15.12:0/2323166578 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb748018680 con 0x7fb7541015a0 2024-02-18T11:39:21.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:18.213+0000 7fb75a9d5700 1 -- 172.21.15.12:0/2323166578 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb75419d710 con 0x7fb7541015a0 2024-02-18T11:39:21.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:18.213+0000 7fb75a9d5700 1 -- 172.21.15.12:0/2323166578 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7541a4fa0 con 0x7fb7541015a0 2024-02-18T11:39:21.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:18.214+0000 7fb75a9d5700 1 -- 172.21.15.12:0/2323166578 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb754102220 con 0x7fb7541015a0 2024-02-18T11:39:21.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:18.215+0000 7fb7517fa700 1 -- 172.21.15.12:0/2323166578 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 137296+0+0 (secure 0 0 0) 0x7fb748010910 con 0x7fb7541015a0 2024-02-18T11:39:21.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:18.216+0000 7fb7517fa700 1 --2- 172.21.15.12:0/2323166578 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fb73c0a28f0 0x7fb73c0a4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:21.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:18.216+0000 7fb753fff700 1 --2- 172.21.15.12:0/2323166578 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fb73c0a28f0 0x7fb73c0a4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:21.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:18.216+0000 7fb7517fa700 1 -- 172.21.15.12:0/2323166578 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb748014070 con 0x7fb7541015a0 2024-02-18T11:39:21.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:18.216+0000 7fb7517fa700 1 --2- 172.21.15.12:0/2323166578 >> [v2:172.21.15.188:6824/1135773350,v1:172.21.15.188:6825/1135773350] conn(0x7fb73c0ac0d0 0x7fb73c0ae4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:21.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:18.216+0000 7fb7517fa700 1 -- 172.21.15.12:0/2323166578 --> [v2:172.21.15.188:6824/1135773350,v1:172.21.15.188:6825/1135773350] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb73c0aeba0 con 0x7fb73c0ac0d0 2024-02-18T11:39:21.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:18.216+0000 7fb7517fa700 1 -- 172.21.15.12:0/2323166578 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb7480d0bd0 con 0x7fb7541015a0 2024-02-18T11:39:21.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:18.217+0000 7fb758f72700 1 --2- 172.21.15.12:0/2323166578 >> [v2:172.21.15.188:6824/1135773350,v1:172.21.15.188:6825/1135773350] conn(0x7fb73c0ac0d0 0x7fb73c0ae4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:21.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:18.217+0000 7fb753fff700 1 --2- 172.21.15.12:0/2323166578 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fb73c0a28f0 0x7fb73c0a4db0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb744005950 tx=0x7fb74400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:39:21.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:18.217+0000 7fb758f72700 1 --2- 172.21.15.12:0/2323166578 >> [v2:172.21.15.188:6824/1135773350,v1:172.21.15.188:6825/1135773350] conn(0x7fb73c0ac0d0 0x7fb73c0ae4f0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:39:21.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:18.218+0000 7fb7517fa700 1 -- 172.21.15.12:0/2323166578 <== osd.11 v2:172.21.15.188:6824/1135773350 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fb73c0aeba0 con 0x7fb73c0ac0d0 2024-02-18T11:39:21.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:18.235+0000 7fb75a9d5700 1 -- 172.21.15.12:0/2323166578 --> [v2:172.21.15.188:6824/1135773350,v1:172.21.15.188:6825/1135773350] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb754110f60 con 0x7fb73c0ac0d0 2024-02-18T11:39:21.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:18.236+0000 7fb7517fa700 1 -- 172.21.15.12:0/2323166578 <== osd.11 v2:172.21.15.188:6824/1135773350 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fb754110f60 con 0x7fb73c0ac0d0 2024-02-18T11:39:21.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:18.236+0000 7fb75a9d5700 1 -- 172.21.15.12:0/2323166578 >> [v2:172.21.15.188:6824/1135773350,v1:172.21.15.188:6825/1135773350] conn(0x7fb73c0ac0d0 msgr2=0x7fb73c0ae4f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:21.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:18.236+0000 7fb75a9d5700 1 --2- 172.21.15.12:0/2323166578 >> [v2:172.21.15.188:6824/1135773350,v1:172.21.15.188:6825/1135773350] conn(0x7fb73c0ac0d0 0x7fb73c0ae4f0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:21.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:18.236+0000 7fb75a9d5700 1 -- 172.21.15.12:0/2323166578 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fb73c0a28f0 msgr2=0x7fb73c0a4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:21.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:18.236+0000 7fb75a9d5700 1 --2- 172.21.15.12:0/2323166578 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fb73c0a28f0 0x7fb73c0a4db0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb744005950 tx=0x7fb74400b410 comp rx=0 tx=0).stop 2024-02-18T11:39:21.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:18.236+0000 7fb75a9d5700 1 -- 172.21.15.12:0/2323166578 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7541015a0 msgr2=0x7fb754198ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:21.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:18.236+0000 7fb75a9d5700 1 --2- 172.21.15.12:0/2323166578 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7541015a0 0x7fb754198ba0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb74800e9a0 tx=0x7fb74800ecb0 comp rx=0 tx=0).stop 2024-02-18T11:39:21.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:18.236+0000 7fb758f72700 1 -- 172.21.15.12:0/2323166578 reap_dead start 2024-02-18T11:39:21.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:18.236+0000 7fb75a9d5700 1 -- 172.21.15.12:0/2323166578 shutdown_connections 2024-02-18T11:39:21.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:18.236+0000 7fb75a9d5700 1 -- 172.21.15.12:0/2323166578 >> 172.21.15.12:0/2323166578 conn(0x7fb754076c00 msgr2=0x7fb7540feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:39:21.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:18.236+0000 7fb75a9d5700 1 -- 172.21.15.12:0/2323166578 shutdown_connections 2024-02-18T11:39:21.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:18.236+0000 7fb75a9d5700 1 -- 172.21.15.12:0/2323166578 wait complete. 2024-02-18T11:39:22.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:39:22 smithi012 ceph-mon[32460]: pgmap v302: 97 pgs: 97 active+clean; 639 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-02-18T11:39:22.223 INFO:teuthology.orchestra.run.smithi012.stdout:352187318292 2024-02-18T11:39:22.224 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.11 2024-02-18T11:39:22.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:39:22 smithi188 ceph-mon[36925]: pgmap v302: 97 pgs: 97 active+clean; 639 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-02-18T11:39:22.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:39:22 smithi158 ceph-mon[36626]: pgmap v302: 97 pgs: 97 active+clean; 639 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-02-18T11:39:22.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:22.837+0000 7f29ea130700 1 -- 172.21.15.12:0/1546357494 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29e410e6b0 msgr2=0x7f29e4110aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:22.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:22.837+0000 7f29ea130700 1 --2- 172.21.15.12:0/1546357494 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29e410e6b0 0x7f29e4110aa0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f29dc00b0a0 tx=0x7f29dc00b3b0 comp rx=0 tx=0).stop 2024-02-18T11:39:22.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:22.838+0000 7f29ea130700 1 -- 172.21.15.12:0/1546357494 shutdown_connections 2024-02-18T11:39:22.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:22.838+0000 7f29ea130700 1 --2- 172.21.15.12:0/1546357494 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29e410e6b0 0x7f29e4110aa0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:22.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:22.838+0000 7f29ea130700 1 --2- 172.21.15.12:0/1546357494 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29e4073ee0 0x7f29e40742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:22.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:22.838+0000 7f29ea130700 1 --2- 172.21.15.12:0/1546357494 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f29e4074800 0x7f29e410e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:22.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:22.838+0000 7f29ea130700 1 -- 172.21.15.12:0/1546357494 >> 172.21.15.12:0/1546357494 conn(0x7f29e4078de0 msgr2=0x7f29e40791f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:39:22.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:22.838+0000 7f29ea130700 1 -- 172.21.15.12:0/1546357494 shutdown_connections 2024-02-18T11:39:22.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:22.838+0000 7f29ea130700 1 -- 172.21.15.12:0/1546357494 wait complete. 2024-02-18T11:39:22.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:22.838+0000 7f29ea130700 1 Processor -- start 2024-02-18T11:39:22.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:22.838+0000 7f29ea130700 1 -- start start 2024-02-18T11:39:22.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:22.839+0000 7f29ea130700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29e4073ee0 0x7f29e419a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:22.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:22.839+0000 7f29ea130700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29e4074800 0x7f29e419a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:22.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:22.839+0000 7f29ea130700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f29e410e6b0 0x7f29e419ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:22.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:22.839+0000 7f29ea130700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e419f390 con 0x7f29e4073ee0 2024-02-18T11:39:22.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:22.839+0000 7f29ea130700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e419f500 con 0x7f29e410e6b0 2024-02-18T11:39:22.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:22.839+0000 7f29ea130700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e419f6c0 con 0x7f29e4074800 2024-02-18T11:39:22.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:22.839+0000 7f29e37fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29e4073ee0 0x7f29e419a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:22.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:22.839+0000 7f29e37fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29e4073ee0 0x7f29e419a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52732/0 (socket says 172.21.15.12:52732) 2024-02-18T11:39:22.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:22.839+0000 7f29e37fe700 1 -- 172.21.15.12:0/463337207 learned_addr learned my addr 172.21.15.12:0/463337207 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:39:22.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:22.839+0000 7f29e3fff700 1 --2- 172.21.15.12:0/463337207 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f29e410e6b0 0x7f29e419ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:22.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:22.839+0000 7f29e2ffd700 1 --2- 172.21.15.12:0/463337207 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29e4074800 0x7f29e419a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:22.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:22.839+0000 7f29e37fe700 1 -- 172.21.15.12:0/463337207 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29e4074800 msgr2=0x7f29e419a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:22.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:22.839+0000 7f29e37fe700 1 --2- 172.21.15.12:0/463337207 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29e4074800 0x7f29e419a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:22.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:22.839+0000 7f29e37fe700 1 -- 172.21.15.12:0/463337207 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f29e410e6b0 msgr2=0x7f29e419ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:22.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:22.839+0000 7f29e37fe700 1 --2- 172.21.15.12:0/463337207 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f29e410e6b0 0x7f29e419ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:22.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:22.839+0000 7f29e37fe700 1 -- 172.21.15.12:0/463337207 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29dc009d00 con 0x7f29e4073ee0 2024-02-18T11:39:22.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:22.839+0000 7f29e2ffd700 1 --2- 172.21.15.12:0/463337207 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29e4074800 0x7f29e419a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:39:22.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:22.839+0000 7f29e37fe700 1 --2- 172.21.15.12:0/463337207 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29e4073ee0 0x7f29e419a4b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f29d400c920 tx=0x7f29d400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:39:22.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:22.839+0000 7f29e0ff9700 1 -- 172.21.15.12:0/463337207 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d4007b00 con 0x7f29e4073ee0 2024-02-18T11:39:22.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:22.839+0000 7f29e0ff9700 1 -- 172.21.15.12:0/463337207 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f29d400ce70 con 0x7f29e4073ee0 2024-02-18T11:39:22.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:22.839+0000 7f29ea130700 1 -- 172.21.15.12:0/463337207 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29e419f9a0 con 0x7f29e4073ee0 2024-02-18T11:39:22.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:22.839+0000 7f29e0ff9700 1 -- 172.21.15.12:0/463337207 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d4005610 con 0x7f29e4073ee0 2024-02-18T11:39:22.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:22.839+0000 7f29ea130700 1 -- 172.21.15.12:0/463337207 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29e41a7200 con 0x7f29e4073ee0 2024-02-18T11:39:22.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:22.840+0000 7f29ea130700 1 -- 172.21.15.12:0/463337207 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f29e41141d0 con 0x7f29e4073ee0 2024-02-18T11:39:22.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:22.841+0000 7f29e0ff9700 1 -- 172.21.15.12:0/463337207 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 137296+0+0 (secure 0 0 0) 0x7f29d4004750 con 0x7f29e4073ee0 2024-02-18T11:39:22.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:22.842+0000 7f29e0ff9700 1 --2- 172.21.15.12:0/463337207 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f29cc0a28f0 0x7f29cc0a4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:22.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:22.842+0000 7f29e2ffd700 1 --2- 172.21.15.12:0/463337207 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f29cc0a28f0 0x7f29cc0a4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:22.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:22.842+0000 7f29e0ff9700 1 -- 172.21.15.12:0/463337207 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f29d40ce540 con 0x7f29e4073ee0 2024-02-18T11:39:22.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:22.842+0000 7f29e0ff9700 1 --2- 172.21.15.12:0/463337207 >> [v2:172.21.15.188:6808/3939495896,v1:172.21.15.188:6809/3939495896] conn(0x7f29cc0ac0d0 0x7f29cc0ae4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:22.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:22.842+0000 7f29e0ff9700 1 -- 172.21.15.12:0/463337207 --> [v2:172.21.15.188:6808/3939495896,v1:172.21.15.188:6809/3939495896] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f29cc0aeba0 con 0x7f29cc0ac0d0 2024-02-18T11:39:22.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:22.842+0000 7f29e0ff9700 1 -- 172.21.15.12:0/463337207 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29d40ce9e0 con 0x7f29e4073ee0 2024-02-18T11:39:22.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:22.842+0000 7f29e2ffd700 1 --2- 172.21.15.12:0/463337207 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f29cc0a28f0 0x7f29cc0a4db0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f29d8005e90 tx=0x7f29d8005d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:39:22.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:22.842+0000 7f29e3fff700 1 --2- 172.21.15.12:0/463337207 >> [v2:172.21.15.188:6808/3939495896,v1:172.21.15.188:6809/3939495896] conn(0x7f29cc0ac0d0 0x7f29cc0ae4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:22.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:22.843+0000 7f29e3fff700 1 --2- 172.21.15.12:0/463337207 >> [v2:172.21.15.188:6808/3939495896,v1:172.21.15.188:6809/3939495896] conn(0x7f29cc0ac0d0 0x7f29cc0ae4f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:39:22.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:22.844+0000 7f29e0ff9700 1 -- 172.21.15.12:0/463337207 <== osd.5 v2:172.21.15.188:6808/3939495896 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f29cc0aeba0 con 0x7f29cc0ac0d0 2024-02-18T11:39:22.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:22.860+0000 7f29ea130700 1 -- 172.21.15.12:0/463337207 --> [v2:172.21.15.188:6808/3939495896,v1:172.21.15.188:6809/3939495896] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f29e4062340 con 0x7f29cc0ac0d0 2024-02-18T11:39:22.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:22.861+0000 7f29e0ff9700 1 -- 172.21.15.12:0/463337207 <== osd.5 v2:172.21.15.188:6808/3939495896 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f29e4062340 con 0x7f29cc0ac0d0 2024-02-18T11:39:22.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:22.861+0000 7f29ea130700 1 -- 172.21.15.12:0/463337207 >> [v2:172.21.15.188:6808/3939495896,v1:172.21.15.188:6809/3939495896] conn(0x7f29cc0ac0d0 msgr2=0x7f29cc0ae4f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:22.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:22.861+0000 7f29ea130700 1 --2- 172.21.15.12:0/463337207 >> [v2:172.21.15.188:6808/3939495896,v1:172.21.15.188:6809/3939495896] conn(0x7f29cc0ac0d0 0x7f29cc0ae4f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:22.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:22.861+0000 7f29ea130700 1 -- 172.21.15.12:0/463337207 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f29cc0a28f0 msgr2=0x7f29cc0a4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:22.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:22.861+0000 7f29ea130700 1 --2- 172.21.15.12:0/463337207 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f29cc0a28f0 0x7f29cc0a4db0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f29d8005e90 tx=0x7f29d8005d20 comp rx=0 tx=0).stop 2024-02-18T11:39:22.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:22.861+0000 7f29ea130700 1 -- 172.21.15.12:0/463337207 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29e4073ee0 msgr2=0x7f29e419a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:22.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:22.861+0000 7f29ea130700 1 --2- 172.21.15.12:0/463337207 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29e4073ee0 0x7f29e419a4b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f29d400c920 tx=0x7f29d400cc30 comp rx=0 tx=0).stop 2024-02-18T11:39:22.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:22.861+0000 7f29e3fff700 1 -- 172.21.15.12:0/463337207 reap_dead start 2024-02-18T11:39:22.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:22.861+0000 7f29ea130700 1 -- 172.21.15.12:0/463337207 shutdown_connections 2024-02-18T11:39:22.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:22.861+0000 7f29ea130700 1 -- 172.21.15.12:0/463337207 >> 172.21.15.12:0/463337207 conn(0x7f29e4078de0 msgr2=0x7f29e40fef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:39:22.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:22.861+0000 7f29ea130700 1 -- 172.21.15.12:0/463337207 shutdown_connections 2024-02-18T11:39:22.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:22.861+0000 7f29ea130700 1 -- 172.21.15.12:0/463337207 wait complete. 2024-02-18T11:39:23.567 INFO:teuthology.orchestra.run.smithi012.stdout:55834574915 2024-02-18T11:39:23.567 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-02-18T11:39:24.336 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:39:24 smithi012 ceph-mon[32460]: pgmap v303: 97 pgs: 97 active+clean; 639 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 6 op/s 2024-02-18T11:39:24.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:39:24 smithi188 ceph-mon[36925]: pgmap v303: 97 pgs: 97 active+clean; 639 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 6 op/s 2024-02-18T11:39:24.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:39:24 smithi158 ceph-mon[36626]: pgmap v303: 97 pgs: 97 active+clean; 639 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 6 op/s 2024-02-18T11:39:24.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.500+0000 7fb0706e0700 1 -- 172.21.15.12:0/2805203790 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb068101ef0 msgr2=0x7fb0681022d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:24.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.500+0000 7fb0706e0700 1 --2- 172.21.15.12:0/2805203790 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb068101ef0 0x7fb0681022d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb05800b0a0 tx=0x7fb05800b3b0 comp rx=0 tx=0).stop 2024-02-18T11:39:24.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.500+0000 7fb0706e0700 1 -- 172.21.15.12:0/2805203790 shutdown_connections 2024-02-18T11:39:24.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.500+0000 7fb0706e0700 1 --2- 172.21.15.12:0/2805203790 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb068106fd0 0x7fb06810eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:24.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.500+0000 7fb0706e0700 1 --2- 172.21.15.12:0/2805203790 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb068101ef0 0x7fb0681022d0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:24.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.500+0000 7fb0706e0700 1 --2- 172.21.15.12:0/2805203790 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb0681028a0 0x7fb0681068f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:24.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.500+0000 7fb0706e0700 1 -- 172.21.15.12:0/2805203790 >> 172.21.15.12:0/2805203790 conn(0x7fb0680fd760 msgr2=0x7fb0680ffb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:39:24.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.500+0000 7fb0706e0700 1 -- 172.21.15.12:0/2805203790 shutdown_connections 2024-02-18T11:39:24.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.500+0000 7fb0706e0700 1 -- 172.21.15.12:0/2805203790 wait complete. 2024-02-18T11:39:24.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.501+0000 7fb0706e0700 1 Processor -- start 2024-02-18T11:39:24.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.501+0000 7fb0706e0700 1 -- start start 2024-02-18T11:39:24.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.501+0000 7fb0706e0700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb068101ef0 0x7fb068198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:24.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.501+0000 7fb0706e0700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0681028a0 0x7fb068198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:24.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.501+0000 7fb0706e0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb068106fd0 0x7fb06819cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:24.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.501+0000 7fb0706e0700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb06819d3a0 con 0x7fb068106fd0 2024-02-18T11:39:24.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.501+0000 7fb0706e0700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb06819d510 con 0x7fb068101ef0 2024-02-18T11:39:24.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.501+0000 7fb0706e0700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb06819d6d0 con 0x7fb0681028a0 2024-02-18T11:39:24.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.501+0000 7fb06ec7d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb068106fd0 0x7fb06819cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:24.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.501+0000 7fb06ec7d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb068106fd0 0x7fb06819cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52758/0 (socket says 172.21.15.12:52758) 2024-02-18T11:39:24.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.501+0000 7fb06ec7d700 1 -- 172.21.15.12:0/2789285692 learned_addr learned my addr 172.21.15.12:0/2789285692 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:39:24.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.501+0000 7fb06e47c700 1 --2- 172.21.15.12:0/2789285692 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb068101ef0 0x7fb068198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:24.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.501+0000 7fb06dc7b700 1 --2- 172.21.15.12:0/2789285692 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0681028a0 0x7fb068198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:24.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.501+0000 7fb06ec7d700 1 -- 172.21.15.12:0/2789285692 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0681028a0 msgr2=0x7fb068198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:24.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.501+0000 7fb06ec7d700 1 --2- 172.21.15.12:0/2789285692 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0681028a0 0x7fb068198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:24.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.501+0000 7fb06ec7d700 1 -- 172.21.15.12:0/2789285692 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb068101ef0 msgr2=0x7fb068198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:24.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.501+0000 7fb06ec7d700 1 --2- 172.21.15.12:0/2789285692 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb068101ef0 0x7fb068198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:24.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.501+0000 7fb06ec7d700 1 -- 172.21.15.12:0/2789285692 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb058009d00 con 0x7fb068106fd0 2024-02-18T11:39:24.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.502+0000 7fb06ec7d700 1 --2- 172.21.15.12:0/2789285692 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb068106fd0 0x7fb06819cd90 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb06000b550 tx=0x7fb06000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:39:24.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.502+0000 7fb0677fe700 1 -- 172.21.15.12:0/2789285692 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb060011830 con 0x7fb068106fd0 2024-02-18T11:39:24.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.502+0000 7fb0706e0700 1 -- 172.21.15.12:0/2789285692 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb06819d9b0 con 0x7fb068106fd0 2024-02-18T11:39:24.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.502+0000 7fb0677fe700 1 -- 172.21.15.12:0/2789285692 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb060011e70 con 0x7fb068106fd0 2024-02-18T11:39:24.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.502+0000 7fb0677fe700 1 -- 172.21.15.12:0/2789285692 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb06000f2e0 con 0x7fb068106fd0 2024-02-18T11:39:24.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.502+0000 7fb0706e0700 1 -- 172.21.15.12:0/2789285692 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0681a5210 con 0x7fb068106fd0 2024-02-18T11:39:24.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.502+0000 7fb0706e0700 1 -- 172.21.15.12:0/2789285692 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb0681033d0 con 0x7fb068106fd0 2024-02-18T11:39:24.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.504+0000 7fb0677fe700 1 -- 172.21.15.12:0/2789285692 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 137296+0+0 (secure 0 0 0) 0x7fb060011990 con 0x7fb068106fd0 2024-02-18T11:39:24.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.504+0000 7fb0677fe700 1 --2- 172.21.15.12:0/2789285692 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fb0500a2870 0x7fb0500a4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:24.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.505+0000 7fb06e47c700 1 --2- 172.21.15.12:0/2789285692 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fb0500a2870 0x7fb0500a4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:24.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.505+0000 7fb0677fe700 1 -- 172.21.15.12:0/2789285692 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb0600ce470 con 0x7fb068106fd0 2024-02-18T11:39:24.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.505+0000 7fb0677fe700 1 --2- 172.21.15.12:0/2789285692 >> [v2:172.21.15.158:6808/3892820997,v1:172.21.15.158:6809/3892820997] conn(0x7fb0500ac050 0x7fb0500ae470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:24.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.505+0000 7fb0677fe700 1 -- 172.21.15.12:0/2789285692 --> [v2:172.21.15.158:6808/3892820997,v1:172.21.15.158:6809/3892820997] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb0500aeb20 con 0x7fb0500ac050 2024-02-18T11:39:24.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.505+0000 7fb0677fe700 1 -- 172.21.15.12:0/2789285692 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0600ce910 con 0x7fb068106fd0 2024-02-18T11:39:24.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.505+0000 7fb06e47c700 1 --2- 172.21.15.12:0/2789285692 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fb0500a2870 0x7fb0500a4d30 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb058006130 tx=0x7fb058006080 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:39:24.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.505+0000 7fb06dc7b700 1 --2- 172.21.15.12:0/2789285692 >> [v2:172.21.15.158:6808/3892820997,v1:172.21.15.158:6809/3892820997] conn(0x7fb0500ac050 0x7fb0500ae470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:24.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.506+0000 7fb06dc7b700 1 --2- 172.21.15.12:0/2789285692 >> [v2:172.21.15.158:6808/3892820997,v1:172.21.15.158:6809/3892820997] conn(0x7fb0500ac050 0x7fb0500ae470 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:39:24.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.506+0000 7fb0677fe700 1 -- 172.21.15.12:0/2789285692 <== osd.4 v2:172.21.15.158:6808/3892820997 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fb0500aeb20 con 0x7fb0500ac050 2024-02-18T11:39:24.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.523+0000 7fb0706e0700 1 -- 172.21.15.12:0/2789285692 --> [v2:172.21.15.158:6808/3892820997,v1:172.21.15.158:6809/3892820997] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb068112130 con 0x7fb0500ac050 2024-02-18T11:39:24.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.523+0000 7fb0677fe700 1 -- 172.21.15.12:0/2789285692 <== osd.4 v2:172.21.15.158:6808/3892820997 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fb068112130 con 0x7fb0500ac050 2024-02-18T11:39:24.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.524+0000 7fb0706e0700 1 -- 172.21.15.12:0/2789285692 >> [v2:172.21.15.158:6808/3892820997,v1:172.21.15.158:6809/3892820997] conn(0x7fb0500ac050 msgr2=0x7fb0500ae470 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:24.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.524+0000 7fb0706e0700 1 --2- 172.21.15.12:0/2789285692 >> [v2:172.21.15.158:6808/3892820997,v1:172.21.15.158:6809/3892820997] conn(0x7fb0500ac050 0x7fb0500ae470 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:24.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.524+0000 7fb0706e0700 1 -- 172.21.15.12:0/2789285692 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fb0500a2870 msgr2=0x7fb0500a4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:24.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.524+0000 7fb0706e0700 1 --2- 172.21.15.12:0/2789285692 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fb0500a2870 0x7fb0500a4d30 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb058006130 tx=0x7fb058006080 comp rx=0 tx=0).stop 2024-02-18T11:39:24.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.524+0000 7fb0706e0700 1 -- 172.21.15.12:0/2789285692 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb068106fd0 msgr2=0x7fb06819cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:24.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.524+0000 7fb0706e0700 1 --2- 172.21.15.12:0/2789285692 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb068106fd0 0x7fb06819cd90 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb06000b550 tx=0x7fb06000b910 comp rx=0 tx=0).stop 2024-02-18T11:39:24.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.524+0000 7fb06ec7d700 1 -- 172.21.15.12:0/2789285692 reap_dead start 2024-02-18T11:39:24.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.524+0000 7fb0706e0700 1 -- 172.21.15.12:0/2789285692 shutdown_connections 2024-02-18T11:39:24.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.524+0000 7fb0706e0700 1 -- 172.21.15.12:0/2789285692 >> 172.21.15.12:0/2789285692 conn(0x7fb0680fd760 msgr2=0x7fb0680feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:39:24.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.524+0000 7fb0706e0700 1 -- 172.21.15.12:0/2789285692 shutdown_connections 2024-02-18T11:39:24.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.524+0000 7fb0706e0700 1 -- 172.21.15.12:0/2789285692 wait complete. 2024-02-18T11:39:24.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.691+0000 7f0328f66700 1 -- 172.21.15.12:0/780371613 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03241028a0 msgr2=0x7f03241068f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:24.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.691+0000 7f0328f66700 1 --2- 172.21.15.12:0/780371613 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03241028a0 0x7f03241068f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f0318009990 tx=0x7f0318009ca0 comp rx=0 tx=0).stop 2024-02-18T11:39:24.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.691+0000 7f0328f66700 1 -- 172.21.15.12:0/780371613 shutdown_connections 2024-02-18T11:39:24.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.691+0000 7f0328f66700 1 --2- 172.21.15.12:0/780371613 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0324106fd0 0x7f032410eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:24.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.691+0000 7f0328f66700 1 --2- 172.21.15.12:0/780371613 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0324101ef0 0x7f03241022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:24.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.691+0000 7f0328f66700 1 --2- 172.21.15.12:0/780371613 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03241028a0 0x7f03241068f0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:24.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.691+0000 7f0328f66700 1 -- 172.21.15.12:0/780371613 >> 172.21.15.12:0/780371613 conn(0x7f03240fd760 msgr2=0x7f03240ffb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:39:24.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.691+0000 7f0328f66700 1 -- 172.21.15.12:0/780371613 shutdown_connections 2024-02-18T11:39:24.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.691+0000 7f0328f66700 1 -- 172.21.15.12:0/780371613 wait complete. 2024-02-18T11:39:24.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.692+0000 7f0328f66700 1 Processor -- start 2024-02-18T11:39:24.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.692+0000 7f0328f66700 1 -- start start 2024-02-18T11:39:24.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.692+0000 7f0328f66700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0324101ef0 0x7f0324073fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:24.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.692+0000 7f0328f66700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03241028a0 0x7f03240744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:24.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.692+0000 7f0328f66700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0324106fd0 0x7f0324074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:24.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.692+0000 7f0328f66700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03241a66d0 con 0x7f0324101ef0 2024-02-18T11:39:24.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.692+0000 7f0328f66700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03241a6840 con 0x7f0324106fd0 2024-02-18T11:39:24.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.692+0000 7f0328f66700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03241a6a00 con 0x7f03241028a0 2024-02-18T11:39:24.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.692+0000 7f032259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0324101ef0 0x7f0324073fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:24.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.692+0000 7f032259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0324101ef0 0x7f0324073fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52778/0 (socket says 172.21.15.12:52778) 2024-02-18T11:39:24.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.692+0000 7f032259c700 1 -- 172.21.15.12:0/2684040157 learned_addr learned my addr 172.21.15.12:0/2684040157 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:39:24.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.693+0000 7f0321d9b700 1 --2- 172.21.15.12:0/2684040157 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03241028a0 0x7f03240744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:24.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.693+0000 7f032259c700 1 -- 172.21.15.12:0/2684040157 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03241028a0 msgr2=0x7f03240744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:24.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.693+0000 7f0322d9d700 1 --2- 172.21.15.12:0/2684040157 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0324106fd0 0x7f0324074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:24.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.693+0000 7f032259c700 1 --2- 172.21.15.12:0/2684040157 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03241028a0 0x7f03240744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:24.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.693+0000 7f032259c700 1 -- 172.21.15.12:0/2684040157 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0324106fd0 msgr2=0x7f0324074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:24.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.693+0000 7f032259c700 1 --2- 172.21.15.12:0/2684040157 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0324106fd0 0x7f0324074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:24.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.693+0000 7f032259c700 1 -- 172.21.15.12:0/2684040157 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03180095f0 con 0x7f0324101ef0 2024-02-18T11:39:24.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.693+0000 7f032259c700 1 --2- 172.21.15.12:0/2684040157 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0324101ef0 0x7f0324073fb0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f031400b530 tx=0x7f031400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:39:24.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.693+0000 7f03137fe700 1 -- 172.21.15.12:0/2684040157 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0314011830 con 0x7f0324101ef0 2024-02-18T11:39:24.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.693+0000 7f03137fe700 1 -- 172.21.15.12:0/2684040157 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0314011e70 con 0x7f0324101ef0 2024-02-18T11:39:24.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.693+0000 7f03137fe700 1 -- 172.21.15.12:0/2684040157 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f031400f2e0 con 0x7f0324101ef0 2024-02-18T11:39:24.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.693+0000 7f0328f66700 1 -- 172.21.15.12:0/2684040157 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03241a6ce0 con 0x7f0324101ef0 2024-02-18T11:39:24.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.693+0000 7f0328f66700 1 -- 172.21.15.12:0/2684040157 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03241a7190 con 0x7f0324101ef0 2024-02-18T11:39:24.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.694+0000 7f0328f66700 1 -- 172.21.15.12:0/2684040157 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0324112090 con 0x7f0324101ef0 2024-02-18T11:39:24.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.695+0000 7f03137fe700 1 -- 172.21.15.12:0/2684040157 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 137296+0+0 (secure 0 0 0) 0x7f03140105f0 con 0x7f0324101ef0 2024-02-18T11:39:24.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.696+0000 7f03137fe700 1 --2- 172.21.15.12:0/2684040157 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f030c0a2870 0x7f030c0a4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:24.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.696+0000 7f0321d9b700 1 --2- 172.21.15.12:0/2684040157 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f030c0a2870 0x7f030c0a4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:24.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.696+0000 7f03137fe700 1 -- 172.21.15.12:0/2684040157 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f03140ce580 con 0x7f0324101ef0 2024-02-18T11:39:24.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.696+0000 7f03137fe700 1 --2- 172.21.15.12:0/2684040157 >> [v2:172.21.15.12:6810/3758512503,v1:172.21.15.12:6811/3758512503] conn(0x7f030c0ac050 0x7f030c0ae470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:24.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.696+0000 7f03137fe700 1 -- 172.21.15.12:0/2684040157 --> [v2:172.21.15.12:6810/3758512503,v1:172.21.15.12:6811/3758512503] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f030c0aeb20 con 0x7f030c0ac050 2024-02-18T11:39:24.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.696+0000 7f03137fe700 1 -- 172.21.15.12:0/2684040157 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f03140ce8f0 con 0x7f0324101ef0 2024-02-18T11:39:24.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.696+0000 7f0322d9d700 1 --2- 172.21.15.12:0/2684040157 >> [v2:172.21.15.12:6810/3758512503,v1:172.21.15.12:6811/3758512503] conn(0x7f030c0ac050 0x7f030c0ae470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:24.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.696+0000 7f0321d9b700 1 --2- 172.21.15.12:0/2684040157 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f030c0a2870 0x7f030c0a4d30 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f031800b5c0 tx=0x7f0318005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:39:24.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.697+0000 7f0322d9d700 1 --2- 172.21.15.12:0/2684040157 >> [v2:172.21.15.12:6810/3758512503,v1:172.21.15.12:6811/3758512503] conn(0x7f030c0ac050 0x7f030c0ae470 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:39:24.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.697+0000 7f03137fe700 1 -- 172.21.15.12:0/2684040157 <== osd.3 v2:172.21.15.12:6810/3758512503 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f030c0aeb20 con 0x7f030c0ac050 2024-02-18T11:39:24.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.714+0000 7f0328f66700 1 -- 172.21.15.12:0/2684040157 --> [v2:172.21.15.12:6810/3758512503,v1:172.21.15.12:6811/3758512503] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f03240623c0 con 0x7f030c0ac050 2024-02-18T11:39:24.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.714+0000 7f03137fe700 1 -- 172.21.15.12:0/2684040157 <== osd.3 v2:172.21.15.12:6810/3758512503 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f03240623c0 con 0x7f030c0ac050 2024-02-18T11:39:24.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.714+0000 7f0328f66700 1 -- 172.21.15.12:0/2684040157 >> [v2:172.21.15.12:6810/3758512503,v1:172.21.15.12:6811/3758512503] conn(0x7f030c0ac050 msgr2=0x7f030c0ae470 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:24.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.714+0000 7f0328f66700 1 --2- 172.21.15.12:0/2684040157 >> [v2:172.21.15.12:6810/3758512503,v1:172.21.15.12:6811/3758512503] conn(0x7f030c0ac050 0x7f030c0ae470 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:24.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.714+0000 7f0328f66700 1 -- 172.21.15.12:0/2684040157 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f030c0a2870 msgr2=0x7f030c0a4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:24.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.715+0000 7f0328f66700 1 --2- 172.21.15.12:0/2684040157 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f030c0a2870 0x7f030c0a4d30 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f031800b5c0 tx=0x7f0318005fb0 comp rx=0 tx=0).stop 2024-02-18T11:39:24.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.715+0000 7f0328f66700 1 -- 172.21.15.12:0/2684040157 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0324101ef0 msgr2=0x7f0324073fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:24.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.715+0000 7f0328f66700 1 --2- 172.21.15.12:0/2684040157 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0324101ef0 0x7f0324073fb0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f031400b530 tx=0x7f031400b8f0 comp rx=0 tx=0).stop 2024-02-18T11:39:24.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.715+0000 7f0322d9d700 1 -- 172.21.15.12:0/2684040157 reap_dead start 2024-02-18T11:39:24.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.715+0000 7f0328f66700 1 -- 172.21.15.12:0/2684040157 shutdown_connections 2024-02-18T11:39:24.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.715+0000 7f0328f66700 1 -- 172.21.15.12:0/2684040157 >> 172.21.15.12:0/2684040157 conn(0x7f03240fd760 msgr2=0x7f03240fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:39:24.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.715+0000 7f0328f66700 1 -- 172.21.15.12:0/2684040157 shutdown_connections 2024-02-18T11:39:24.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:24.715+0000 7f0328f66700 1 -- 172.21.15.12:0/2684040157 wait complete. 2024-02-18T11:39:25.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:25.171+0000 7f69d0c2c700 1 -- 172.21.15.12:0/2698965664 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f69cc069960 msgr2=0x7f69cc069dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:25.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:25.171+0000 7f69d0c2c700 1 --2- 172.21.15.12:0/2698965664 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f69cc069960 0x7f69cc069dc0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f69b40099c0 tx=0x7f69b4009cd0 comp rx=0 tx=0).stop 2024-02-18T11:39:25.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:25.171+0000 7f69d0c2c700 1 -- 172.21.15.12:0/2698965664 shutdown_connections 2024-02-18T11:39:25.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:25.171+0000 7f69d0c2c700 1 --2- 172.21.15.12:0/2698965664 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69cc06a300 0x7f69cc10eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:25.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:25.171+0000 7f69d0c2c700 1 --2- 172.21.15.12:0/2698965664 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f69cc107d00 0x7f69cc1080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:25.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:25.171+0000 7f69d0c2c700 1 --2- 172.21.15.12:0/2698965664 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f69cc069960 0x7f69cc069dc0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:25.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:25.171+0000 7f69d0c2c700 1 -- 172.21.15.12:0/2698965664 >> 172.21.15.12:0/2698965664 conn(0x7f69cc077ae0 msgr2=0x7f69cc077ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:39:25.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:25.171+0000 7f69d0c2c700 1 -- 172.21.15.12:0/2698965664 shutdown_connections 2024-02-18T11:39:25.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:25.171+0000 7f69d0c2c700 1 -- 172.21.15.12:0/2698965664 wait complete. 2024-02-18T11:39:25.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:25.172+0000 7f69d0c2c700 1 Processor -- start 2024-02-18T11:39:25.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:25.172+0000 7f69d0c2c700 1 -- start start 2024-02-18T11:39:25.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:25.172+0000 7f69d0c2c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f69cc069960 0x7f69cc1981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:25.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:25.172+0000 7f69d0c2c700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f69cc06a300 0x7f69cc1986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:25.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:25.172+0000 7f69d0c2c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69cc107d00 0x7f69cc19ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:25.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:25.172+0000 7f69d0c2c700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69cc19d080 con 0x7f69cc069960 2024-02-18T11:39:25.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:25.172+0000 7f69d0c2c700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69cc19d1f0 con 0x7f69cc06a300 2024-02-18T11:39:25.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:25.172+0000 7f69d0c2c700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69cc19d3b0 con 0x7f69cc107d00 2024-02-18T11:39:25.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:25.172+0000 7f69ca59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f69cc069960 0x7f69cc1981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:25.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:25.172+0000 7f69ca59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f69cc069960 0x7f69cc1981a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52794/0 (socket says 172.21.15.12:52794) 2024-02-18T11:39:25.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:25.172+0000 7f69ca59c700 1 -- 172.21.15.12:0/1827110306 learned_addr learned my addr 172.21.15.12:0/1827110306 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:39:25.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:25.172+0000 7f69c9d9b700 1 --2- 172.21.15.12:0/1827110306 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f69cc06a300 0x7f69cc1986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:25.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:25.172+0000 7f69cad9d700 1 --2- 172.21.15.12:0/1827110306 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69cc107d00 0x7f69cc19ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:25.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:25.172+0000 7f69ca59c700 1 -- 172.21.15.12:0/1827110306 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69cc107d00 msgr2=0x7f69cc19ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:25.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:25.172+0000 7f69ca59c700 1 --2- 172.21.15.12:0/1827110306 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69cc107d00 0x7f69cc19ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:25.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:25.172+0000 7f69ca59c700 1 -- 172.21.15.12:0/1827110306 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f69cc06a300 msgr2=0x7f69cc1986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:25.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:25.172+0000 7f69ca59c700 1 --2- 172.21.15.12:0/1827110306 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f69cc06a300 0x7f69cc1986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:25.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:25.172+0000 7f69ca59c700 1 -- 172.21.15.12:0/1827110306 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69b40095f0 con 0x7f69cc069960 2024-02-18T11:39:25.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:25.172+0000 7f69cad9d700 1 --2- 172.21.15.12:0/1827110306 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69cc107d00 0x7f69cc19ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:39:25.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:25.173+0000 7f69ca59c700 1 --2- 172.21.15.12:0/1827110306 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f69cc069960 0x7f69cc1981a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f69bc00ada0 tx=0x7f69bc00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:39:25.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:25.173+0000 7f69c37fe700 1 -- 172.21.15.12:0/1827110306 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69bc00f070 con 0x7f69cc069960 2024-02-18T11:39:25.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:25.173+0000 7f69c37fe700 1 -- 172.21.15.12:0/1827110306 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f69bc00ae90 con 0x7f69cc069960 2024-02-18T11:39:25.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:25.173+0000 7f69d0c2c700 1 -- 172.21.15.12:0/1827110306 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69cc19d690 con 0x7f69cc069960 2024-02-18T11:39:25.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:25.173+0000 7f69c37fe700 1 -- 172.21.15.12:0/1827110306 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69bc005480 con 0x7f69cc069960 2024-02-18T11:39:25.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:25.173+0000 7f69d0c2c700 1 -- 172.21.15.12:0/1827110306 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69cc1a4ef0 con 0x7f69cc069960 2024-02-18T11:39:25.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:25.173+0000 7f69d0c2c700 1 -- 172.21.15.12:0/1827110306 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f69cc112000 con 0x7f69cc069960 2024-02-18T11:39:25.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:25.175+0000 7f69c37fe700 1 -- 172.21.15.12:0/1827110306 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 137296+0+0 (secure 0 0 0) 0x7f69bc004ad0 con 0x7f69cc069960 2024-02-18T11:39:25.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:25.175+0000 7f69c37fe700 1 --2- 172.21.15.12:0/1827110306 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f69b80a2870 0x7f69b80a4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:25.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:25.176+0000 7f69c9d9b700 1 --2- 172.21.15.12:0/1827110306 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f69b80a2870 0x7f69b80a4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:25.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:25.176+0000 7f69c37fe700 1 -- 172.21.15.12:0/1827110306 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f69bc0ce620 con 0x7f69cc069960 2024-02-18T11:39:25.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:25.176+0000 7f69c37fe700 1 --2- 172.21.15.12:0/1827110306 >> [v2:172.21.15.12:6826/1825928986,v1:172.21.15.12:6827/1825928986] conn(0x7f69b80ac050 0x7f69b80ae470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:25.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:25.176+0000 7f69c37fe700 1 -- 172.21.15.12:0/1827110306 --> [v2:172.21.15.12:6826/1825928986,v1:172.21.15.12:6827/1825928986] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f69b80aeb20 con 0x7f69b80ac050 2024-02-18T11:39:25.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:25.176+0000 7f69c37fe700 1 -- 172.21.15.12:0/1827110306 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f69bc0ce990 con 0x7f69cc069960 2024-02-18T11:39:25.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:25.176+0000 7f69cad9d700 1 --2- 172.21.15.12:0/1827110306 >> [v2:172.21.15.12:6826/1825928986,v1:172.21.15.12:6827/1825928986] conn(0x7f69b80ac050 0x7f69b80ae470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:25.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:25.176+0000 7f69c9d9b700 1 --2- 172.21.15.12:0/1827110306 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f69b80a2870 0x7f69b80a4d30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f69b4009990 tx=0x7f69b40036e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:39:25.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:25.176+0000 7f69cad9d700 1 --2- 172.21.15.12:0/1827110306 >> [v2:172.21.15.12:6826/1825928986,v1:172.21.15.12:6827/1825928986] conn(0x7f69b80ac050 0x7f69b80ae470 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:39:25.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:25.177+0000 7f69c37fe700 1 -- 172.21.15.12:0/1827110306 <== osd.9 v2:172.21.15.12:6826/1825928986 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f69b80aeb20 con 0x7f69b80ac050 2024-02-18T11:39:25.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:25.193+0000 7f69d0c2c700 1 -- 172.21.15.12:0/1827110306 --> [v2:172.21.15.12:6826/1825928986,v1:172.21.15.12:6827/1825928986] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f69cc0623c0 con 0x7f69b80ac050 2024-02-18T11:39:25.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:25.194+0000 7f69c37fe700 1 -- 172.21.15.12:0/1827110306 <== osd.9 v2:172.21.15.12:6826/1825928986 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f69cc0623c0 con 0x7f69b80ac050 2024-02-18T11:39:25.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:25.194+0000 7f69d0c2c700 1 -- 172.21.15.12:0/1827110306 >> [v2:172.21.15.12:6826/1825928986,v1:172.21.15.12:6827/1825928986] conn(0x7f69b80ac050 msgr2=0x7f69b80ae470 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:25.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:25.194+0000 7f69d0c2c700 1 --2- 172.21.15.12:0/1827110306 >> [v2:172.21.15.12:6826/1825928986,v1:172.21.15.12:6827/1825928986] conn(0x7f69b80ac050 0x7f69b80ae470 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:25.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:25.194+0000 7f69d0c2c700 1 -- 172.21.15.12:0/1827110306 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f69b80a2870 msgr2=0x7f69b80a4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:25.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:25.194+0000 7f69d0c2c700 1 --2- 172.21.15.12:0/1827110306 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f69b80a2870 0x7f69b80a4d30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f69b4009990 tx=0x7f69b40036e0 comp rx=0 tx=0).stop 2024-02-18T11:39:25.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:25.194+0000 7f69d0c2c700 1 -- 172.21.15.12:0/1827110306 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f69cc069960 msgr2=0x7f69cc1981a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:25.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:25.194+0000 7f69d0c2c700 1 --2- 172.21.15.12:0/1827110306 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f69cc069960 0x7f69cc1981a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f69bc00ada0 tx=0x7f69bc00c330 comp rx=0 tx=0).stop 2024-02-18T11:39:25.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:25.194+0000 7f69cad9d700 1 -- 172.21.15.12:0/1827110306 reap_dead start 2024-02-18T11:39:25.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:25.194+0000 7f69d0c2c700 1 -- 172.21.15.12:0/1827110306 shutdown_connections 2024-02-18T11:39:25.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:25.194+0000 7f69d0c2c700 1 -- 172.21.15.12:0/1827110306 >> 172.21.15.12:0/1827110306 conn(0x7f69cc077ae0 msgr2=0x7f69cc0ff7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:39:25.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:25.194+0000 7f69d0c2c700 1 -- 172.21.15.12:0/1827110306 shutdown_connections 2024-02-18T11:39:25.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:25.194+0000 7f69d0c2c700 1 -- 172.21.15.12:0/1827110306 wait complete. 2024-02-18T11:39:25.627 INFO:teuthology.orchestra.run.smithi012.stdout:150323855416 2024-02-18T11:39:25.627 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-02-18T11:39:26.047 INFO:teuthology.orchestra.run.smithi012.stdout:171798691891 2024-02-18T11:39:26.047 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-02-18T11:39:26.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:39:26 smithi188 ceph-mon[36925]: pgmap v304: 97 pgs: 97 active+clean; 639 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-02-18T11:39:26.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:39:26 smithi012 ceph-mon[32460]: pgmap v304: 97 pgs: 97 active+clean; 639 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-02-18T11:39:26.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:39:26 smithi158 ceph-mon[36626]: pgmap v304: 97 pgs: 97 active+clean; 639 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-02-18T11:39:27.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:26.866+0000 7efd28c05700 1 -- 172.21.15.12:0/391581679 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd24106d20 msgr2=0x7efd24107100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:27.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:26.866+0000 7efd28c05700 1 --2- 172.21.15.12:0/391581679 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd24106d20 0x7efd24107100 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7efd1400b0a0 tx=0x7efd1400b3b0 comp rx=0 tx=0).stop 2024-02-18T11:39:27.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:26.866+0000 7efd28c05700 1 -- 172.21.15.12:0/391581679 shutdown_connections 2024-02-18T11:39:27.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:26.866+0000 7efd28c05700 1 --2- 172.21.15.12:0/391581679 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efd24100c60 0x7efd2410ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:27.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:26.866+0000 7efd28c05700 1 --2- 172.21.15.12:0/391581679 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd24106d20 0x7efd24107100 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:27.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:26.866+0000 7efd28c05700 1 --2- 172.21.15.12:0/391581679 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd241002a0 0x7efd24100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:27.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:26.866+0000 7efd28c05700 1 -- 172.21.15.12:0/391581679 >> 172.21.15.12:0/391581679 conn(0x7efd240771b0 msgr2=0x7efd240fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:39:27.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:26.866+0000 7efd28c05700 1 -- 172.21.15.12:0/391581679 shutdown_connections 2024-02-18T11:39:27.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:26.866+0000 7efd28c05700 1 -- 172.21.15.12:0/391581679 wait complete. 2024-02-18T11:39:27.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:26.867+0000 7efd28c05700 1 Processor -- start 2024-02-18T11:39:27.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:26.867+0000 7efd28c05700 1 -- start start 2024-02-18T11:39:27.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:26.867+0000 7efd28c05700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd241002a0 0x7efd24101480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:27.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:26.867+0000 7efd28c05700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efd24100c60 0x7efd241019c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:27.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:26.867+0000 7efd28c05700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd24106d20 0x7efd24101f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:27.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:26.867+0000 7efd28c05700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd241a6910 con 0x7efd241002a0 2024-02-18T11:39:27.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:26.867+0000 7efd28c05700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd241a6a80 con 0x7efd24100c60 2024-02-18T11:39:27.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:26.867+0000 7efd28c05700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd241a6c40 con 0x7efd24106d20 2024-02-18T11:39:27.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:26.867+0000 7efd2259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd241002a0 0x7efd24101480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:27.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:26.867+0000 7efd2259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd241002a0 0x7efd24101480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52806/0 (socket says 172.21.15.12:52806) 2024-02-18T11:39:27.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:26.867+0000 7efd2259c700 1 -- 172.21.15.12:0/365960980 learned_addr learned my addr 172.21.15.12:0/365960980 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:39:27.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:26.867+0000 7efd22d9d700 1 --2- 172.21.15.12:0/365960980 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd24106d20 0x7efd24101f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:27.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:26.867+0000 7efd21d9b700 1 --2- 172.21.15.12:0/365960980 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efd24100c60 0x7efd241019c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:27.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:26.867+0000 7efd2259c700 1 -- 172.21.15.12:0/365960980 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd24106d20 msgr2=0x7efd24101f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:27.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:26.867+0000 7efd2259c700 1 --2- 172.21.15.12:0/365960980 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd24106d20 0x7efd24101f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:27.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:26.867+0000 7efd2259c700 1 -- 172.21.15.12:0/365960980 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efd24100c60 msgr2=0x7efd241019c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:27.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:26.867+0000 7efd2259c700 1 --2- 172.21.15.12:0/365960980 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efd24100c60 0x7efd241019c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:27.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:26.867+0000 7efd2259c700 1 -- 172.21.15.12:0/365960980 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd14009d00 con 0x7efd241002a0 2024-02-18T11:39:27.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:26.867+0000 7efd22d9d700 1 --2- 172.21.15.12:0/365960980 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd24106d20 0x7efd24101f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:39:27.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:26.867+0000 7efd21d9b700 1 --2- 172.21.15.12:0/365960980 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efd24100c60 0x7efd241019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:39:27.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:26.868+0000 7efd2259c700 1 --2- 172.21.15.12:0/365960980 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd241002a0 0x7efd24101480 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7efd14006070 tx=0x7efd1400cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:39:27.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:26.868+0000 7efd1b7fe700 1 -- 172.21.15.12:0/365960980 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd1400e040 con 0x7efd241002a0 2024-02-18T11:39:27.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:26.868+0000 7efd28c05700 1 -- 172.21.15.12:0/365960980 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd241a6ec0 con 0x7efd241002a0 2024-02-18T11:39:27.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:26.868+0000 7efd1b7fe700 1 -- 172.21.15.12:0/365960980 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efd14007a70 con 0x7efd241002a0 2024-02-18T11:39:27.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:26.868+0000 7efd1b7fe700 1 -- 172.21.15.12:0/365960980 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd1401b6b0 con 0x7efd241002a0 2024-02-18T11:39:27.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:26.868+0000 7efd28c05700 1 -- 172.21.15.12:0/365960980 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd241a72d0 con 0x7efd241002a0 2024-02-18T11:39:27.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:26.869+0000 7efd28c05700 1 -- 172.21.15.12:0/365960980 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7efd241121a0 con 0x7efd241002a0 2024-02-18T11:39:27.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:26.870+0000 7efd1b7fe700 1 -- 172.21.15.12:0/365960980 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 137296+0+0 (secure 0 0 0) 0x7efd14004200 con 0x7efd241002a0 2024-02-18T11:39:27.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:26.871+0000 7efd1b7fe700 1 --2- 172.21.15.12:0/365960980 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7efd100a2870 0x7efd100a4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:27.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:26.871+0000 7efd21d9b700 1 --2- 172.21.15.12:0/365960980 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7efd100a2870 0x7efd100a4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:27.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:26.871+0000 7efd1b7fe700 1 -- 172.21.15.12:0/365960980 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 7979+0+0 (secure 0 0 0) 0x7efd140d0510 con 0x7efd241002a0 2024-02-18T11:39:27.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:26.871+0000 7efd1b7fe700 1 --2- 172.21.15.12:0/365960980 >> [v2:172.21.15.158:6824/3239052167,v1:172.21.15.158:6825/3239052167] conn(0x7efd100ac050 0x7efd100ae470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:27.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:26.871+0000 7efd1b7fe700 1 -- 172.21.15.12:0/365960980 --> [v2:172.21.15.158:6824/3239052167,v1:172.21.15.158:6825/3239052167] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efd100aeb20 con 0x7efd100ac050 2024-02-18T11:39:27.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:26.871+0000 7efd1b7fe700 1 -- 172.21.15.12:0/365960980 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7efd140d09b0 con 0x7efd241002a0 2024-02-18T11:39:27.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:26.871+0000 7efd21d9b700 1 --2- 172.21.15.12:0/365960980 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7efd100a2870 0x7efd100a4d30 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7efd0c005fd0 tx=0x7efd0c005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:39:27.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:26.871+0000 7efd22d9d700 1 --2- 172.21.15.12:0/365960980 >> [v2:172.21.15.158:6824/3239052167,v1:172.21.15.158:6825/3239052167] conn(0x7efd100ac050 0x7efd100ae470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:27.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:26.872+0000 7efd22d9d700 1 --2- 172.21.15.12:0/365960980 >> [v2:172.21.15.158:6824/3239052167,v1:172.21.15.158:6825/3239052167] conn(0x7efd100ac050 0x7efd100ae470 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:39:27.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:26.873+0000 7efd1b7fe700 1 -- 172.21.15.12:0/365960980 <== osd.10 v2:172.21.15.158:6824/3239052167 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7efd100aeb20 con 0x7efd100ac050 2024-02-18T11:39:27.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:26.889+0000 7efd28c05700 1 -- 172.21.15.12:0/365960980 --> [v2:172.21.15.158:6824/3239052167,v1:172.21.15.158:6825/3239052167] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7efd24062340 con 0x7efd100ac050 2024-02-18T11:39:27.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:26.889+0000 7efd1b7fe700 1 -- 172.21.15.12:0/365960980 <== osd.10 v2:172.21.15.158:6824/3239052167 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7efd24062340 con 0x7efd100ac050 2024-02-18T11:39:27.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:26.890+0000 7efd28c05700 1 -- 172.21.15.12:0/365960980 >> [v2:172.21.15.158:6824/3239052167,v1:172.21.15.158:6825/3239052167] conn(0x7efd100ac050 msgr2=0x7efd100ae470 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:27.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:26.890+0000 7efd28c05700 1 --2- 172.21.15.12:0/365960980 >> [v2:172.21.15.158:6824/3239052167,v1:172.21.15.158:6825/3239052167] conn(0x7efd100ac050 0x7efd100ae470 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:27.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:26.890+0000 7efd28c05700 1 -- 172.21.15.12:0/365960980 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7efd100a2870 msgr2=0x7efd100a4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:27.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:26.890+0000 7efd28c05700 1 --2- 172.21.15.12:0/365960980 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7efd100a2870 0x7efd100a4d30 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7efd0c005fd0 tx=0x7efd0c005ee0 comp rx=0 tx=0).stop 2024-02-18T11:39:27.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:26.890+0000 7efd28c05700 1 -- 172.21.15.12:0/365960980 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd241002a0 msgr2=0x7efd24101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:27.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:26.890+0000 7efd28c05700 1 --2- 172.21.15.12:0/365960980 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd241002a0 0x7efd24101480 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7efd14006070 tx=0x7efd1400cf40 comp rx=0 tx=0).stop 2024-02-18T11:39:27.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:26.890+0000 7efd22d9d700 1 -- 172.21.15.12:0/365960980 reap_dead start 2024-02-18T11:39:27.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:26.890+0000 7efd28c05700 1 -- 172.21.15.12:0/365960980 shutdown_connections 2024-02-18T11:39:27.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:26.890+0000 7efd28c05700 1 -- 172.21.15.12:0/365960980 >> 172.21.15.12:0/365960980 conn(0x7efd240771b0 msgr2=0x7efd24104cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:39:27.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:26.890+0000 7efd28c05700 1 -- 172.21.15.12:0/365960980 shutdown_connections 2024-02-18T11:39:27.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:26.890+0000 7efd28c05700 1 -- 172.21.15.12:0/365960980 wait complete. 2024-02-18T11:39:27.592 INFO:teuthology.orchestra.run.smithi012.stdout:77309411393 2024-02-18T11:39:27.592 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-02-18T11:39:27.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:27.775+0000 7fc0cdc39700 1 -- 172.21.15.12:0/1759913723 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc0c8100bb0 msgr2=0x7fc0c8101010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:27.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:27.775+0000 7fc0cdc39700 1 --2- 172.21.15.12:0/1759913723 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc0c8100bb0 0x7fc0c8101010 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc009990 tx=0x7fc0bc009ca0 comp rx=0 tx=0).stop 2024-02-18T11:39:27.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:27.775+0000 7fc0cdc39700 1 -- 172.21.15.12:0/1759913723 shutdown_connections 2024-02-18T11:39:27.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:27.775+0000 7fc0cdc39700 1 --2- 172.21.15.12:0/1759913723 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0c8101550 0x7fc0c810da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:27.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:27.775+0000 7fc0cdc39700 1 --2- 172.21.15.12:0/1759913723 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc0c8106bf0 0x7fc0c8106fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:27.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:27.775+0000 7fc0cdc39700 1 --2- 172.21.15.12:0/1759913723 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc0c8100bb0 0x7fc0c8101010 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:27.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:27.775+0000 7fc0cdc39700 1 -- 172.21.15.12:0/1759913723 >> 172.21.15.12:0/1759913723 conn(0x7fc0c8077ae0 msgr2=0x7fc0c8077ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:39:27.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:27.775+0000 7fc0cdc39700 1 -- 172.21.15.12:0/1759913723 shutdown_connections 2024-02-18T11:39:27.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:27.775+0000 7fc0cdc39700 1 -- 172.21.15.12:0/1759913723 wait complete. 2024-02-18T11:39:27.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:27.775+0000 7fc0cdc39700 1 Processor -- start 2024-02-18T11:39:27.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:27.776+0000 7fc0cdc39700 1 -- start start 2024-02-18T11:39:27.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:27.776+0000 7fc0cdc39700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0c8101550 0x7fc0c8198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:27.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:27.776+0000 7fc0cdc39700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc0c8106bf0 0x7fc0c8198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:27.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:27.776+0000 7fc0cdc39700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc0c819cce0 0x7fc0c819d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:27.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:27.776+0000 7fc0cdc39700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c819d6d0 con 0x7fc0c8106bf0 2024-02-18T11:39:27.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:27.776+0000 7fc0cdc39700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c819d840 con 0x7fc0c819cce0 2024-02-18T11:39:27.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:27.776+0000 7fc0cdc39700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c819da00 con 0x7fc0c8101550 2024-02-18T11:39:27.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:27.776+0000 7fc0c6ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc0c8106bf0 0x7fc0c8198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:27.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:27.776+0000 7fc0c6ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc0c8106bf0 0x7fc0c8198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52828/0 (socket says 172.21.15.12:52828) 2024-02-18T11:39:27.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:27.776+0000 7fc0c6ffd700 1 -- 172.21.15.12:0/3083914278 learned_addr learned my addr 172.21.15.12:0/3083914278 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:39:27.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:27.776+0000 7fc0c7fff700 1 --2- 172.21.15.12:0/3083914278 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc0c819cce0 0x7fc0c819d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:27.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:27.776+0000 7fc0c77fe700 1 --2- 172.21.15.12:0/3083914278 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0c8101550 0x7fc0c8198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:27.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:27.776+0000 7fc0c6ffd700 1 -- 172.21.15.12:0/3083914278 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0c8101550 msgr2=0x7fc0c8198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:27.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:27.776+0000 7fc0c6ffd700 1 --2- 172.21.15.12:0/3083914278 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0c8101550 0x7fc0c8198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:27.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:27.776+0000 7fc0c6ffd700 1 -- 172.21.15.12:0/3083914278 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc0c819cce0 msgr2=0x7fc0c819d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:27.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:27.776+0000 7fc0c6ffd700 1 --2- 172.21.15.12:0/3083914278 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc0c819cce0 0x7fc0c819d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:27.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:27.777+0000 7fc0c6ffd700 1 -- 172.21.15.12:0/3083914278 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0bc0095f0 con 0x7fc0c8106bf0 2024-02-18T11:39:27.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:27.777+0000 7fc0c77fe700 1 --2- 172.21.15.12:0/3083914278 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0c8101550 0x7fc0c8198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:39:27.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:27.777+0000 7fc0c6ffd700 1 --2- 172.21.15.12:0/3083914278 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc0c8106bf0 0x7fc0c8198950 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc006010 tx=0x7fc0bc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:39:27.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:27.777+0000 7fc0c4ff9700 1 -- 172.21.15.12:0/3083914278 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc01d070 con 0x7fc0c8106bf0 2024-02-18T11:39:27.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:27.777+0000 7fc0c4ff9700 1 -- 172.21.15.12:0/3083914278 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc0bc021410 con 0x7fc0c8106bf0 2024-02-18T11:39:27.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:27.777+0000 7fc0cdc39700 1 -- 172.21.15.12:0/3083914278 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0c81a4f90 con 0x7fc0c8106bf0 2024-02-18T11:39:27.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:27.777+0000 7fc0c4ff9700 1 -- 172.21.15.12:0/3083914278 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc017630 con 0x7fc0c8106bf0 2024-02-18T11:39:27.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:27.777+0000 7fc0cdc39700 1 -- 172.21.15.12:0/3083914278 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0c81a5340 con 0x7fc0c8106bf0 2024-02-18T11:39:27.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:27.778+0000 7fc0cdc39700 1 -- 172.21.15.12:0/3083914278 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc0c81022a0 con 0x7fc0c8106bf0 2024-02-18T11:39:27.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:27.779+0000 7fc0c4ff9700 1 -- 172.21.15.12:0/3083914278 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 137296+0+0 (secure 0 0 0) 0x7fc0bc021580 con 0x7fc0c8106bf0 2024-02-18T11:39:27.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:27.780+0000 7fc0c4ff9700 1 --2- 172.21.15.12:0/3083914278 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fc0b00a28f0 0x7fc0b00a4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:27.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:27.780+0000 7fc0c77fe700 1 --2- 172.21.15.12:0/3083914278 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fc0b00a28f0 0x7fc0b00a4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:27.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:27.780+0000 7fc0c4ff9700 1 -- 172.21.15.12:0/3083914278 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc0bc0cfa20 con 0x7fc0c8106bf0 2024-02-18T11:39:27.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:27.780+0000 7fc0c4ff9700 1 --2- 172.21.15.12:0/3083914278 >> [v2:172.21.15.12:6818/1206337767,v1:172.21.15.12:6819/1206337767] conn(0x7fc0b00ac0d0 0x7fc0b00ae4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:27.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:27.780+0000 7fc0c4ff9700 1 -- 172.21.15.12:0/3083914278 --> [v2:172.21.15.12:6818/1206337767,v1:172.21.15.12:6819/1206337767] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc0b00aeba0 con 0x7fc0b00ac0d0 2024-02-18T11:39:27.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:27.781+0000 7fc0c4ff9700 1 -- 172.21.15.12:0/3083914278 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc0bc0cfd90 con 0x7fc0c8106bf0 2024-02-18T11:39:27.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:27.781+0000 7fc0c7fff700 1 --2- 172.21.15.12:0/3083914278 >> [v2:172.21.15.12:6818/1206337767,v1:172.21.15.12:6819/1206337767] conn(0x7fc0b00ac0d0 0x7fc0b00ae4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:27.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:27.781+0000 7fc0c77fe700 1 --2- 172.21.15.12:0/3083914278 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fc0b00a28f0 0x7fc0b00a4db0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc0b8009c90 tx=0x7fc0b8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:39:27.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:27.781+0000 7fc0c7fff700 1 --2- 172.21.15.12:0/3083914278 >> [v2:172.21.15.12:6818/1206337767,v1:172.21.15.12:6819/1206337767] conn(0x7fc0b00ac0d0 0x7fc0b00ae4f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:39:27.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:27.782+0000 7fc0c4ff9700 1 -- 172.21.15.12:0/3083914278 <== osd.6 v2:172.21.15.12:6818/1206337767 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fc0b00aeba0 con 0x7fc0b00ac0d0 2024-02-18T11:39:27.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:27.806+0000 7fc0cdc39700 1 -- 172.21.15.12:0/3083914278 --> [v2:172.21.15.12:6818/1206337767,v1:172.21.15.12:6819/1206337767] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc0c8110ee0 con 0x7fc0b00ac0d0 2024-02-18T11:39:27.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:27.807+0000 7fc0c4ff9700 1 -- 172.21.15.12:0/3083914278 <== osd.6 v2:172.21.15.12:6818/1206337767 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fc0c8110ee0 con 0x7fc0b00ac0d0 2024-02-18T11:39:27.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:27.807+0000 7fc0cdc39700 1 -- 172.21.15.12:0/3083914278 >> [v2:172.21.15.12:6818/1206337767,v1:172.21.15.12:6819/1206337767] conn(0x7fc0b00ac0d0 msgr2=0x7fc0b00ae4f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:27.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:27.807+0000 7fc0cdc39700 1 --2- 172.21.15.12:0/3083914278 >> [v2:172.21.15.12:6818/1206337767,v1:172.21.15.12:6819/1206337767] conn(0x7fc0b00ac0d0 0x7fc0b00ae4f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:27.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:27.807+0000 7fc0cdc39700 1 -- 172.21.15.12:0/3083914278 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fc0b00a28f0 msgr2=0x7fc0b00a4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:27.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:27.807+0000 7fc0cdc39700 1 --2- 172.21.15.12:0/3083914278 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fc0b00a28f0 0x7fc0b00a4db0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc0b8009c90 tx=0x7fc0b8009260 comp rx=0 tx=0).stop 2024-02-18T11:39:27.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:27.807+0000 7fc0cdc39700 1 -- 172.21.15.12:0/3083914278 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc0c8106bf0 msgr2=0x7fc0c8198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:27.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:27.807+0000 7fc0cdc39700 1 --2- 172.21.15.12:0/3083914278 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc0c8106bf0 0x7fc0c8198950 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc006010 tx=0x7fc0bc004730 comp rx=0 tx=0).stop 2024-02-18T11:39:27.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:27.807+0000 7fc0c7fff700 1 -- 172.21.15.12:0/3083914278 reap_dead start 2024-02-18T11:39:27.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:27.807+0000 7fc0cdc39700 1 -- 172.21.15.12:0/3083914278 shutdown_connections 2024-02-18T11:39:27.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:27.807+0000 7fc0cdc39700 1 -- 172.21.15.12:0/3083914278 >> 172.21.15.12:0/3083914278 conn(0x7fc0c8077ae0 msgr2=0x7fc0c80fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:39:27.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:27.807+0000 7fc0cdc39700 1 -- 172.21.15.12:0/3083914278 shutdown_connections 2024-02-18T11:39:27.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:27.807+0000 7fc0cdc39700 1 -- 172.21.15.12:0/3083914278 wait complete. 2024-02-18T11:39:28.172 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:39:28 smithi012 ceph-mon[32460]: pgmap v305: 97 pgs: 97 active+clean; 639 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 170 B/s wr, 7 op/s 2024-02-18T11:39:28.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:39:28 smithi188 ceph-mon[36925]: pgmap v305: 97 pgs: 97 active+clean; 639 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 170 B/s wr, 7 op/s 2024-02-18T11:39:28.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:39:28 smithi158 ceph-mon[36626]: pgmap v305: 97 pgs: 97 active+clean; 639 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 170 B/s wr, 7 op/s 2024-02-18T11:39:28.723 INFO:teuthology.orchestra.run.smithi012.stdout:227633266728 2024-02-18T11:39:28.724 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-02-18T11:39:29.172 INFO:teuthology.orchestra.run.smithi012.stdout:296352743454 2024-02-18T11:39:29.172 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.9 2024-02-18T11:39:29.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.360+0000 7fa08bc7c700 1 -- 172.21.15.12:0/1621574582 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa084100be0 msgr2=0x7fa084101060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:29.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.360+0000 7fa08bc7c700 1 --2- 172.21.15.12:0/1621574582 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa084100be0 0x7fa084101060 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa0740099c0 tx=0x7fa074009cd0 comp rx=0 tx=0).stop 2024-02-18T11:39:29.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.360+0000 7fa08bc7c700 1 -- 172.21.15.12:0/1621574582 shutdown_connections 2024-02-18T11:39:29.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.360+0000 7fa08bc7c700 1 --2- 172.21.15.12:0/1621574582 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0841015a0 0x7fa08410d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:29.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.360+0000 7fa08bc7c700 1 --2- 172.21.15.12:0/1621574582 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa084106c20 0x7fa084107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:29.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.360+0000 7fa08bc7c700 1 --2- 172.21.15.12:0/1621574582 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa084100be0 0x7fa084101060 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:29.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.360+0000 7fa08bc7c700 1 -- 172.21.15.12:0/1621574582 >> 172.21.15.12:0/1621574582 conn(0x7fa084076c00 msgr2=0x7fa084077010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:39:29.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.360+0000 7fa08bc7c700 1 -- 172.21.15.12:0/1621574582 shutdown_connections 2024-02-18T11:39:29.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.360+0000 7fa08bc7c700 1 -- 172.21.15.12:0/1621574582 wait complete. 2024-02-18T11:39:29.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.361+0000 7fa08bc7c700 1 Processor -- start 2024-02-18T11:39:29.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.361+0000 7fa08bc7c700 1 -- start start 2024-02-18T11:39:29.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.361+0000 7fa08bc7c700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa084100be0 0x7fa0841980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:29.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.361+0000 7fa08bc7c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0841015a0 0x7fa084198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:29.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.361+0000 7fa08bc7c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa084106c20 0x7fa08419c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:29.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.361+0000 7fa08bc7c700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa08419cfb0 con 0x7fa084106c20 2024-02-18T11:39:29.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.361+0000 7fa08bc7c700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa08419d120 con 0x7fa084100be0 2024-02-18T11:39:29.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.361+0000 7fa08bc7c700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa08419d2e0 con 0x7fa0841015a0 2024-02-18T11:39:29.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.361+0000 7fa089a18700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa084100be0 0x7fa0841980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:29.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.361+0000 7fa08a219700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa084106c20 0x7fa08419c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:29.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.361+0000 7fa089a18700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa084100be0 0x7fa0841980d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.158:3300/0 says I am v2:172.21.15.12:59764/0 (socket says 172.21.15.12:59764) 2024-02-18T11:39:29.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.361+0000 7fa089a18700 1 -- 172.21.15.12:0/2542036068 learned_addr learned my addr 172.21.15.12:0/2542036068 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:39:29.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.361+0000 7fa089217700 1 --2- 172.21.15.12:0/2542036068 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0841015a0 0x7fa084198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:29.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.362+0000 7fa08a219700 1 -- 172.21.15.12:0/2542036068 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0841015a0 msgr2=0x7fa084198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:29.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.362+0000 7fa08a219700 1 --2- 172.21.15.12:0/2542036068 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0841015a0 0x7fa084198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:29.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.362+0000 7fa08a219700 1 -- 172.21.15.12:0/2542036068 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa084100be0 msgr2=0x7fa0841980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:29.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.362+0000 7fa08a219700 1 --2- 172.21.15.12:0/2542036068 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa084100be0 0x7fa0841980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:29.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.362+0000 7fa08a219700 1 -- 172.21.15.12:0/2542036068 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0740095f0 con 0x7fa084106c20 2024-02-18T11:39:29.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.362+0000 7fa089a18700 1 --2- 172.21.15.12:0/2542036068 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa084100be0 0x7fa0841980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-18T11:39:29.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.362+0000 7fa08a219700 1 --2- 172.21.15.12:0/2542036068 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa084106c20 0x7fa08419c9a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa07c007f10 tx=0x7fa07c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:39:29.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.362+0000 7fa07affd700 1 -- 172.21.15.12:0/2542036068 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa07c00ddd0 con 0x7fa084106c20 2024-02-18T11:39:29.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.362+0000 7fa08bc7c700 1 -- 172.21.15.12:0/2542036068 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa08419d5c0 con 0x7fa084106c20 2024-02-18T11:39:29.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.362+0000 7fa08bc7c700 1 -- 172.21.15.12:0/2542036068 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0841a4e20 con 0x7fa084106c20 2024-02-18T11:39:29.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.362+0000 7fa07affd700 1 -- 172.21.15.12:0/2542036068 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa07c00f040 con 0x7fa084106c20 2024-02-18T11:39:29.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.362+0000 7fa07affd700 1 -- 172.21.15.12:0/2542036068 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa07c015400 con 0x7fa084106c20 2024-02-18T11:39:29.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.363+0000 7fa08bc7c700 1 -- 172.21.15.12:0/2542036068 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa068001030 con 0x7fa084106c20 2024-02-18T11:39:29.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.365+0000 7fa07affd700 1 -- 172.21.15.12:0/2542036068 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 137296+0+0 (secure 0 0 0) 0x7fa07c004ad0 con 0x7fa084106c20 2024-02-18T11:39:29.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.366+0000 7fa07affd700 1 --2- 172.21.15.12:0/2542036068 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fa0700a2950 0x7fa0700a4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:29.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.366+0000 7fa089a18700 1 --2- 172.21.15.12:0/2542036068 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fa0700a2950 0x7fa0700a4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:29.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.367+0000 7fa07affd700 1 -- 172.21.15.12:0/2542036068 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa07c0ce8a0 con 0x7fa084106c20 2024-02-18T11:39:29.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.367+0000 7fa07affd700 1 --2- 172.21.15.12:0/2542036068 >> [v2:172.21.15.12:6802/536522426,v1:172.21.15.12:6803/536522426] conn(0x7fa0700ac130 0x7fa0700ae550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:29.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.367+0000 7fa07affd700 1 -- 172.21.15.12:0/2542036068 --> [v2:172.21.15.12:6802/536522426,v1:172.21.15.12:6803/536522426] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa0700aec00 con 0x7fa0700ac130 2024-02-18T11:39:29.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.367+0000 7fa07affd700 1 -- 172.21.15.12:0/2542036068 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa07c015400 con 0x7fa084106c20 2024-02-18T11:39:29.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.367+0000 7fa089a18700 1 --2- 172.21.15.12:0/2542036068 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fa0700a2950 0x7fa0700a4e10 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa080005950 tx=0x7fa08000b500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:39:29.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.367+0000 7fa089217700 1 --2- 172.21.15.12:0/2542036068 >> [v2:172.21.15.12:6802/536522426,v1:172.21.15.12:6803/536522426] conn(0x7fa0700ac130 0x7fa0700ae550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:29.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.368+0000 7fa089217700 1 --2- 172.21.15.12:0/2542036068 >> [v2:172.21.15.12:6802/536522426,v1:172.21.15.12:6803/536522426] conn(0x7fa0700ac130 0x7fa0700ae550 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:39:29.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.368+0000 7fa07affd700 1 -- 172.21.15.12:0/2542036068 <== osd.0 v2:172.21.15.12:6802/536522426 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fa0700aec00 con 0x7fa0700ac130 2024-02-18T11:39:29.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.385+0000 7fa08bc7c700 1 -- 172.21.15.12:0/2542036068 --> [v2:172.21.15.12:6802/536522426,v1:172.21.15.12:6803/536522426] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa068002dc0 con 0x7fa0700ac130 2024-02-18T11:39:29.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.386+0000 7fa07affd700 1 -- 172.21.15.12:0/2542036068 <== osd.0 v2:172.21.15.12:6802/536522426 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa068002dc0 con 0x7fa0700ac130 2024-02-18T11:39:29.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.386+0000 7fa08bc7c700 1 -- 172.21.15.12:0/2542036068 >> [v2:172.21.15.12:6802/536522426,v1:172.21.15.12:6803/536522426] conn(0x7fa0700ac130 msgr2=0x7fa0700ae550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:29.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.386+0000 7fa08bc7c700 1 --2- 172.21.15.12:0/2542036068 >> [v2:172.21.15.12:6802/536522426,v1:172.21.15.12:6803/536522426] conn(0x7fa0700ac130 0x7fa0700ae550 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:29.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.386+0000 7fa08bc7c700 1 -- 172.21.15.12:0/2542036068 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fa0700a2950 msgr2=0x7fa0700a4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:29.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.386+0000 7fa08bc7c700 1 --2- 172.21.15.12:0/2542036068 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fa0700a2950 0x7fa0700a4e10 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa080005950 tx=0x7fa08000b500 comp rx=0 tx=0).stop 2024-02-18T11:39:29.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.387+0000 7fa08bc7c700 1 -- 172.21.15.12:0/2542036068 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa084106c20 msgr2=0x7fa08419c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:29.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.387+0000 7fa08bc7c700 1 --2- 172.21.15.12:0/2542036068 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa084106c20 0x7fa08419c9a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa07c007f10 tx=0x7fa07c00d3b0 comp rx=0 tx=0).stop 2024-02-18T11:39:29.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.387+0000 7fa08a219700 1 -- 172.21.15.12:0/2542036068 reap_dead start 2024-02-18T11:39:29.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.387+0000 7fa08bc7c700 1 -- 172.21.15.12:0/2542036068 shutdown_connections 2024-02-18T11:39:29.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.387+0000 7fa08bc7c700 1 -- 172.21.15.12:0/2542036068 >> 172.21.15.12:0/2542036068 conn(0x7fa084076c00 msgr2=0x7fa0840ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:39:29.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.387+0000 7fa08bc7c700 1 -- 172.21.15.12:0/2542036068 shutdown_connections 2024-02-18T11:39:29.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.387+0000 7fa08bc7c700 1 -- 172.21.15.12:0/2542036068 wait complete. 2024-02-18T11:39:29.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.870+0000 7f92ca10c700 1 -- 172.21.15.12:0/1889131160 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92c4100850 msgr2=0x7f92c4100c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:29.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.870+0000 7f92ca10c700 1 --2- 172.21.15.12:0/1889131160 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92c4100850 0x7f92c4100c30 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f92b4009990 tx=0x7f92b4009ca0 comp rx=0 tx=0).stop 2024-02-18T11:39:29.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.871+0000 7f92ca10c700 1 -- 172.21.15.12:0/1889131160 shutdown_connections 2024-02-18T11:39:29.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.871+0000 7f92ca10c700 1 --2- 172.21.15.12:0/1889131160 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92c4107070 0x7f92c410ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:29.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.871+0000 7f92ca10c700 1 --2- 172.21.15.12:0/1889131160 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92c4100850 0x7f92c4100c30 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:29.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.871+0000 7f92ca10c700 1 --2- 172.21.15.12:0/1889131160 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f92c4101200 0x7f92c4106940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:29.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.871+0000 7f92ca10c700 1 -- 172.21.15.12:0/1889131160 >> 172.21.15.12:0/1889131160 conn(0x7f92c4075f10 msgr2=0x7f92c4076320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:39:29.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.871+0000 7f92ca10c700 1 -- 172.21.15.12:0/1889131160 shutdown_connections 2024-02-18T11:39:29.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.871+0000 7f92ca10c700 1 -- 172.21.15.12:0/1889131160 wait complete. 2024-02-18T11:39:29.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.871+0000 7f92ca10c700 1 Processor -- start 2024-02-18T11:39:29.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.871+0000 7f92ca10c700 1 -- start start 2024-02-18T11:39:29.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.872+0000 7f92ca10c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92c4100850 0x7f92c41982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:29.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.872+0000 7f92ca10c700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f92c4101200 0x7f92c41987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:29.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.872+0000 7f92ca10c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92c4107070 0x7f92c419cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:29.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.872+0000 7f92ca10c700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c419d190 con 0x7f92c4100850 2024-02-18T11:39:29.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.872+0000 7f92ca10c700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c419d300 con 0x7f92c4101200 2024-02-18T11:39:29.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.872+0000 7f92ca10c700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c419d4c0 con 0x7f92c4107070 2024-02-18T11:39:29.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.872+0000 7f92c37fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92c4100850 0x7f92c41982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:29.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.872+0000 7f92c37fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92c4100850 0x7f92c41982b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52864/0 (socket says 172.21.15.12:52864) 2024-02-18T11:39:29.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.872+0000 7f92c37fe700 1 -- 172.21.15.12:0/1092679033 learned_addr learned my addr 172.21.15.12:0/1092679033 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:39:29.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.872+0000 7f92c3fff700 1 --2- 172.21.15.12:0/1092679033 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92c4107070 0x7f92c419cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:29.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.872+0000 7f92c2ffd700 1 --2- 172.21.15.12:0/1092679033 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f92c4101200 0x7f92c41987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:29.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.872+0000 7f92c37fe700 1 -- 172.21.15.12:0/1092679033 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92c4107070 msgr2=0x7f92c419cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:29.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.872+0000 7f92c37fe700 1 --2- 172.21.15.12:0/1092679033 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92c4107070 0x7f92c419cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:29.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.872+0000 7f92c37fe700 1 -- 172.21.15.12:0/1092679033 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f92c4101200 msgr2=0x7f92c41987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:29.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.872+0000 7f92c37fe700 1 --2- 172.21.15.12:0/1092679033 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f92c4101200 0x7f92c41987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:29.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.872+0000 7f92c37fe700 1 -- 172.21.15.12:0/1092679033 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92b40095f0 con 0x7f92c4100850 2024-02-18T11:39:29.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.872+0000 7f92c2ffd700 1 --2- 172.21.15.12:0/1092679033 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f92c4101200 0x7f92c41987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:39:29.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.872+0000 7f92c3fff700 1 --2- 172.21.15.12:0/1092679033 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92c4107070 0x7f92c419cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:39:29.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.872+0000 7f92c37fe700 1 --2- 172.21.15.12:0/1092679033 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92c4100850 0x7f92c41982b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f92b40098e0 tx=0x7f92b40037c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:39:29.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.872+0000 7f92c0ff9700 1 -- 172.21.15.12:0/1092679033 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b401d070 con 0x7f92c4100850 2024-02-18T11:39:29.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.872+0000 7f92ca10c700 1 -- 172.21.15.12:0/1092679033 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92c419d740 con 0x7f92c4100850 2024-02-18T11:39:29.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.873+0000 7f92c0ff9700 1 -- 172.21.15.12:0/1092679033 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f92b400fbf0 con 0x7f92c4100850 2024-02-18T11:39:29.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.873+0000 7f92c0ff9700 1 -- 172.21.15.12:0/1092679033 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b4017650 con 0x7f92c4100850 2024-02-18T11:39:29.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.873+0000 7f92ca10c700 1 -- 172.21.15.12:0/1092679033 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92c41a5000 con 0x7f92c4100850 2024-02-18T11:39:29.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.873+0000 7f92ca10c700 1 -- 172.21.15.12:0/1092679033 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f92c41120f0 con 0x7f92c4100850 2024-02-18T11:39:29.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.875+0000 7f92c0ff9700 1 -- 172.21.15.12:0/1092679033 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 137296+0+0 (secure 0 0 0) 0x7f92b4022470 con 0x7f92c4100850 2024-02-18T11:39:29.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.875+0000 7f92c0ff9700 1 --2- 172.21.15.12:0/1092679033 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f92ac0a2870 0x7f92ac0a4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:29.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.875+0000 7f92c2ffd700 1 --2- 172.21.15.12:0/1092679033 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f92ac0a2870 0x7f92ac0a4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:29.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.875+0000 7f92c0ff9700 1 -- 172.21.15.12:0/1092679033 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f92b40d00a0 con 0x7f92c4100850 2024-02-18T11:39:29.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.875+0000 7f92c0ff9700 1 --2- 172.21.15.12:0/1092679033 >> [v2:172.21.15.188:6816/2110170682,v1:172.21.15.188:6817/2110170682] conn(0x7f92ac0ac050 0x7f92ac0ae470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:29.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.875+0000 7f92c0ff9700 1 -- 172.21.15.12:0/1092679033 --> [v2:172.21.15.188:6816/2110170682,v1:172.21.15.188:6817/2110170682] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f92ac0aeb20 con 0x7f92ac0ac050 2024-02-18T11:39:29.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.875+0000 7f92c0ff9700 1 -- 172.21.15.12:0/1092679033 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f92b40d0410 con 0x7f92c4100850 2024-02-18T11:39:29.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.876+0000 7f92c2ffd700 1 --2- 172.21.15.12:0/1092679033 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f92ac0a2870 0x7f92ac0a4d30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f92c4106b30 tx=0x7f92b8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:39:29.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.876+0000 7f92c3fff700 1 --2- 172.21.15.12:0/1092679033 >> [v2:172.21.15.188:6816/2110170682,v1:172.21.15.188:6817/2110170682] conn(0x7f92ac0ac050 0x7f92ac0ae470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:29.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.876+0000 7f92c3fff700 1 --2- 172.21.15.12:0/1092679033 >> [v2:172.21.15.188:6816/2110170682,v1:172.21.15.188:6817/2110170682] conn(0x7f92ac0ac050 0x7f92ac0ae470 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:39:29.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.877+0000 7f92c0ff9700 1 -- 172.21.15.12:0/1092679033 <== osd.8 v2:172.21.15.188:6816/2110170682 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f92ac0aeb20 con 0x7f92ac0ac050 2024-02-18T11:39:29.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.893+0000 7f92ca10c700 1 -- 172.21.15.12:0/1092679033 --> [v2:172.21.15.188:6816/2110170682,v1:172.21.15.188:6817/2110170682] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f92c4062340 con 0x7f92ac0ac050 2024-02-18T11:39:29.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.894+0000 7f92c0ff9700 1 -- 172.21.15.12:0/1092679033 <== osd.8 v2:172.21.15.188:6816/2110170682 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f92c4062340 con 0x7f92ac0ac050 2024-02-18T11:39:29.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.894+0000 7f92ca10c700 1 -- 172.21.15.12:0/1092679033 >> [v2:172.21.15.188:6816/2110170682,v1:172.21.15.188:6817/2110170682] conn(0x7f92ac0ac050 msgr2=0x7f92ac0ae470 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:29.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.894+0000 7f92ca10c700 1 --2- 172.21.15.12:0/1092679033 >> [v2:172.21.15.188:6816/2110170682,v1:172.21.15.188:6817/2110170682] conn(0x7f92ac0ac050 0x7f92ac0ae470 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:29.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.894+0000 7f92ca10c700 1 -- 172.21.15.12:0/1092679033 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f92ac0a2870 msgr2=0x7f92ac0a4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:29.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.894+0000 7f92ca10c700 1 --2- 172.21.15.12:0/1092679033 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f92ac0a2870 0x7f92ac0a4d30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f92c4106b30 tx=0x7f92b8008040 comp rx=0 tx=0).stop 2024-02-18T11:39:29.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.894+0000 7f92ca10c700 1 -- 172.21.15.12:0/1092679033 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92c4100850 msgr2=0x7f92c41982b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:29.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.894+0000 7f92ca10c700 1 --2- 172.21.15.12:0/1092679033 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92c4100850 0x7f92c41982b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f92b40098e0 tx=0x7f92b40037c0 comp rx=0 tx=0).stop 2024-02-18T11:39:29.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.894+0000 7f92c3fff700 1 -- 172.21.15.12:0/1092679033 reap_dead start 2024-02-18T11:39:29.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.895+0000 7f92ca10c700 1 -- 172.21.15.12:0/1092679033 shutdown_connections 2024-02-18T11:39:29.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.895+0000 7f92ca10c700 1 -- 172.21.15.12:0/1092679033 >> 172.21.15.12:0/1092679033 conn(0x7f92c4075f10 msgr2=0x7f92c40ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:39:29.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.895+0000 7f92ca10c700 1 -- 172.21.15.12:0/1092679033 shutdown_connections 2024-02-18T11:39:29.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:29.895+0000 7f92ca10c700 1 -- 172.21.15.12:0/1092679033 wait complete. 2024-02-18T11:39:30.372 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:39:30 smithi012 ceph-mon[32460]: pgmap v306: 97 pgs: 97 active+clean; 639 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 170 B/s wr, 6 op/s 2024-02-18T11:39:30.398 INFO:teuthology.orchestra.run.smithi012.stdout:115964117053 2024-02-18T11:39:30.399 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-02-18T11:39:30.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:39:30 smithi158 ceph-mon[36626]: pgmap v306: 97 pgs: 97 active+clean; 639 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 170 B/s wr, 6 op/s 2024-02-18T11:39:30.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:39:30 smithi188 ceph-mon[36925]: pgmap v306: 97 pgs: 97 active+clean; 639 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 170 B/s wr, 6 op/s 2024-02-18T11:39:30.829 INFO:teuthology.orchestra.run.smithi012.stdout:206158430255 2024-02-18T11:39:30.830 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-02-18T11:39:31.740 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T11:39:31.741 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T11:39:31.785 INFO:teuthology.orchestra.run.smithi012.stdout:261993005092 2024-02-18T11:39:31.786 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.8 2024-02-18T11:39:32.208 INFO:teuthology.orchestra.run.smithi012.stdout:317827579930 2024-02-18T11:39:32.209 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.10 2024-02-18T11:39:32.324 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T11:39:32.324 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T11:39:32.325 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T11:39:32.325 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T11:39:32.326 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T11:39:32.411 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:39:32 smithi012 ceph-mon[32460]: pgmap v307: 97 pgs: 97 active+clean; 639 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 170 B/s wr, 6 op/s 2024-02-18T11:39:32.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:39:32 smithi158 ceph-mon[36626]: pgmap v307: 97 pgs: 97 active+clean; 639 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 170 B/s wr, 6 op/s 2024-02-18T11:39:32.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:39:32 smithi188 ceph-mon[36925]: pgmap v307: 97 pgs: 97 active+clean; 639 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 170 B/s wr, 6 op/s 2024-02-18T11:39:33.010 INFO:teuthology.orchestra.run.smithi012.stdout:34359738444 2024-02-18T11:39:33.010 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-02-18T11:39:34.259 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:39:34 smithi012 ceph-mon[32460]: pgmap v308: 97 pgs: 97 active+clean; 639 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-18T11:39:34.406 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:39:34 smithi188 ceph-mon[36925]: pgmap v308: 97 pgs: 97 active+clean; 639 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-18T11:39:34.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:39:34 smithi158 ceph-mon[36626]: pgmap v308: 97 pgs: 97 active+clean; 639 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-18T11:39:36.149 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T11:39:36.172 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:39:36 smithi012 ceph-mon[32460]: pgmap v309: 97 pgs: 97 active+clean; 639 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-18T11:39:36.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:39:36 smithi158 ceph-mon[36626]: pgmap v309: 97 pgs: 97 active+clean; 639 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-18T11:39:36.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:39:36 smithi188 ceph-mon[36925]: pgmap v309: 97 pgs: 97 active+clean; 639 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-18T11:39:37.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:37.404+0000 7fe2ebab0700 1 -- 172.21.15.12:0/4201825822 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2e406a5b0 msgr2=0x7fe2e406a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:37.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:37.404+0000 7fe2ebab0700 1 --2- 172.21.15.12:0/4201825822 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2e406a5b0 0x7fe2e406a990 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe2e0009990 tx=0x7fe2e0009ca0 comp rx=0 tx=0).stop 2024-02-18T11:39:37.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:37.404+0000 7fe2ebab0700 1 -- 172.21.15.12:0/4201825822 shutdown_connections 2024-02-18T11:39:37.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:37.404+0000 7fe2ebab0700 1 --2- 172.21.15.12:0/4201825822 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe2e4107010 0x7fe2e410ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:37.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:37.404+0000 7fe2ebab0700 1 --2- 172.21.15.12:0/4201825822 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2e406a5b0 0x7fe2e406a990 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:37.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:37.404+0000 7fe2ebab0700 1 --2- 172.21.15.12:0/4201825822 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2e406af60 0x7fe2e4106930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:37.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:37.404+0000 7fe2ebab0700 1 -- 172.21.15.12:0/4201825822 >> 172.21.15.12:0/4201825822 conn(0x7fe2e4077a70 msgr2=0x7fe2e4077e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:39:37.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:37.404+0000 7fe2ebab0700 1 -- 172.21.15.12:0/4201825822 shutdown_connections 2024-02-18T11:39:37.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:37.404+0000 7fe2ebab0700 1 -- 172.21.15.12:0/4201825822 wait complete. 2024-02-18T11:39:37.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:37.405+0000 7fe2ebab0700 1 Processor -- start 2024-02-18T11:39:37.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:37.405+0000 7fe2ebab0700 1 -- start start 2024-02-18T11:39:37.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:37.405+0000 7fe2ebab0700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2e406a5b0 0x7fe2e4100f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:37.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:37.405+0000 7fe2ebab0700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe2e406af60 0x7fe2e41014a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:37.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:37.405+0000 7fe2ebab0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2e4107010 0x7fe2e41019e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:37.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:37.405+0000 7fe2ebab0700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e41a65e0 con 0x7fe2e4107010 2024-02-18T11:39:37.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:37.405+0000 7fe2ebab0700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e41a6750 con 0x7fe2e406af60 2024-02-18T11:39:37.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:37.405+0000 7fe2ebab0700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e41a6910 con 0x7fe2e406a5b0 2024-02-18T11:39:37.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:37.405+0000 7fe2ea04d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2e4107010 0x7fe2e41019e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:37.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:37.405+0000 7fe2ea04d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2e4107010 0x7fe2e41019e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40944/0 (socket says 172.21.15.12:40944) 2024-02-18T11:39:37.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:37.405+0000 7fe2ea04d700 1 -- 172.21.15.12:0/888236769 learned_addr learned my addr 172.21.15.12:0/888236769 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:39:37.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:37.406+0000 7fe2e984c700 1 --2- 172.21.15.12:0/888236769 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2e406a5b0 0x7fe2e4100f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:37.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:37.406+0000 7fe2ea04d700 1 -- 172.21.15.12:0/888236769 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2e406a5b0 msgr2=0x7fe2e4100f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:37.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:37.406+0000 7fe2ea04d700 1 --2- 172.21.15.12:0/888236769 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2e406a5b0 0x7fe2e4100f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:37.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:37.406+0000 7fe2ea04d700 1 -- 172.21.15.12:0/888236769 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe2e406af60 msgr2=0x7fe2e41014a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:37.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:37.406+0000 7fe2e904b700 1 --2- 172.21.15.12:0/888236769 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe2e406af60 0x7fe2e41014a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:37.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:37.406+0000 7fe2ea04d700 1 --2- 172.21.15.12:0/888236769 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe2e406af60 0x7fe2e41014a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:37.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:37.406+0000 7fe2ea04d700 1 -- 172.21.15.12:0/888236769 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e00095f0 con 0x7fe2e4107010 2024-02-18T11:39:37.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:37.406+0000 7fe2e904b700 1 --2- 172.21.15.12:0/888236769 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe2e406af60 0x7fe2e41014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:39:37.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:37.406+0000 7fe2ea04d700 1 --2- 172.21.15.12:0/888236769 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2e4107010 0x7fe2e41019e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc00cca0 tx=0x7fe2dc00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:39:37.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:37.406+0000 7fe2d6ffd700 1 -- 172.21.15.12:0/888236769 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc00cea0 con 0x7fe2e4107010 2024-02-18T11:39:37.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:37.406+0000 7fe2d6ffd700 1 -- 172.21.15.12:0/888236769 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe2dc00ad60 con 0x7fe2e4107010 2024-02-18T11:39:37.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:37.406+0000 7fe2ebab0700 1 -- 172.21.15.12:0/888236769 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2e41a6bf0 con 0x7fe2e4107010 2024-02-18T11:39:37.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:37.406+0000 7fe2d6ffd700 1 -- 172.21.15.12:0/888236769 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc011780 con 0x7fe2e4107010 2024-02-18T11:39:37.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:37.406+0000 7fe2ebab0700 1 -- 172.21.15.12:0/888236769 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2e41a7140 con 0x7fe2e4107010 2024-02-18T11:39:37.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:37.407+0000 7fe2ebab0700 1 -- 172.21.15.12:0/888236769 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2e410a240 con 0x7fe2e4107010 2024-02-18T11:39:37.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:37.408+0000 7fe2d6ffd700 1 -- 172.21.15.12:0/888236769 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 137296+0+0 (secure 0 0 0) 0x7fe2dc012420 con 0x7fe2e4107010 2024-02-18T11:39:37.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:37.409+0000 7fe2d6ffd700 1 --2- 172.21.15.12:0/888236769 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fe2d00a28f0 0x7fe2d00a4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:37.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:37.409+0000 7fe2e984c700 1 --2- 172.21.15.12:0/888236769 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fe2d00a28f0 0x7fe2d00a4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:37.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:37.409+0000 7fe2d6ffd700 1 -- 172.21.15.12:0/888236769 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fe2dc0ce9a0 con 0x7fe2e4107010 2024-02-18T11:39:37.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:37.409+0000 7fe2e984c700 1 --2- 172.21.15.12:0/888236769 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fe2d00a28f0 0x7fe2d00a4db0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe2e0006010 tx=0x7fe2e001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:39:37.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:37.412+0000 7fe2d6ffd700 1 -- 172.21.15.12:0/888236769 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe2dc097890 con 0x7fe2e4107010 2024-02-18T11:39:37.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:37.617+0000 7fe2ebab0700 1 -- 172.21.15.12:0/888236769 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fe2e4068970 con 0x7fe2e4107010 2024-02-18T11:39:37.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:37.617+0000 7fe2d6ffd700 1 -- 172.21.15.12:0/888236769 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe2dc09af50 con 0x7fe2e4107010 2024-02-18T11:39:37.612 INFO:teuthology.orchestra.run.smithi012.stdout:171798691893 2024-02-18T11:39:37.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:37.619+0000 7fe2ebab0700 1 -- 172.21.15.12:0/888236769 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fe2d00a28f0 msgr2=0x7fe2d00a4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:37.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:37.619+0000 7fe2ebab0700 1 --2- 172.21.15.12:0/888236769 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fe2d00a28f0 0x7fe2d00a4db0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe2e0006010 tx=0x7fe2e001a040 comp rx=0 tx=0).stop 2024-02-18T11:39:37.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:37.619+0000 7fe2ebab0700 1 -- 172.21.15.12:0/888236769 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2e4107010 msgr2=0x7fe2e41019e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:37.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:37.619+0000 7fe2ebab0700 1 --2- 172.21.15.12:0/888236769 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2e4107010 0x7fe2e41019e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc00cca0 tx=0x7fe2dc00a340 comp rx=0 tx=0).stop 2024-02-18T11:39:37.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:37.620+0000 7fe2ebab0700 1 -- 172.21.15.12:0/888236769 shutdown_connections 2024-02-18T11:39:37.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:37.620+0000 7fe2ebab0700 1 --2- 172.21.15.12:0/888236769 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fe2d00a28f0 0x7fe2d00a4db0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:37.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:37.620+0000 7fe2ebab0700 1 --2- 172.21.15.12:0/888236769 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2e4107010 0x7fe2e41019e0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:37.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:37.620+0000 7fe2ebab0700 1 --2- 172.21.15.12:0/888236769 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2e406a5b0 0x7fe2e4100f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:37.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:37.620+0000 7fe2ebab0700 1 --2- 172.21.15.12:0/888236769 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe2e406af60 0x7fe2e41014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:37.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:37.620+0000 7fe2ebab0700 1 -- 172.21.15.12:0/888236769 >> 172.21.15.12:0/888236769 conn(0x7fe2e4077a70 msgr2=0x7fe2e40fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:39:37.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:37.620+0000 7fe2ebab0700 1 -- 172.21.15.12:0/888236769 shutdown_connections 2024-02-18T11:39:37.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:37.620+0000 7fe2ebab0700 1 -- 172.21.15.12:0/888236769 wait complete. 2024-02-18T11:39:38.423 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:39:38 smithi012 ceph-mon[32460]: pgmap v310: 97 pgs: 97 active+clean; 639 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-02-18T11:39:38.423 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:39:38 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/888236769' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-18T11:39:38.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:39:38 smithi158 ceph-mon[36626]: pgmap v310: 97 pgs: 97 active+clean; 639 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-02-18T11:39:38.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:39:38 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/888236769' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-18T11:39:38.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:39:38 smithi188 ceph-mon[36925]: pgmap v310: 97 pgs: 97 active+clean; 639 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-02-18T11:39:38.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:39:38 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/888236769' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-18T11:39:38.653 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T11:39:40.170 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:39:40 smithi012 ceph-mon[32460]: pgmap v311: 97 pgs: 97 active+clean; 639 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s 2024-02-18T11:39:40.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:39:40 smithi158 ceph-mon[36626]: pgmap v311: 97 pgs: 97 active+clean; 639 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s 2024-02-18T11:39:40.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:39:40 smithi188 ceph-mon[36925]: pgmap v311: 97 pgs: 97 active+clean; 639 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s 2024-02-18T11:39:40.694 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T11:39:40.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:40.704+0000 7f527a6e0700 1 -- 172.21.15.12:0/3793883724 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5274101550 msgr2=0x7f527410da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:40.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:40.704+0000 7f527a6e0700 1 --2- 172.21.15.12:0/3793883724 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5274101550 0x7f527410da30 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f526c00b0a0 tx=0x7f526c00b3b0 comp rx=0 tx=0).stop 2024-02-18T11:39:40.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:40.704+0000 7f527a6e0700 1 -- 172.21.15.12:0/3793883724 shutdown_connections 2024-02-18T11:39:40.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:40.704+0000 7f527a6e0700 1 --2- 172.21.15.12:0/3793883724 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5274101550 0x7f527410da30 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:40.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:40.704+0000 7f527a6e0700 1 --2- 172.21.15.12:0/3793883724 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5274106bf0 0x7f5274106fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:40.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:40.704+0000 7f527a6e0700 1 --2- 172.21.15.12:0/3793883724 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5274100bb0 0x7f5274101010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:40.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:40.704+0000 7f527a6e0700 1 -- 172.21.15.12:0/3793883724 >> 172.21.15.12:0/3793883724 conn(0x7f5274077ae0 msgr2=0x7f5274077ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:39:40.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:40.704+0000 7f527a6e0700 1 -- 172.21.15.12:0/3793883724 shutdown_connections 2024-02-18T11:39:40.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:40.704+0000 7f527a6e0700 1 -- 172.21.15.12:0/3793883724 wait complete. 2024-02-18T11:39:40.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:40.705+0000 7f527a6e0700 1 Processor -- start 2024-02-18T11:39:40.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:40.705+0000 7f527a6e0700 1 -- start start 2024-02-18T11:39:40.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:40.705+0000 7f527a6e0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5274100bb0 0x7f52741990b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:40.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:40.705+0000 7f527a6e0700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5274101550 0x7f52741995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:40.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:40.705+0000 7f527a6e0700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5274106bf0 0x7f52741a4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:40.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:40.705+0000 7f527a6e0700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5274199e90 con 0x7f5274100bb0 2024-02-18T11:39:40.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:40.705+0000 7f527a6e0700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52741a4670 con 0x7f5274106bf0 2024-02-18T11:39:40.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:40.705+0000 7f527a6e0700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52741a47b0 con 0x7f5274101550 2024-02-18T11:39:40.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:40.706+0000 7f5273fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5274100bb0 0x7f52741990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:40.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:40.706+0000 7f5273fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5274100bb0 0x7f52741990b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40962/0 (socket says 172.21.15.12:40962) 2024-02-18T11:39:40.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:40.706+0000 7f5273fff700 1 -- 172.21.15.12:0/808949685 learned_addr learned my addr 172.21.15.12:0/808949685 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:39:40.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:40.706+0000 7f52737fe700 1 --2- 172.21.15.12:0/808949685 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5274101550 0x7f52741995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:40.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:40.706+0000 7f5278c7d700 1 --2- 172.21.15.12:0/808949685 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5274106bf0 0x7f52741a4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:40.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:40.706+0000 7f5273fff700 1 -- 172.21.15.12:0/808949685 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5274101550 msgr2=0x7f52741995f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:40.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:40.706+0000 7f5273fff700 1 --2- 172.21.15.12:0/808949685 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5274101550 0x7f52741995f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:40.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:40.706+0000 7f5273fff700 1 -- 172.21.15.12:0/808949685 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5274106bf0 msgr2=0x7f52741a4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:40.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:40.706+0000 7f5273fff700 1 --2- 172.21.15.12:0/808949685 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5274106bf0 0x7f52741a4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:40.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:40.706+0000 7f5273fff700 1 -- 172.21.15.12:0/808949685 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f526c009d00 con 0x7f5274100bb0 2024-02-18T11:39:40.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:40.706+0000 7f5273fff700 1 --2- 172.21.15.12:0/808949685 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5274100bb0 0x7f52741990b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f526400d980 tx=0x7f526400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:39:40.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:40.706+0000 7f52717fa700 1 -- 172.21.15.12:0/808949685 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52640098d0 con 0x7f5274100bb0 2024-02-18T11:39:40.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:40.707+0000 7f52717fa700 1 -- 172.21.15.12:0/808949685 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5264010460 con 0x7f5274100bb0 2024-02-18T11:39:40.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:40.707+0000 7f527a6e0700 1 -- 172.21.15.12:0/808949685 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52741a4a60 con 0x7f5274100bb0 2024-02-18T11:39:40.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:40.707+0000 7f52717fa700 1 -- 172.21.15.12:0/808949685 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f526400f620 con 0x7f5274100bb0 2024-02-18T11:39:40.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:40.707+0000 7f527a6e0700 1 -- 172.21.15.12:0/808949685 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52741a4fb0 con 0x7f5274100bb0 2024-02-18T11:39:40.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:40.708+0000 7f527a6e0700 1 -- 172.21.15.12:0/808949685 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5274109110 con 0x7f5274100bb0 2024-02-18T11:39:40.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:40.709+0000 7f52717fa700 1 -- 172.21.15.12:0/808949685 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 137296+0+0 (secure 0 0 0) 0x7f52640108f0 con 0x7f5274100bb0 2024-02-18T11:39:40.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:40.710+0000 7f52717fa700 1 --2- 172.21.15.12:0/808949685 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f525c0a28f0 0x7f525c0a4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:40.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:40.710+0000 7f52717fa700 1 -- 172.21.15.12:0/808949685 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f52640ce1c0 con 0x7f5274100bb0 2024-02-18T11:39:40.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:40.710+0000 7f52737fe700 1 --2- 172.21.15.12:0/808949685 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f525c0a28f0 0x7f525c0a4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:40.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:40.710+0000 7f52737fe700 1 --2- 172.21.15.12:0/808949685 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f525c0a28f0 0x7f525c0a4db0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5274105970 tx=0x7f526800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:39:40.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:40.714+0000 7f52717fa700 1 -- 172.21.15.12:0/808949685 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5264096fa0 con 0x7f5274100bb0 2024-02-18T11:39:40.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:40.924+0000 7f527a6e0700 1 -- 172.21.15.12:0/808949685 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f5274110bd0 con 0x7f5274100bb0 2024-02-18T11:39:40.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:40.924+0000 7f52717fa700 1 -- 172.21.15.12:0/808949685 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5264020090 con 0x7f5274100bb0 2024-02-18T11:39:40.919 INFO:teuthology.orchestra.run.smithi012.stdout:206158430257 2024-02-18T11:39:40.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:40.927+0000 7f527a6e0700 1 -- 172.21.15.12:0/808949685 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f525c0a28f0 msgr2=0x7f525c0a4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:40.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:40.927+0000 7f527a6e0700 1 --2- 172.21.15.12:0/808949685 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f525c0a28f0 0x7f525c0a4db0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5274105970 tx=0x7f526800a380 comp rx=0 tx=0).stop 2024-02-18T11:39:40.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:40.927+0000 7f527a6e0700 1 -- 172.21.15.12:0/808949685 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5274100bb0 msgr2=0x7f52741990b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:40.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:40.927+0000 7f527a6e0700 1 --2- 172.21.15.12:0/808949685 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5274100bb0 0x7f52741990b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f526400d980 tx=0x7f526400dc90 comp rx=0 tx=0).stop 2024-02-18T11:39:40.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:40.927+0000 7f527a6e0700 1 -- 172.21.15.12:0/808949685 shutdown_connections 2024-02-18T11:39:40.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:40.927+0000 7f527a6e0700 1 --2- 172.21.15.12:0/808949685 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5274106bf0 0x7f52741a4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:40.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:40.927+0000 7f527a6e0700 1 --2- 172.21.15.12:0/808949685 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f525c0a28f0 0x7f525c0a4db0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:40.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:40.927+0000 7f527a6e0700 1 --2- 172.21.15.12:0/808949685 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5274100bb0 0x7f52741990b0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:40.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:40.927+0000 7f527a6e0700 1 --2- 172.21.15.12:0/808949685 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5274101550 0x7f52741995f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:40.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:40.927+0000 7f527a6e0700 1 -- 172.21.15.12:0/808949685 >> 172.21.15.12:0/808949685 conn(0x7f5274077ae0 msgr2=0x7f52740ff660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:39:40.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:40.927+0000 7f527a6e0700 1 -- 172.21.15.12:0/808949685 shutdown_connections 2024-02-18T11:39:40.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:40.927+0000 7f527a6e0700 1 -- 172.21.15.12:0/808949685 wait complete. 2024-02-18T11:39:41.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:39:41 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/808949685' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-02-18T11:39:41.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:39:41 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/808949685' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-02-18T11:39:41.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:39:41 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/808949685' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-02-18T11:39:41.774 INFO:tasks.cephadm.ceph_manager.ceph:need seq 171798691891 got 171798691893 for osd.5 2024-02-18T11:39:41.775 DEBUG:teuthology.parallel:result is None 2024-02-18T11:39:42.382 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:39:42 smithi012 ceph-mon[32460]: pgmap v312: 97 pgs: 97 active+clean; 639 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-02-18T11:39:42.416 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430255 got 206158430257 for osd.6 2024-02-18T11:39:42.416 DEBUG:teuthology.parallel:result is None 2024-02-18T11:39:42.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:39:42 smithi158 ceph-mon[36626]: pgmap v312: 97 pgs: 97 active+clean; 639 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-02-18T11:39:42.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:39:42 smithi188 ceph-mon[36925]: pgmap v312: 97 pgs: 97 active+clean; 639 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-02-18T11:39:44.423 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:39:44 smithi012 ceph-mon[32460]: pgmap v313: 97 pgs: 97 active+clean; 639 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-02-18T11:39:44.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:39:44 smithi158 ceph-mon[36626]: pgmap v313: 97 pgs: 97 active+clean; 639 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-02-18T11:39:44.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:39:44 smithi188 ceph-mon[36925]: pgmap v313: 97 pgs: 97 active+clean; 639 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-02-18T11:39:44.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:44.731+0000 7f93fa302700 1 -- 172.21.15.12:0/3048139151 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f93f4101200 msgr2=0x7f93f4106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:44.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:44.731+0000 7f93fa302700 1 --2- 172.21.15.12:0/3048139151 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f93f4101200 0x7f93f4106940 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f93e8009990 tx=0x7f93e8009ca0 comp rx=0 tx=0).stop 2024-02-18T11:39:44.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:44.731+0000 7f93fa302700 1 -- 172.21.15.12:0/3048139151 shutdown_connections 2024-02-18T11:39:44.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:44.731+0000 7f93fa302700 1 --2- 172.21.15.12:0/3048139151 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f93f4107070 0x7f93f410ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:44.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:44.731+0000 7f93fa302700 1 --2- 172.21.15.12:0/3048139151 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93f4100850 0x7f93f4100c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:44.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:44.731+0000 7f93fa302700 1 --2- 172.21.15.12:0/3048139151 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f93f4101200 0x7f93f4106940 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:44.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:44.731+0000 7f93fa302700 1 -- 172.21.15.12:0/3048139151 >> 172.21.15.12:0/3048139151 conn(0x7f93f4075ef0 msgr2=0x7f93f4076300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:39:44.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:44.731+0000 7f93fa302700 1 -- 172.21.15.12:0/3048139151 shutdown_connections 2024-02-18T11:39:44.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:44.731+0000 7f93fa302700 1 -- 172.21.15.12:0/3048139151 wait complete. 2024-02-18T11:39:44.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:44.732+0000 7f93fa302700 1 Processor -- start 2024-02-18T11:39:44.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:44.732+0000 7f93fa302700 1 -- start start 2024-02-18T11:39:44.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:44.732+0000 7f93fa302700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93f4100850 0x7f93f419a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:44.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:44.732+0000 7f93fa302700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f93f4101200 0x7f93f419a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:44.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:44.733+0000 7f93fa302700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f93f4107070 0x7f93f419ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:44.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:44.733+0000 7f93fa302700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f419f2f0 con 0x7f93f4107070 2024-02-18T11:39:44.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:44.733+0000 7f93fa302700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f419f460 con 0x7f93f4101200 2024-02-18T11:39:44.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:44.733+0000 7f93fa302700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f419f620 con 0x7f93f4100850 2024-02-18T11:39:44.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:44.733+0000 7f93f889f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f93f4107070 0x7f93f419ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:44.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:44.733+0000 7f93f889f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f93f4107070 0x7f93f419ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59354/0 (socket says 172.21.15.12:59354) 2024-02-18T11:39:44.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:44.733+0000 7f93f889f700 1 -- 172.21.15.12:0/2190076996 learned_addr learned my addr 172.21.15.12:0/2190076996 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:39:44.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:44.733+0000 7f93f37fe700 1 --2- 172.21.15.12:0/2190076996 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f93f4101200 0x7f93f419a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:44.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:44.733+0000 7f93f3fff700 1 --2- 172.21.15.12:0/2190076996 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93f4100850 0x7f93f419a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:44.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:44.733+0000 7f93f889f700 1 -- 172.21.15.12:0/2190076996 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93f4100850 msgr2=0x7f93f419a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:44.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:44.733+0000 7f93f889f700 1 --2- 172.21.15.12:0/2190076996 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93f4100850 0x7f93f419a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:44.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:44.733+0000 7f93f889f700 1 -- 172.21.15.12:0/2190076996 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f93f4101200 msgr2=0x7f93f419a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:44.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:44.733+0000 7f93f889f700 1 --2- 172.21.15.12:0/2190076996 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f93f4101200 0x7f93f419a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:44.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:44.733+0000 7f93f889f700 1 -- 172.21.15.12:0/2190076996 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93e80095f0 con 0x7f93f4107070 2024-02-18T11:39:44.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:44.733+0000 7f93f37fe700 1 --2- 172.21.15.12:0/2190076996 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f93f4101200 0x7f93f419a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:39:44.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:44.733+0000 7f93f889f700 1 --2- 172.21.15.12:0/2190076996 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f93f4107070 0x7f93f419ece0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f93ec007d70 tx=0x7f93ec00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:39:44.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:44.733+0000 7f93f17fa700 1 -- 172.21.15.12:0/2190076996 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93ec00ad40 con 0x7f93f4107070 2024-02-18T11:39:44.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:44.733+0000 7f93f17fa700 1 -- 172.21.15.12:0/2190076996 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f93ec012900 con 0x7f93f4107070 2024-02-18T11:39:44.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:44.733+0000 7f93fa302700 1 -- 172.21.15.12:0/2190076996 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93f419f900 con 0x7f93f4107070 2024-02-18T11:39:44.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:44.733+0000 7f93f17fa700 1 -- 172.21.15.12:0/2190076996 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93ec011880 con 0x7f93f4107070 2024-02-18T11:39:44.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:44.733+0000 7f93fa302700 1 -- 172.21.15.12:0/2190076996 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93f41a7160 con 0x7f93f4107070 2024-02-18T11:39:44.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:44.734+0000 7f93fa302700 1 -- 172.21.15.12:0/2190076996 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93f4103eb0 con 0x7f93f4107070 2024-02-18T11:39:44.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:44.736+0000 7f93f17fa700 1 -- 172.21.15.12:0/2190076996 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 137296+0+0 (secure 0 0 0) 0x7f93ec012a70 con 0x7f93f4107070 2024-02-18T11:39:44.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:44.737+0000 7f93f17fa700 1 --2- 172.21.15.12:0/2190076996 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f93dc0a28f0 0x7f93dc0a4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:44.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:44.737+0000 7f93f3fff700 1 --2- 172.21.15.12:0/2190076996 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f93dc0a28f0 0x7f93dc0a4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:44.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:44.737+0000 7f93f17fa700 1 -- 172.21.15.12:0/2190076996 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f93ec0cebf0 con 0x7f93f4107070 2024-02-18T11:39:44.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:44.737+0000 7f93f3fff700 1 --2- 172.21.15.12:0/2190076996 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f93dc0a28f0 0x7f93dc0a4db0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f93e4006fd0 tx=0x7f93e4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:39:44.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:44.741+0000 7f93f17fa700 1 -- 172.21.15.12:0/2190076996 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f93ec0979e0 con 0x7f93f4107070 2024-02-18T11:39:44.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:44.947+0000 7f93fa302700 1 -- 172.21.15.12:0/2190076996 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7f93f4068970 con 0x7f93f4107070 2024-02-18T11:39:44.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:44.947+0000 7f93f17fa700 1 -- 172.21.15.12:0/2190076996 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f93ec016020 con 0x7f93f4107070 2024-02-18T11:39:44.944 INFO:teuthology.orchestra.run.smithi012.stdout:296352743458 2024-02-18T11:39:44.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:44.949+0000 7f93fa302700 1 -- 172.21.15.12:0/2190076996 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f93dc0a28f0 msgr2=0x7f93dc0a4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:44.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:44.949+0000 7f93fa302700 1 --2- 172.21.15.12:0/2190076996 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f93dc0a28f0 0x7f93dc0a4db0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f93e4006fd0 tx=0x7f93e4008040 comp rx=0 tx=0).stop 2024-02-18T11:39:44.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:44.949+0000 7f93fa302700 1 -- 172.21.15.12:0/2190076996 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f93f4107070 msgr2=0x7f93f419ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:44.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:44.949+0000 7f93fa302700 1 --2- 172.21.15.12:0/2190076996 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f93f4107070 0x7f93f419ece0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f93ec007d70 tx=0x7f93ec00a340 comp rx=0 tx=0).stop 2024-02-18T11:39:44.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:44.950+0000 7f93fa302700 1 -- 172.21.15.12:0/2190076996 shutdown_connections 2024-02-18T11:39:44.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:44.950+0000 7f93fa302700 1 --2- 172.21.15.12:0/2190076996 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f93dc0a28f0 0x7f93dc0a4db0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:44.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:44.950+0000 7f93fa302700 1 --2- 172.21.15.12:0/2190076996 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f93f4107070 0x7f93f419ece0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:44.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:44.950+0000 7f93fa302700 1 --2- 172.21.15.12:0/2190076996 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93f4100850 0x7f93f419a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:44.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:44.950+0000 7f93fa302700 1 --2- 172.21.15.12:0/2190076996 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f93f4101200 0x7f93f419a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:44.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:44.950+0000 7f93fa302700 1 -- 172.21.15.12:0/2190076996 >> 172.21.15.12:0/2190076996 conn(0x7f93f4075ef0 msgr2=0x7f93f40fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:39:44.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:44.950+0000 7f93fa302700 1 -- 172.21.15.12:0/2190076996 shutdown_connections 2024-02-18T11:39:44.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:44.950+0000 7f93fa302700 1 -- 172.21.15.12:0/2190076996 wait complete. 2024-02-18T11:39:45.265 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:39:45 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/2190076996' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-02-18T11:39:45.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:39:45 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/2190076996' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-02-18T11:39:45.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:39:45 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/2190076996' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-02-18T11:39:46.244 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:39:46 smithi012 ceph-mon[32460]: pgmap v314: 97 pgs: 97 active+clean; 639 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-18T11:39:46.303 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T11:39:46.304 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T11:39:46.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:39:46 smithi158 ceph-mon[36626]: pgmap v314: 97 pgs: 97 active+clean; 639 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-18T11:39:46.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:39:46 smithi188 ceph-mon[36925]: pgmap v314: 97 pgs: 97 active+clean; 639 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-18T11:39:47.368 INFO:tasks.cephadm.ceph_manager.ceph:need seq 296352743454 got 296352743458 for osd.9 2024-02-18T11:39:47.368 DEBUG:teuthology.parallel:result is None 2024-02-18T11:39:47.423 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:39:47 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/315924845' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-18T11:39:47.423 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:39:47 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:39:47.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:39:47 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/315924845' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-18T11:39:47.495 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:39:47 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:39:47.572 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:39:47 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/315924845' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-18T11:39:47.572 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:39:47 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:39:48.376 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:39:48 smithi012 ceph-mon[32460]: pgmap v315: 97 pgs: 97 active+clean; 639 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-02-18T11:39:48.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.488+0000 7f2e67fd6700 1 -- 172.21.15.12:0/1186453572 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e60106cc0 msgr2=0x7f2e601070a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:48.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.488+0000 7f2e67fd6700 1 --2- 172.21.15.12:0/1186453572 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e60106cc0 0x7f2e601070a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2e5000b0a0 tx=0x7f2e5000b3b0 comp rx=0 tx=0).stop 2024-02-18T11:39:48.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.488+0000 7f2e67fd6700 1 -- 172.21.15.12:0/1186453572 shutdown_connections 2024-02-18T11:39:48.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.488+0000 7f2e67fd6700 1 --2- 172.21.15.12:0/1186453572 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2e6006adf0 0x7f2e6010eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:48.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.488+0000 7f2e67fd6700 1 --2- 172.21.15.12:0/1186453572 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e60106cc0 0x7f2e601070a0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:48.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.488+0000 7f2e67fd6700 1 --2- 172.21.15.12:0/1186453572 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2e6006a430 0x7f2e6006a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:48.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.488+0000 7f2e67fd6700 1 -- 172.21.15.12:0/1186453572 >> 172.21.15.12:0/1186453572 conn(0x7f2e60077c10 msgr2=0x7f2e60078020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:39:48.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.489+0000 7f2e67fd6700 1 -- 172.21.15.12:0/1186453572 shutdown_connections 2024-02-18T11:39:48.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.489+0000 7f2e67fd6700 1 -- 172.21.15.12:0/1186453572 wait complete. 2024-02-18T11:39:48.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.489+0000 7f2e67fd6700 1 Processor -- start 2024-02-18T11:39:48.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.489+0000 7f2e67fd6700 1 -- start start 2024-02-18T11:39:48.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.489+0000 7f2e67fd6700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2e6006a430 0x7f2e60100c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:48.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.489+0000 7f2e67fd6700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2e6006adf0 0x7f2e601011a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:48.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.489+0000 7f2e67fd6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e60106cc0 0x7f2e601016e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:48.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.489+0000 7f2e67fd6700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e601a67d0 con 0x7f2e60106cc0 2024-02-18T11:39:48.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.489+0000 7f2e67fd6700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e601a6940 con 0x7f2e6006a430 2024-02-18T11:39:48.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.489+0000 7f2e67fd6700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e601a6b00 con 0x7f2e6006adf0 2024-02-18T11:39:48.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.490+0000 7f2e65d72700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2e6006a430 0x7f2e60100c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:48.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.490+0000 7f2e65571700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2e6006adf0 0x7f2e601011a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:48.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.490+0000 7f2e65d72700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2e6006a430 0x7f2e60100c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.158:3300/0 says I am v2:172.21.15.12:43864/0 (socket says 172.21.15.12:43864) 2024-02-18T11:39:48.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.490+0000 7f2e65d72700 1 -- 172.21.15.12:0/2590784890 learned_addr learned my addr 172.21.15.12:0/2590784890 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:39:48.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.490+0000 7f2e66573700 1 --2- 172.21.15.12:0/2590784890 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e60106cc0 0x7f2e601016e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:48.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.490+0000 7f2e65d72700 1 -- 172.21.15.12:0/2590784890 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2e6006adf0 msgr2=0x7f2e601011a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:48.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.490+0000 7f2e65d72700 1 --2- 172.21.15.12:0/2590784890 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2e6006adf0 0x7f2e601011a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:48.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.490+0000 7f2e65d72700 1 -- 172.21.15.12:0/2590784890 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e60106cc0 msgr2=0x7f2e601016e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:48.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.490+0000 7f2e65d72700 1 --2- 172.21.15.12:0/2590784890 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e60106cc0 0x7f2e601016e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:48.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.490+0000 7f2e65d72700 1 -- 172.21.15.12:0/2590784890 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e50009d00 con 0x7f2e6006a430 2024-02-18T11:39:48.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.490+0000 7f2e65d72700 1 --2- 172.21.15.12:0/2590784890 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2e6006a430 0x7f2e60100c60 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2e5000b780 tx=0x7f2e5000bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:39:48.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.490+0000 7f2e5effd700 1 -- 172.21.15.12:0/2590784890 <== mon.2 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e5000e040 con 0x7f2e6006a430 2024-02-18T11:39:48.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.490+0000 7f2e5effd700 1 -- 172.21.15.12:0/2590784890 <== mon.2 v2:172.21.15.158:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2e5001d470 con 0x7f2e6006a430 2024-02-18T11:39:48.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.490+0000 7f2e67fd6700 1 -- 172.21.15.12:0/2590784890 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e601a6d80 con 0x7f2e6006a430 2024-02-18T11:39:48.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.490+0000 7f2e5effd700 1 -- 172.21.15.12:0/2590784890 <== mon.2 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e500052a0 con 0x7f2e6006a430 2024-02-18T11:39:48.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.490+0000 7f2e67fd6700 1 -- 172.21.15.12:0/2590784890 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e601a71a0 con 0x7f2e6006a430 2024-02-18T11:39:48.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.491+0000 7f2e67fd6700 1 -- 172.21.15.12:0/2590784890 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e6010a220 con 0x7f2e6006a430 2024-02-18T11:39:48.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.492+0000 7f2e5effd700 1 -- 172.21.15.12:0/2590784890 <== mon.2 v2:172.21.15.158:3300/0 4 ==== mgrmap(e 17) v1 ==== 137296+0+0 (secure 0 0 0) 0x7f2e5001d5e0 con 0x7f2e6006a430 2024-02-18T11:39:48.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.493+0000 7f2e5effd700 1 --2- 172.21.15.12:0/2590784890 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f2e480a2870 0x7f2e480a4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:48.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.493+0000 7f2e65571700 1 --2- 172.21.15.12:0/2590784890 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f2e480a2870 0x7f2e480a4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:48.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.493+0000 7f2e5effd700 1 -- 172.21.15.12:0/2590784890 <== mon.2 v2:172.21.15.158:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f2e500cf8e0 con 0x7f2e6006a430 2024-02-18T11:39:48.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.493+0000 7f2e65571700 1 --2- 172.21.15.12:0/2590784890 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f2e480a2870 0x7f2e480a4d30 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2e60078180 tx=0x7f2e54009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:39:48.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.497+0000 7f2e5effd700 1 -- 172.21.15.12:0/2590784890 <== mon.2 v2:172.21.15.158:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2e50098650 con 0x7f2e6006a430 2024-02-18T11:39:48.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:39:48 smithi158 ceph-mon[36626]: pgmap v315: 97 pgs: 97 active+clean; 639 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-02-18T11:39:48.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:39:48 smithi188 ceph-mon[36925]: pgmap v315: 97 pgs: 97 active+clean; 639 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-02-18T11:39:48.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.707+0000 7f2e67fd6700 1 -- 172.21.15.12:0/2590784890 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f2e60102190 con 0x7f2e6006a430 2024-02-18T11:39:48.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.712+0000 7f2e5effd700 1 -- 172.21.15.12:0/2590784890 <== mon.2 v2:172.21.15.158:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2e5009bd10 con 0x7f2e6006a430 2024-02-18T11:39:48.706 INFO:teuthology.orchestra.run.smithi012.stdout:227633266734 2024-02-18T11:39:48.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.714+0000 7f2e67fd6700 1 -- 172.21.15.12:0/2590784890 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f2e480a2870 msgr2=0x7f2e480a4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:48.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.714+0000 7f2e67fd6700 1 --2- 172.21.15.12:0/2590784890 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f2e480a2870 0x7f2e480a4d30 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2e60078180 tx=0x7f2e54009260 comp rx=0 tx=0).stop 2024-02-18T11:39:48.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.714+0000 7f2e67fd6700 1 -- 172.21.15.12:0/2590784890 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2e6006a430 msgr2=0x7f2e60100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:48.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.714+0000 7f2e67fd6700 1 --2- 172.21.15.12:0/2590784890 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2e6006a430 0x7f2e60100c60 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2e5000b780 tx=0x7f2e5000bea0 comp rx=0 tx=0).stop 2024-02-18T11:39:48.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.714+0000 7f2e67fd6700 1 -- 172.21.15.12:0/2590784890 shutdown_connections 2024-02-18T11:39:48.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.714+0000 7f2e67fd6700 1 --2- 172.21.15.12:0/2590784890 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f2e480a2870 0x7f2e480a4d30 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:48.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.714+0000 7f2e67fd6700 1 --2- 172.21.15.12:0/2590784890 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e60106cc0 0x7f2e601016e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:48.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.714+0000 7f2e67fd6700 1 --2- 172.21.15.12:0/2590784890 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2e6006a430 0x7f2e60100c60 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:48.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.714+0000 7f2e67fd6700 1 --2- 172.21.15.12:0/2590784890 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2e6006adf0 0x7f2e601011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:48.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.714+0000 7f2e67fd6700 1 -- 172.21.15.12:0/2590784890 >> 172.21.15.12:0/2590784890 conn(0x7f2e60077c10 msgr2=0x7f2e600feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:39:48.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.714+0000 7f2e67fd6700 1 -- 172.21.15.12:0/2590784890 shutdown_connections 2024-02-18T11:39:48.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.714+0000 7f2e67fd6700 1 -- 172.21.15.12:0/2590784890 wait complete. 2024-02-18T11:39:48.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.703+0000 7f31178f4700 1 -- 172.21.15.12:0/3617731957 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f311006a450 msgr2=0x7f311006a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:48.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.703+0000 7f31178f4700 1 --2- 172.21.15.12:0/3617731957 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f311006a450 0x7f311006a830 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3100009990 tx=0x7f3100009ca0 comp rx=0 tx=0).stop 2024-02-18T11:39:48.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.703+0000 7f31178f4700 1 -- 172.21.15.12:0/3617731957 shutdown_connections 2024-02-18T11:39:48.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.703+0000 7f31178f4700 1 --2- 172.21.15.12:0/3617731957 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3110107030 0x7f311010ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:48.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.703+0000 7f31178f4700 1 --2- 172.21.15.12:0/3617731957 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f311006a450 0x7f311006a830 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:48.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.703+0000 7f31178f4700 1 --2- 172.21.15.12:0/3617731957 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f311006ae00 0x7f3110106900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:48.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.703+0000 7f31178f4700 1 -- 172.21.15.12:0/3617731957 >> 172.21.15.12:0/3617731957 conn(0x7f3110076c00 msgr2=0x7f3110077010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:39:48.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.703+0000 7f31178f4700 1 -- 172.21.15.12:0/3617731957 shutdown_connections 2024-02-18T11:39:48.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.703+0000 7f31178f4700 1 -- 172.21.15.12:0/3617731957 wait complete. 2024-02-18T11:39:48.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.704+0000 7f31178f4700 1 Processor -- start 2024-02-18T11:39:48.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.704+0000 7f31178f4700 1 -- start start 2024-02-18T11:39:48.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.704+0000 7f31178f4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f311006a450 0x7f3110100cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:48.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.704+0000 7f31178f4700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f311006ae00 0x7f31101011f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:48.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.704+0000 7f31178f4700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3110107030 0x7f3110101730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:48.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.704+0000 7f31178f4700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31101a65f0 con 0x7f311006a450 2024-02-18T11:39:48.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.704+0000 7f31178f4700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31101a6760 con 0x7f3110107030 2024-02-18T11:39:48.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.704+0000 7f31178f4700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31101a6920 con 0x7f311006ae00 2024-02-18T11:39:48.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.704+0000 7f3115690700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f311006a450 0x7f3110100cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:48.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.704+0000 7f3115e91700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3110107030 0x7f3110101730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:48.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.704+0000 7f3114e8f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f311006ae00 0x7f31101011f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:48.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.704+0000 7f3115690700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f311006a450 0x7f3110100cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59400/0 (socket says 172.21.15.12:59400) 2024-02-18T11:39:48.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.704+0000 7f3115690700 1 -- 172.21.15.12:0/3640018799 learned_addr learned my addr 172.21.15.12:0/3640018799 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:39:48.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.704+0000 7f3115e91700 1 -- 172.21.15.12:0/3640018799 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f311006ae00 msgr2=0x7f31101011f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:48.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.704+0000 7f3115e91700 1 --2- 172.21.15.12:0/3640018799 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f311006ae00 0x7f31101011f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:48.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.704+0000 7f3115e91700 1 -- 172.21.15.12:0/3640018799 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f311006a450 msgr2=0x7f3110100cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:48.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.704+0000 7f3115e91700 1 --2- 172.21.15.12:0/3640018799 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f311006a450 0x7f3110100cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:48.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.704+0000 7f3115e91700 1 -- 172.21.15.12:0/3640018799 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31000095f0 con 0x7f3110107030 2024-02-18T11:39:48.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.704+0000 7f3114e8f700 1 --2- 172.21.15.12:0/3640018799 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f311006ae00 0x7f31101011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-18T11:39:48.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.704+0000 7f3115690700 1 --2- 172.21.15.12:0/3640018799 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f311006a450 0x7f3110100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-18T11:39:48.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.705+0000 7f3115e91700 1 --2- 172.21.15.12:0/3640018799 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3110107030 0x7f3110101730 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3108007f10 tx=0x7f310800d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:39:48.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.705+0000 7f31067fc700 1 -- 172.21.15.12:0/3640018799 <== mon.2 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3108009ea0 con 0x7f3110107030 2024-02-18T11:39:48.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.705+0000 7f31067fc700 1 -- 172.21.15.12:0/3640018799 <== mon.2 v2:172.21.15.158:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f310800f040 con 0x7f3110107030 2024-02-18T11:39:48.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.705+0000 7f31178f4700 1 -- 172.21.15.12:0/3640018799 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31101a6c00 con 0x7f3110107030 2024-02-18T11:39:48.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.705+0000 7f31067fc700 1 -- 172.21.15.12:0/3640018799 <== mon.2 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3108004400 con 0x7f3110107030 2024-02-18T11:39:48.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.705+0000 7f31178f4700 1 -- 172.21.15.12:0/3640018799 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31101a7100 con 0x7f3110107030 2024-02-18T11:39:48.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.706+0000 7f31178f4700 1 -- 172.21.15.12:0/3640018799 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f311010a200 con 0x7f3110107030 2024-02-18T11:39:48.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.707+0000 7f31067fc700 1 -- 172.21.15.12:0/3640018799 <== mon.2 v2:172.21.15.158:3300/0 4 ==== mgrmap(e 17) v1 ==== 137296+0+0 (secure 0 0 0) 0x7f3108003a00 con 0x7f3110107030 2024-02-18T11:39:48.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.707+0000 7f31067fc700 1 --2- 172.21.15.12:0/3640018799 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f30fc0a28f0 0x7f30fc0a4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:48.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.708+0000 7f3115690700 1 --2- 172.21.15.12:0/3640018799 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f30fc0a28f0 0x7f30fc0a4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:48.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.708+0000 7f31067fc700 1 -- 172.21.15.12:0/3640018799 <== mon.2 v2:172.21.15.158:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f31080cdd40 con 0x7f3110107030 2024-02-18T11:39:48.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.708+0000 7f3115690700 1 --2- 172.21.15.12:0/3640018799 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f30fc0a28f0 0x7f30fc0a4db0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f310000b5c0 tx=0x7f310000bb80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:39:48.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.712+0000 7f31067fc700 1 -- 172.21.15.12:0/3640018799 <== mon.2 v2:172.21.15.158:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3108096aa0 con 0x7f3110107030 2024-02-18T11:39:48.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.915+0000 7f31178f4700 1 -- 172.21.15.12:0/3640018799 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f3110111d40 con 0x7f3110107030 2024-02-18T11:39:48.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.916+0000 7f31067fc700 1 -- 172.21.15.12:0/3640018799 <== mon.2 v2:172.21.15.158:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3108019020 con 0x7f3110107030 2024-02-18T11:39:48.910 INFO:teuthology.orchestra.run.smithi012.stdout:115964117057 2024-02-18T11:39:48.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.918+0000 7f31178f4700 1 -- 172.21.15.12:0/3640018799 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f30fc0a28f0 msgr2=0x7f30fc0a4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:48.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.918+0000 7f31178f4700 1 --2- 172.21.15.12:0/3640018799 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f30fc0a28f0 0x7f30fc0a4db0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f310000b5c0 tx=0x7f310000bb80 comp rx=0 tx=0).stop 2024-02-18T11:39:48.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.918+0000 7f31178f4700 1 -- 172.21.15.12:0/3640018799 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3110107030 msgr2=0x7f3110101730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:48.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.918+0000 7f31178f4700 1 --2- 172.21.15.12:0/3640018799 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3110107030 0x7f3110101730 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3108007f10 tx=0x7f310800d3b0 comp rx=0 tx=0).stop 2024-02-18T11:39:48.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.918+0000 7f31178f4700 1 -- 172.21.15.12:0/3640018799 shutdown_connections 2024-02-18T11:39:48.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.918+0000 7f31178f4700 1 --2- 172.21.15.12:0/3640018799 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3110107030 0x7f3110101730 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:48.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.918+0000 7f31178f4700 1 --2- 172.21.15.12:0/3640018799 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f30fc0a28f0 0x7f30fc0a4db0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:48.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.918+0000 7f31178f4700 1 --2- 172.21.15.12:0/3640018799 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f311006a450 0x7f3110100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:48.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.918+0000 7f31178f4700 1 --2- 172.21.15.12:0/3640018799 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f311006ae00 0x7f31101011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:48.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.918+0000 7f31178f4700 1 -- 172.21.15.12:0/3640018799 >> 172.21.15.12:0/3640018799 conn(0x7f3110076c00 msgr2=0x7f31100fef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:39:48.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.918+0000 7f31178f4700 1 -- 172.21.15.12:0/3640018799 shutdown_connections 2024-02-18T11:39:48.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.918+0000 7f31178f4700 1 -- 172.21.15.12:0/3640018799 wait complete. 2024-02-18T11:39:48.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.998+0000 7f6007239700 1 -- 172.21.15.12:0/554092325 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6000069960 msgr2=0x7f6000069dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:48.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.998+0000 7f6007239700 1 --2- 172.21.15.12:0/554092325 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6000069960 0x7f6000069dc0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5fec0099c0 tx=0x7f5fec009cd0 comp rx=0 tx=0).stop 2024-02-18T11:39:48.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.998+0000 7f6007239700 1 -- 172.21.15.12:0/554092325 shutdown_connections 2024-02-18T11:39:48.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.998+0000 7f6007239700 1 --2- 172.21.15.12:0/554092325 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f600006a300 0x7f600010eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:48.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.998+0000 7f6007239700 1 --2- 172.21.15.12:0/554092325 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6000107d00 0x7f60001080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:48.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.998+0000 7f6007239700 1 --2- 172.21.15.12:0/554092325 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6000069960 0x7f6000069dc0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:48.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.998+0000 7f6007239700 1 -- 172.21.15.12:0/554092325 >> 172.21.15.12:0/554092325 conn(0x7f6000077ae0 msgr2=0x7f6000077ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:39:48.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.998+0000 7f6007239700 1 -- 172.21.15.12:0/554092325 shutdown_connections 2024-02-18T11:39:48.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.998+0000 7f6007239700 1 -- 172.21.15.12:0/554092325 wait complete. 2024-02-18T11:39:48.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.999+0000 7f6007239700 1 Processor -- start 2024-02-18T11:39:48.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.999+0000 7f6007239700 1 -- start start 2024-02-18T11:39:48.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.999+0000 7f6007239700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6000069960 0x7f6000100c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:48.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.999+0000 7f6007239700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f600006a300 0x7f60001011c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:48.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.999+0000 7f6007239700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6000107d00 0x7f6000101700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:48.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.999+0000 7f6007239700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60001a6680 con 0x7f6000107d00 2024-02-18T11:39:48.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.999+0000 7f6007239700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60001a67f0 con 0x7f6000069960 2024-02-18T11:39:48.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.999+0000 7f6007239700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60001a69b0 con 0x7f600006a300 2024-02-18T11:39:48.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.999+0000 7f60057d6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6000107d00 0x7f6000101700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:48.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.999+0000 7f60057d6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6000107d00 0x7f6000101700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59408/0 (socket says 172.21.15.12:59408) 2024-02-18T11:39:48.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.999+0000 7f6004fd5700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6000069960 0x7f6000100c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:48.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.999+0000 7f60057d6700 1 -- 172.21.15.12:0/2531766299 learned_addr learned my addr 172.21.15.12:0/2531766299 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:39:48.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.999+0000 7f5ffffff700 1 --2- 172.21.15.12:0/2531766299 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f600006a300 0x7f60001011c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:48.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.999+0000 7f60057d6700 1 -- 172.21.15.12:0/2531766299 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f600006a300 msgr2=0x7f60001011c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:48.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:48.999+0000 7f60057d6700 1 --2- 172.21.15.12:0/2531766299 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f600006a300 0x7f60001011c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:48.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:49.000+0000 7f60057d6700 1 -- 172.21.15.12:0/2531766299 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6000069960 msgr2=0x7f6000100c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:48.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:49.000+0000 7f60057d6700 1 --2- 172.21.15.12:0/2531766299 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6000069960 0x7f6000100c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:48.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:49.000+0000 7f60057d6700 1 -- 172.21.15.12:0/2531766299 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fec0095f0 con 0x7f6000107d00 2024-02-18T11:39:48.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:49.000+0000 7f6004fd5700 1 --2- 172.21.15.12:0/2531766299 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6000069960 0x7f6000100c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-18T11:39:48.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:49.000+0000 7f5ffffff700 1 --2- 172.21.15.12:0/2531766299 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f600006a300 0x7f60001011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-18T11:39:48.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:49.000+0000 7f60057d6700 1 --2- 172.21.15.12:0/2531766299 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6000107d00 0x7f6000101700 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f5ff8007ee0 tx=0x7f5ff800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:39:48.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:49.000+0000 7f5ffdffb700 1 -- 172.21.15.12:0/2531766299 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff800dd20 con 0x7f6000107d00 2024-02-18T11:39:48.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:49.000+0000 7f5ffdffb700 1 -- 172.21.15.12:0/2531766299 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5ff800f040 con 0x7f6000107d00 2024-02-18T11:39:48.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:49.000+0000 7f6007239700 1 -- 172.21.15.12:0/2531766299 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60001a6c90 con 0x7f6000107d00 2024-02-18T11:39:49.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:49.001+0000 7f5ffdffb700 1 -- 172.21.15.12:0/2531766299 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff80048e0 con 0x7f6000107d00 2024-02-18T11:39:49.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:49.001+0000 7f6007239700 1 -- 172.21.15.12:0/2531766299 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60001a7110 con 0x7f6000107d00 2024-02-18T11:39:49.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:49.002+0000 7f6007239700 1 -- 172.21.15.12:0/2531766299 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f600010a220 con 0x7f6000107d00 2024-02-18T11:39:49.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:49.002+0000 7f5ffdffb700 1 -- 172.21.15.12:0/2531766299 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 137296+0+0 (secure 0 0 0) 0x7f5ff8003a00 con 0x7f6000107d00 2024-02-18T11:39:49.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:49.003+0000 7f5ffdffb700 1 --2- 172.21.15.12:0/2531766299 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f5ff00a2870 0x7f5ff00a4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:49.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:49.003+0000 7f6004fd5700 1 --2- 172.21.15.12:0/2531766299 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f5ff00a2870 0x7f5ff00a4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:49.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:49.003+0000 7f5ffdffb700 1 -- 172.21.15.12:0/2531766299 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f5ff80cde90 con 0x7f6000107d00 2024-02-18T11:39:49.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:49.003+0000 7f6004fd5700 1 --2- 172.21.15.12:0/2531766299 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f5ff00a2870 0x7f5ff00a4d30 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5ff4009fd0 tx=0x7f5ff4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:39:49.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:49.007+0000 7f5ffdffb700 1 -- 172.21.15.12:0/2531766299 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5ff804fd20 con 0x7f6000107d00 2024-02-18T11:39:49.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:49.209+0000 7f6007239700 1 -- 172.21.15.12:0/2531766299 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7f60001021b0 con 0x7f6000107d00 2024-02-18T11:39:49.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:49.209+0000 7f5ffdffb700 1 -- 172.21.15.12:0/2531766299 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f5ff8096b70 con 0x7f6000107d00 2024-02-18T11:39:49.203 INFO:teuthology.orchestra.run.smithi012.stdout:352187318297 2024-02-18T11:39:49.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:49.211+0000 7f6007239700 1 -- 172.21.15.12:0/2531766299 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f5ff00a2870 msgr2=0x7f5ff00a4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:49.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:49.211+0000 7f6007239700 1 --2- 172.21.15.12:0/2531766299 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f5ff00a2870 0x7f5ff00a4d30 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5ff4009fd0 tx=0x7f5ff4009260 comp rx=0 tx=0).stop 2024-02-18T11:39:49.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:49.211+0000 7f6007239700 1 -- 172.21.15.12:0/2531766299 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6000107d00 msgr2=0x7f6000101700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:49.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:49.211+0000 7f6007239700 1 --2- 172.21.15.12:0/2531766299 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6000107d00 0x7f6000101700 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f5ff8007ee0 tx=0x7f5ff800d3b0 comp rx=0 tx=0).stop 2024-02-18T11:39:49.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:49.211+0000 7f6007239700 1 -- 172.21.15.12:0/2531766299 shutdown_connections 2024-02-18T11:39:49.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:49.211+0000 7f6007239700 1 --2- 172.21.15.12:0/2531766299 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f5ff00a2870 0x7f5ff00a4d30 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:49.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:49.211+0000 7f6007239700 1 --2- 172.21.15.12:0/2531766299 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6000107d00 0x7f6000101700 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:49.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:49.211+0000 7f6007239700 1 --2- 172.21.15.12:0/2531766299 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6000069960 0x7f6000100c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:49.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:49.211+0000 7f6007239700 1 --2- 172.21.15.12:0/2531766299 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f600006a300 0x7f60001011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:49.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:49.211+0000 7f6007239700 1 -- 172.21.15.12:0/2531766299 >> 172.21.15.12:0/2531766299 conn(0x7f6000077ae0 msgr2=0x7f60000ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:39:49.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:49.211+0000 7f6007239700 1 -- 172.21.15.12:0/2531766299 shutdown_connections 2024-02-18T11:39:49.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:49.211+0000 7f6007239700 1 -- 172.21.15.12:0/2531766299 wait complete. 2024-02-18T11:39:49.600 INFO:tasks.cephadm.ceph_manager.ceph:need seq 115964117053 got 115964117057 for osd.3 2024-02-18T11:39:49.600 DEBUG:teuthology.parallel:result is None 2024-02-18T11:39:49.717 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:39:49 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:39:49.717 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:39:49 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:39:49.717 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:39:49 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:39:49.717 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:39:49 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:39:49.718 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:39:49 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/2590784890' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-02-18T11:39:49.718 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:39:49 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/3640018799' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-18T11:39:49.718 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:39:49 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/2531766299' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-02-18T11:39:49.744 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:39:49 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:39:49.744 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:39:49 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:39:49.744 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:39:49 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:39:49.745 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:39:49 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:39:49.745 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:39:49 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/2590784890' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-02-18T11:39:49.745 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:39:49 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/3640018799' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-18T11:39:49.745 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:39:49 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/2531766299' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-02-18T11:39:49.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:39:49 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:39:49.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:39:49 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:39:49.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:39:49 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:39:49.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:39:49 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:39:49.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:39:49 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/2590784890' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-02-18T11:39:49.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:39:49 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/3640018799' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-18T11:39:49.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:39:49 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/2531766299' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-02-18T11:39:50.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:46.505+0000 7f24c3db3700 1 -- 172.21.15.12:0/2952524331 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24bc107ce0 msgr2=0x7f24bc1080c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:50.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:46.505+0000 7f24c3db3700 1 --2- 172.21.15.12:0/2952524331 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24bc107ce0 0x7f24bc1080c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f24b80099c0 tx=0x7f24b8009cd0 comp rx=0 tx=0).stop 2024-02-18T11:39:50.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:46.505+0000 7f24c3db3700 1 -- 172.21.15.12:0/2952524331 shutdown_connections 2024-02-18T11:39:50.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:46.505+0000 7f24c3db3700 1 --2- 172.21.15.12:0/2952524331 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f24bc1026c0 0x7f24bc10eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:50.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:46.505+0000 7f24c3db3700 1 --2- 172.21.15.12:0/2952524331 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24bc107ce0 0x7f24bc1080c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:50.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:46.505+0000 7f24c3db3700 1 --2- 172.21.15.12:0/2952524331 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24bc101d00 0x7f24bc102180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:50.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:46.505+0000 7f24c3db3700 1 -- 172.21.15.12:0/2952524331 >> 172.21.15.12:0/2952524331 conn(0x7f24bc078de0 msgr2=0x7f24bc0791f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:39:50.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:46.506+0000 7f24c3db3700 1 -- 172.21.15.12:0/2952524331 shutdown_connections 2024-02-18T11:39:50.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:46.506+0000 7f24c3db3700 1 -- 172.21.15.12:0/2952524331 wait complete. 2024-02-18T11:39:50.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:46.506+0000 7f24c3db3700 1 Processor -- start 2024-02-18T11:39:50.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:46.506+0000 7f24c3db3700 1 -- start start 2024-02-18T11:39:50.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:46.507+0000 7f24c3db3700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f24bc101d00 0x7f24bc1981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:50.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:46.507+0000 7f24c3db3700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24bc1026c0 0x7f24bc1986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:50.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:46.507+0000 7f24c3db3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24bc107ce0 0x7f24bc19ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:50.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:46.507+0000 7f24c3db3700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24bc19d080 con 0x7f24bc107ce0 2024-02-18T11:39:50.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:46.507+0000 7f24c3db3700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24bc19d1f0 con 0x7f24bc101d00 2024-02-18T11:39:50.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:46.507+0000 7f24c3db3700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24bc19d3b0 con 0x7f24bc1026c0 2024-02-18T11:39:50.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:46.507+0000 7f24c2350700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24bc107ce0 0x7f24bc19ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:50.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:46.507+0000 7f24c2350700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24bc107ce0 0x7f24bc19ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59368/0 (socket says 172.21.15.12:59368) 2024-02-18T11:39:50.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:46.507+0000 7f24c2350700 1 -- 172.21.15.12:0/315924845 learned_addr learned my addr 172.21.15.12:0/315924845 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:39:50.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:46.507+0000 7f24c1b4f700 1 --2- 172.21.15.12:0/315924845 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f24bc101d00 0x7f24bc1981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:50.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:46.507+0000 7f24c134e700 1 --2- 172.21.15.12:0/315924845 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24bc1026c0 0x7f24bc1986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:50.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:46.507+0000 7f24c2350700 1 -- 172.21.15.12:0/315924845 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24bc1026c0 msgr2=0x7f24bc1986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:50.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:46.507+0000 7f24c2350700 1 --2- 172.21.15.12:0/315924845 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24bc1026c0 0x7f24bc1986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:50.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:46.507+0000 7f24c2350700 1 -- 172.21.15.12:0/315924845 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f24bc101d00 msgr2=0x7f24bc1981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:50.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:46.507+0000 7f24c2350700 1 --2- 172.21.15.12:0/315924845 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f24bc101d00 0x7f24bc1981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:50.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:46.507+0000 7f24c2350700 1 -- 172.21.15.12:0/315924845 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24b80095f0 con 0x7f24bc107ce0 2024-02-18T11:39:50.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:46.507+0000 7f24c134e700 1 --2- 172.21.15.12:0/315924845 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24bc1026c0 0x7f24bc1986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:39:50.096 INFO:teuthology.orchestra.run.smithi012.stdout:77309411398 2024-02-18T11:39:50.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:46.507+0000 7f24c2350700 1 --2- 172.21.15.12:0/315924845 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24bc107ce0 0x7f24bc19ca70 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f24b400ada0 tx=0x7f24b400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:39:50.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:46.507+0000 7f24b2ffd700 1 -- 172.21.15.12:0/315924845 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b400f070 con 0x7f24bc107ce0 2024-02-18T11:39:50.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:46.507+0000 7f24c3db3700 1 -- 172.21.15.12:0/315924845 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24bc19d690 con 0x7f24bc107ce0 2024-02-18T11:39:50.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:46.507+0000 7f24b2ffd700 1 -- 172.21.15.12:0/315924845 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f24b4007b30 con 0x7f24bc107ce0 2024-02-18T11:39:50.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:46.507+0000 7f24b2ffd700 1 -- 172.21.15.12:0/315924845 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b40053c0 con 0x7f24bc107ce0 2024-02-18T11:39:50.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:46.508+0000 7f24c3db3700 1 -- 172.21.15.12:0/315924845 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24bc1a4ef0 con 0x7f24bc107ce0 2024-02-18T11:39:50.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:46.508+0000 7f24c3db3700 1 -- 172.21.15.12:0/315924845 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24bc10a1c0 con 0x7f24bc107ce0 2024-02-18T11:39:50.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:46.510+0000 7f24b2ffd700 1 -- 172.21.15.12:0/315924845 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 137296+0+0 (secure 0 0 0) 0x7f24b400c980 con 0x7f24bc107ce0 2024-02-18T11:39:50.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:46.510+0000 7f24b2ffd700 1 --2- 172.21.15.12:0/315924845 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f24a80a2870 0x7f24a80a4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:50.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:46.510+0000 7f24c1b4f700 1 --2- 172.21.15.12:0/315924845 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f24a80a2870 0x7f24a80a4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:50.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:46.510+0000 7f24b2ffd700 1 -- 172.21.15.12:0/315924845 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f24b40ce0b0 con 0x7f24bc107ce0 2024-02-18T11:39:50.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:46.511+0000 7f24c1b4f700 1 --2- 172.21.15.12:0/315924845 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f24a80a2870 0x7f24a80a4d30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f24b8009990 tx=0x7f24b8003760 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:39:50.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:46.514+0000 7f24b2ffd700 1 -- 172.21.15.12:0/315924845 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f24b4096f20 con 0x7f24bc107ce0 2024-02-18T11:39:50.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:46.723+0000 7f24c3db3700 1 -- 172.21.15.12:0/315924845 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f24bc199500 con 0x7f24bc107ce0 2024-02-18T11:39:50.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:46.723+0000 7f24b2ffd700 1 -- 172.21.15.12:0/315924845 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f24b409a5e0 con 0x7f24bc107ce0 2024-02-18T11:39:50.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:46.725+0000 7f24c3db3700 1 -- 172.21.15.12:0/315924845 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f24a80a2870 msgr2=0x7f24a80a4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:50.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:46.725+0000 7f24c3db3700 1 --2- 172.21.15.12:0/315924845 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f24a80a2870 0x7f24a80a4d30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f24b8009990 tx=0x7f24b8003760 comp rx=0 tx=0).stop 2024-02-18T11:39:50.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:46.725+0000 7f24c3db3700 1 -- 172.21.15.12:0/315924845 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24bc107ce0 msgr2=0x7f24bc19ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:50.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:46.725+0000 7f24c3db3700 1 --2- 172.21.15.12:0/315924845 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24bc107ce0 0x7f24bc19ca70 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f24b400ada0 tx=0x7f24b400c330 comp rx=0 tx=0).stop 2024-02-18T11:39:50.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:46.725+0000 7f24c3db3700 1 -- 172.21.15.12:0/315924845 shutdown_connections 2024-02-18T11:39:50.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:46.725+0000 7f24c3db3700 1 --2- 172.21.15.12:0/315924845 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f24a80a2870 0x7f24a80a4d30 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:50.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:46.725+0000 7f24c3db3700 1 --2- 172.21.15.12:0/315924845 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24bc107ce0 0x7f24bc19ca70 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:50.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:46.725+0000 7f24c3db3700 1 --2- 172.21.15.12:0/315924845 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f24bc101d00 0x7f24bc1981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:50.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:46.725+0000 7f24c3db3700 1 --2- 172.21.15.12:0/315924845 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24bc1026c0 0x7f24bc1986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:50.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:46.725+0000 7f24c3db3700 1 -- 172.21.15.12:0/315924845 >> 172.21.15.12:0/315924845 conn(0x7f24bc078de0 msgr2=0x7f24bc1008c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:39:50.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:46.725+0000 7f24c3db3700 1 -- 172.21.15.12:0/315924845 shutdown_connections 2024-02-18T11:39:50.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:46.725+0000 7f24c3db3700 1 -- 172.21.15.12:0/315924845 wait complete. 2024-02-18T11:39:50.648 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411393 got 77309411398 for osd.2 2024-02-18T11:39:50.648 DEBUG:teuthology.parallel:result is None 2024-02-18T11:39:50.744 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:39:50 smithi158 ceph-mon[36626]: pgmap v316: 97 pgs: 97 active+clean; 639 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-18T11:39:50.754 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:39:50 smithi012 ceph-mon[32460]: pgmap v316: 97 pgs: 97 active+clean; 639 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-18T11:39:50.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:39:50 smithi188 ceph-mon[36925]: pgmap v316: 97 pgs: 97 active+clean; 639 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-18T11:39:51.744 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:39:51 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-02-18T11:39:51.744 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:39:51 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-02-18T11:39:51.745 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:39:51 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]: dispatch 2024-02-18T11:39:51.745 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:39:51 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-02-18T11:39:51.745 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:39:51 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-02-18T11:39:51.745 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:39:51 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [3, 8]}]: dispatch 2024-02-18T11:39:51.745 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:39:51 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 6]}]: dispatch 2024-02-18T11:39:51.745 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:39:51 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 8]}]: dispatch 2024-02-18T11:39:51.758 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:39:51 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-02-18T11:39:51.758 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:39:51 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-02-18T11:39:51.758 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:39:51 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]: dispatch 2024-02-18T11:39:51.758 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:39:51 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-02-18T11:39:51.758 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:39:51 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-02-18T11:39:51.758 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:39:51 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [3, 8]}]: dispatch 2024-02-18T11:39:51.758 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:39:51 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 6]}]: dispatch 2024-02-18T11:39:51.758 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:39:51 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 8]}]: dispatch 2024-02-18T11:39:51.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:39:51 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-02-18T11:39:51.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:39:51 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-02-18T11:39:51.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:39:51 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]: dispatch 2024-02-18T11:39:51.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:39:51 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-02-18T11:39:51.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:39:51 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-02-18T11:39:51.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:39:51 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [3, 8]}]: dispatch 2024-02-18T11:39:51.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:39:51 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 6]}]: dispatch 2024-02-18T11:39:51.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:39:51 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 8]}]: dispatch 2024-02-18T11:39:52.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.255+0000 7f5fe6a37700 1 -- 172.21.15.12:0/107355764 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fe0107e10 msgr2=0x7f5fe01081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:52.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.255+0000 7f5fe6a37700 1 --2- 172.21.15.12:0/107355764 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fe0107e10 0x7f5fe01081f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5fd0009990 tx=0x7f5fd0009ca0 comp rx=0 tx=0).stop 2024-02-18T11:39:52.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.255+0000 7f5fe6a37700 1 -- 172.21.15.12:0/107355764 shutdown_connections 2024-02-18T11:39:52.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.255+0000 7f5fe6a37700 1 --2- 172.21.15.12:0/107355764 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fe01026f0 0x7f5fe010eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:52.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.255+0000 7f5fe6a37700 1 --2- 172.21.15.12:0/107355764 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fe0107e10 0x7f5fe01081f0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:52.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.255+0000 7f5fe6a37700 1 --2- 172.21.15.12:0/107355764 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5fe0101d30 0x7f5fe01021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:52.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.255+0000 7f5fe6a37700 1 -- 172.21.15.12:0/107355764 >> 172.21.15.12:0/107355764 conn(0x7f5fe0078d70 msgr2=0x7f5fe0079180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:39:52.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.255+0000 7f5fe6a37700 1 -- 172.21.15.12:0/107355764 shutdown_connections 2024-02-18T11:39:52.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.255+0000 7f5fe6a37700 1 -- 172.21.15.12:0/107355764 wait complete. 2024-02-18T11:39:52.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.256+0000 7f5fe6a37700 1 Processor -- start 2024-02-18T11:39:52.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.256+0000 7f5fe6a37700 1 -- start start 2024-02-18T11:39:52.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.256+0000 7f5fe6a37700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fe0101d30 0x7f5fe0198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:52.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.256+0000 7f5fe6a37700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5fe01026f0 0x7f5fe0198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:52.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.256+0000 7f5fe6a37700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fe0107e10 0x7f5fe019cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:52.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.256+0000 7f5fe6a37700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe019d0f0 con 0x7f5fe0101d30 2024-02-18T11:39:52.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.256+0000 7f5fe6a37700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe019d260 con 0x7f5fe01026f0 2024-02-18T11:39:52.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.256+0000 7f5fe6a37700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe019d420 con 0x7f5fe0107e10 2024-02-18T11:39:52.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.256+0000 7f5fdffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fe0101d30 0x7f5fe0198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:52.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.256+0000 7f5fdffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fe0101d30 0x7f5fe0198210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40982/0 (socket says 172.21.15.12:40982) 2024-02-18T11:39:52.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.256+0000 7f5fdffff700 1 -- 172.21.15.12:0/146543087 learned_addr learned my addr 172.21.15.12:0/146543087 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:39:52.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.256+0000 7f5fe4fd4700 1 --2- 172.21.15.12:0/146543087 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fe0107e10 0x7f5fe019cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:52.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.256+0000 7f5fdf7fe700 1 --2- 172.21.15.12:0/146543087 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5fe01026f0 0x7f5fe0198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:52.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.257+0000 7f5fdffff700 1 -- 172.21.15.12:0/146543087 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fe0107e10 msgr2=0x7f5fe019cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:52.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.257+0000 7f5fdffff700 1 --2- 172.21.15.12:0/146543087 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fe0107e10 0x7f5fe019cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:52.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.257+0000 7f5fdffff700 1 -- 172.21.15.12:0/146543087 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5fe01026f0 msgr2=0x7f5fe0198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:52.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.257+0000 7f5fdffff700 1 --2- 172.21.15.12:0/146543087 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5fe01026f0 0x7f5fe0198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:52.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.257+0000 7f5fdffff700 1 -- 172.21.15.12:0/146543087 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fd00095f0 con 0x7f5fe0101d30 2024-02-18T11:39:52.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.257+0000 7f5fdffff700 1 --2- 172.21.15.12:0/146543087 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fe0101d30 0x7f5fe0198210 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f5fd0004760 tx=0x7f5fd0004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:39:52.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.257+0000 7f5fdd7fa700 1 -- 172.21.15.12:0/146543087 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd001d070 con 0x7f5fe0101d30 2024-02-18T11:39:52.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.257+0000 7f5fe6a37700 1 -- 172.21.15.12:0/146543087 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fe019d6a0 con 0x7f5fe0101d30 2024-02-18T11:39:52.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.257+0000 7f5fdd7fa700 1 -- 172.21.15.12:0/146543087 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5fd0003f50 con 0x7f5fe0101d30 2024-02-18T11:39:52.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.257+0000 7f5fdd7fa700 1 -- 172.21.15.12:0/146543087 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd0022df0 con 0x7f5fe0101d30 2024-02-18T11:39:52.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.257+0000 7f5fe6a37700 1 -- 172.21.15.12:0/146543087 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fe01a4f60 con 0x7f5fe0101d30 2024-02-18T11:39:52.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.258+0000 7f5fe6a37700 1 -- 172.21.15.12:0/146543087 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fe0068970 con 0x7f5fe0101d30 2024-02-18T11:39:52.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.259+0000 7f5fdd7fa700 1 -- 172.21.15.12:0/146543087 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 137296+0+0 (secure 0 0 0) 0x7f5fd00040c0 con 0x7f5fe0101d30 2024-02-18T11:39:52.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.260+0000 7f5fdd7fa700 1 --2- 172.21.15.12:0/146543087 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f5fcc0a2870 0x7f5fcc0a4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:52.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.260+0000 7f5fdf7fe700 1 --2- 172.21.15.12:0/146543087 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f5fcc0a2870 0x7f5fcc0a4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:52.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.260+0000 7f5fdd7fa700 1 -- 172.21.15.12:0/146543087 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5fd00cff50 con 0x7f5fe0101d30 2024-02-18T11:39:52.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.260+0000 7f5fdf7fe700 1 --2- 172.21.15.12:0/146543087 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f5fcc0a2870 0x7f5fcc0a4d30 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5fc8006fd0 tx=0x7f5fc8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:39:52.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.263+0000 7f5fdd7fa700 1 -- 172.21.15.12:0/146543087 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5fd0098c20 con 0x7f5fe0101d30 2024-02-18T11:39:52.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.452+0000 7f1a11d1b700 1 -- 172.21.15.12:0/2921017789 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1a0c106bf0 msgr2=0x7f1a0c106fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:52.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.452+0000 7f1a11d1b700 1 --2- 172.21.15.12:0/2921017789 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1a0c106bf0 0x7f1a0c106fd0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f19fc009990 tx=0x7f19fc009ca0 comp rx=0 tx=0).stop 2024-02-18T11:39:52.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.452+0000 7f1a11d1b700 1 -- 172.21.15.12:0/2921017789 shutdown_connections 2024-02-18T11:39:52.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.452+0000 7f1a11d1b700 1 --2- 172.21.15.12:0/2921017789 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1a0c101550 0x7f1a0c10da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:52.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.452+0000 7f1a11d1b700 1 --2- 172.21.15.12:0/2921017789 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1a0c106bf0 0x7f1a0c106fd0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:52.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.452+0000 7f1a11d1b700 1 --2- 172.21.15.12:0/2921017789 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a0c100bb0 0x7f1a0c101010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:52.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.452+0000 7f1a11d1b700 1 -- 172.21.15.12:0/2921017789 >> 172.21.15.12:0/2921017789 conn(0x7f1a0c077ae0 msgr2=0x7f1a0c077ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:39:52.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.452+0000 7f1a11d1b700 1 -- 172.21.15.12:0/2921017789 shutdown_connections 2024-02-18T11:39:52.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.452+0000 7f1a11d1b700 1 -- 172.21.15.12:0/2921017789 wait complete. 2024-02-18T11:39:52.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.453+0000 7f1a11d1b700 1 Processor -- start 2024-02-18T11:39:52.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.453+0000 7f1a11d1b700 1 -- start start 2024-02-18T11:39:52.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.453+0000 7f1a11d1b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a0c100bb0 0x7f1a0c19a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:52.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.453+0000 7f1a11d1b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1a0c101550 0x7f1a0c19a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:52.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.453+0000 7f1a11d1b700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1a0c106bf0 0x7f1a0c19ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:52.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.453+0000 7f1a11d1b700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0c19f2a0 con 0x7f1a0c101550 2024-02-18T11:39:52.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.453+0000 7f1a11d1b700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0c19f410 con 0x7f1a0c106bf0 2024-02-18T11:39:52.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.453+0000 7f1a11d1b700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0c19f5d0 con 0x7f1a0c100bb0 2024-02-18T11:39:52.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.453+0000 7f1a0affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1a0c101550 0x7f1a0c19a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:52.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.454+0000 7f1a0affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1a0c101550 0x7f1a0c19a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40994/0 (socket says 172.21.15.12:40994) 2024-02-18T11:39:52.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.454+0000 7f1a0affd700 1 -- 172.21.15.12:0/2278773612 learned_addr learned my addr 172.21.15.12:0/2278773612 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:39:52.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.454+0000 7f1a0b7fe700 1 --2- 172.21.15.12:0/2278773612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a0c100bb0 0x7f1a0c19a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:52.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.454+0000 7f1a0bfff700 1 --2- 172.21.15.12:0/2278773612 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1a0c106bf0 0x7f1a0c19ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:52.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.454+0000 7f1a0affd700 1 -- 172.21.15.12:0/2278773612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a0c100bb0 msgr2=0x7f1a0c19a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:52.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.454+0000 7f1a0affd700 1 --2- 172.21.15.12:0/2278773612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a0c100bb0 0x7f1a0c19a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:52.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.454+0000 7f1a0affd700 1 -- 172.21.15.12:0/2278773612 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1a0c106bf0 msgr2=0x7f1a0c19ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:52.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.454+0000 7f1a0affd700 1 --2- 172.21.15.12:0/2278773612 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1a0c106bf0 0x7f1a0c19ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:52.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.454+0000 7f1a0affd700 1 -- 172.21.15.12:0/2278773612 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19fc0095f0 con 0x7f1a0c101550 2024-02-18T11:39:52.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.454+0000 7f1a0bfff700 1 --2- 172.21.15.12:0/2278773612 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1a0c106bf0 0x7f1a0c19ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:39:52.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.454+0000 7f1a0affd700 1 --2- 172.21.15.12:0/2278773612 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1a0c101550 0x7f1a0c19a950 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1a0000e9a0 tx=0x7f1a0000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:39:52.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.454+0000 7f1a08ff9700 1 -- 172.21.15.12:0/2278773612 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a00009790 con 0x7f1a0c101550 2024-02-18T11:39:52.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.454+0000 7f1a11d1b700 1 -- 172.21.15.12:0/2278773612 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a0c19f8b0 con 0x7f1a0c101550 2024-02-18T11:39:52.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.454+0000 7f1a11d1b700 1 -- 172.21.15.12:0/2278773612 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a0c1a7110 con 0x7f1a0c101550 2024-02-18T11:39:52.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.454+0000 7f1a08ff9700 1 -- 172.21.15.12:0/2278773612 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1a0000cbd0 con 0x7f1a0c101550 2024-02-18T11:39:52.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.454+0000 7f1a08ff9700 1 -- 172.21.15.12:0/2278773612 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a00018680 con 0x7f1a0c101550 2024-02-18T11:39:52.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.455+0000 7f1a11d1b700 1 -- 172.21.15.12:0/2278773612 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19f8005330 con 0x7f1a0c101550 2024-02-18T11:39:52.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.457+0000 7f1a08ff9700 1 -- 172.21.15.12:0/2278773612 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 137296+0+0 (secure 0 0 0) 0x7f1a0000cd40 con 0x7f1a0c101550 2024-02-18T11:39:52.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.458+0000 7f1a08ff9700 1 --2- 172.21.15.12:0/2278773612 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f19f40a2870 0x7f19f40a4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:52.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.458+0000 7f1a08ff9700 1 -- 172.21.15.12:0/2278773612 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1a00014070 con 0x7f1a0c101550 2024-02-18T11:39:52.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.458+0000 7f1a0b7fe700 1 --2- 172.21.15.12:0/2278773612 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f19f40a2870 0x7f19f40a4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:52.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.459+0000 7f1a0b7fe700 1 --2- 172.21.15.12:0/2278773612 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f19f40a2870 0x7f19f40a4d30 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f19fc006010 tx=0x7f19fc00b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:39:52.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.461+0000 7f1a08ff9700 1 -- 172.21.15.12:0/2278773612 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1a000978f0 con 0x7f1a0c101550 2024-02-18T11:39:52.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.469+0000 7f5fe6a37700 1 -- 172.21.15.12:0/146543087 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7f5fe019d830 con 0x7f5fe0101d30 2024-02-18T11:39:52.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.470+0000 7f5fdd7fa700 1 -- 172.21.15.12:0/146543087 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5fd009c2e0 con 0x7f5fe0101d30 2024-02-18T11:39:52.464 INFO:teuthology.orchestra.run.smithi012.stdout:261993005097 2024-02-18T11:39:52.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.472+0000 7f5fe6a37700 1 -- 172.21.15.12:0/146543087 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f5fcc0a2870 msgr2=0x7f5fcc0a4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:52.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.472+0000 7f5fe6a37700 1 --2- 172.21.15.12:0/146543087 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f5fcc0a2870 0x7f5fcc0a4d30 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5fc8006fd0 tx=0x7f5fc8008040 comp rx=0 tx=0).stop 2024-02-18T11:39:52.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.472+0000 7f5fe6a37700 1 -- 172.21.15.12:0/146543087 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fe0101d30 msgr2=0x7f5fe0198210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:52.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.472+0000 7f5fe6a37700 1 --2- 172.21.15.12:0/146543087 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fe0101d30 0x7f5fe0198210 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f5fd0004760 tx=0x7f5fd0004790 comp rx=0 tx=0).stop 2024-02-18T11:39:52.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.472+0000 7f5fe6a37700 1 -- 172.21.15.12:0/146543087 shutdown_connections 2024-02-18T11:39:52.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.472+0000 7f5fe6a37700 1 --2- 172.21.15.12:0/146543087 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fe0107e10 0x7f5fe019cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:52.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.472+0000 7f5fe6a37700 1 --2- 172.21.15.12:0/146543087 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f5fcc0a2870 0x7f5fcc0a4d30 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:52.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.472+0000 7f5fe6a37700 1 --2- 172.21.15.12:0/146543087 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fe0101d30 0x7f5fe0198210 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:52.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.472+0000 7f5fe6a37700 1 --2- 172.21.15.12:0/146543087 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5fe01026f0 0x7f5fe0198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:52.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.472+0000 7f5fe6a37700 1 -- 172.21.15.12:0/146543087 >> 172.21.15.12:0/146543087 conn(0x7f5fe0078d70 msgr2=0x7f5fe0100a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:39:52.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.472+0000 7f5fe6a37700 1 -- 172.21.15.12:0/146543087 shutdown_connections 2024-02-18T11:39:52.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.472+0000 7f5fe6a37700 1 -- 172.21.15.12:0/146543087 wait complete. 2024-02-18T11:39:52.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:39:52 smithi012 ceph-mon[32460]: pgmap v317: 97 pgs: 97 active+clean; 639 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-18T11:39:52.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:39:52 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-02-18T11:39:52.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:39:52 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-02-18T11:39:52.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:39:52 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2024-02-18T11:39:52.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:39:52 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-02-18T11:39:52.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:39:52 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-02-18T11:39:52.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:39:52 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [3, 8]}]': finished 2024-02-18T11:39:52.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:39:52 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 6]}]': finished 2024-02-18T11:39:52.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:39:52 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 8]}]': finished 2024-02-18T11:39:52.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:39:52 smithi012 ceph-mon[32460]: osdmap e88: 12 total, 12 up, 12 in 2024-02-18T11:39:52.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:39:52 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/146543087' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-02-18T11:39:52.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.681+0000 7f1a11d1b700 1 -- 172.21.15.12:0/2278773612 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f19f80058b0 con 0x7f1a0c101550 2024-02-18T11:39:52.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.682+0000 7f1a08ff9700 1 -- 172.21.15.12:0/2278773612 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1a0009afb0 con 0x7f1a0c101550 2024-02-18T11:39:52.676 INFO:teuthology.orchestra.run.smithi012.stdout:150323855421 2024-02-18T11:39:52.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.683+0000 7f1a11d1b700 1 -- 172.21.15.12:0/2278773612 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f19f40a2870 msgr2=0x7f19f40a4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:52.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.683+0000 7f1a11d1b700 1 --2- 172.21.15.12:0/2278773612 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f19f40a2870 0x7f19f40a4d30 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f19fc006010 tx=0x7f19fc00b260 comp rx=0 tx=0).stop 2024-02-18T11:39:52.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.683+0000 7f1a11d1b700 1 -- 172.21.15.12:0/2278773612 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1a0c101550 msgr2=0x7f1a0c19a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:52.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.683+0000 7f1a11d1b700 1 --2- 172.21.15.12:0/2278773612 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1a0c101550 0x7f1a0c19a950 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1a0000e9a0 tx=0x7f1a0000ecb0 comp rx=0 tx=0).stop 2024-02-18T11:39:52.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.684+0000 7f1a11d1b700 1 -- 172.21.15.12:0/2278773612 shutdown_connections 2024-02-18T11:39:52.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.684+0000 7f1a11d1b700 1 --2- 172.21.15.12:0/2278773612 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1a0c106bf0 0x7f1a0c19ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:52.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.684+0000 7f1a11d1b700 1 --2- 172.21.15.12:0/2278773612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a0c100bb0 0x7f1a0c19a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:52.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.684+0000 7f1a11d1b700 1 --2- 172.21.15.12:0/2278773612 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f19f40a2870 0x7f19f40a4d30 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:52.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.684+0000 7f1a11d1b700 1 --2- 172.21.15.12:0/2278773612 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1a0c101550 0x7f1a0c19a950 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:52.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.684+0000 7f1a11d1b700 1 -- 172.21.15.12:0/2278773612 >> 172.21.15.12:0/2278773612 conn(0x7f1a0c077ae0 msgr2=0x7f1a0c0ff8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:39:52.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.684+0000 7f1a11d1b700 1 -- 172.21.15.12:0/2278773612 shutdown_connections 2024-02-18T11:39:52.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:52.684+0000 7f1a11d1b700 1 -- 172.21.15.12:0/2278773612 wait complete. 2024-02-18T11:39:52.744 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:39:52 smithi158 ceph-mon[36626]: pgmap v317: 97 pgs: 97 active+clean; 639 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-18T11:39:52.744 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:39:52 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-02-18T11:39:52.744 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:39:52 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-02-18T11:39:52.744 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:39:52 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2024-02-18T11:39:52.745 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:39:52 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-02-18T11:39:52.745 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:39:52 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-02-18T11:39:52.745 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:39:52 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [3, 8]}]': finished 2024-02-18T11:39:52.745 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:39:52 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 6]}]': finished 2024-02-18T11:39:52.745 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:39:52 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 8]}]': finished 2024-02-18T11:39:52.745 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:39:52 smithi158 ceph-mon[36626]: osdmap e88: 12 total, 12 up, 12 in 2024-02-18T11:39:52.745 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:39:52 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/146543087' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-02-18T11:39:52.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:39:52 smithi188 ceph-mon[36925]: pgmap v317: 97 pgs: 97 active+clean; 639 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-18T11:39:52.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:39:52 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-02-18T11:39:52.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:39:52 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-02-18T11:39:52.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:39:52 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2024-02-18T11:39:52.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:39:52 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-02-18T11:39:52.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:39:52 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-02-18T11:39:52.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:39:52 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [3, 8]}]': finished 2024-02-18T11:39:52.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:39:52 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 6]}]': finished 2024-02-18T11:39:52.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:39:52 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 8]}]': finished 2024-02-18T11:39:52.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:39:52 smithi188 ceph-mon[36925]: osdmap e88: 12 total, 12 up, 12 in 2024-02-18T11:39:52.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:39:52 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/146543087' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-02-18T11:39:53.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.564+0000 7f5e9267e700 1 -- 172.21.15.12:0/2502338333 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e8c1017b0 msgr2=0x7f5e8c1057e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:53.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.564+0000 7f5e9267e700 1 --2- 172.21.15.12:0/2502338333 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e8c1017b0 0x7f5e8c1057e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f5e80009990 tx=0x7f5e80009ca0 comp rx=0 tx=0).stop 2024-02-18T11:39:53.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.564+0000 7f5e9267e700 1 -- 172.21.15.12:0/2502338333 shutdown_connections 2024-02-18T11:39:53.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.564+0000 7f5e9267e700 1 --2- 172.21.15.12:0/2502338333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e8c105ec0 0x7f5e8c10da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:53.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.564+0000 7f5e9267e700 1 --2- 172.21.15.12:0/2502338333 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5e8c100e00 0x7f5e8c1011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:53.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.564+0000 7f5e9267e700 1 --2- 172.21.15.12:0/2502338333 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e8c1017b0 0x7f5e8c1057e0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:53.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.564+0000 7f5e9267e700 1 -- 172.21.15.12:0/2502338333 >> 172.21.15.12:0/2502338333 conn(0x7f5e8c077b20 msgr2=0x7f5e8c077f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:39:53.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.564+0000 7f5e9267e700 1 -- 172.21.15.12:0/2502338333 shutdown_connections 2024-02-18T11:39:53.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.564+0000 7f5e9267e700 1 -- 172.21.15.12:0/2502338333 wait complete. 2024-02-18T11:39:53.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.565+0000 7f5e9267e700 1 Processor -- start 2024-02-18T11:39:53.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.565+0000 7f5e9267e700 1 -- start start 2024-02-18T11:39:53.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.565+0000 7f5e9267e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e8c100e00 0x7f5e8c1990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:53.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.565+0000 7f5e9267e700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5e8c1017b0 0x7f5e8c199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:53.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.565+0000 7f5e9267e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e8c105ec0 0x7f5e8c1a4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:53.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.565+0000 7f5e9267e700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8c199f10 con 0x7f5e8c100e00 2024-02-18T11:39:53.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.565+0000 7f5e9267e700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8c1a4690 con 0x7f5e8c1017b0 2024-02-18T11:39:53.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.565+0000 7f5e9267e700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8c1a4800 con 0x7f5e8c105ec0 2024-02-18T11:39:53.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.565+0000 7f5e8bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e8c100e00 0x7f5e8c1990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:53.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.565+0000 7f5e8bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e8c100e00 0x7f5e8c1990d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41006/0 (socket says 172.21.15.12:41006) 2024-02-18T11:39:53.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.565+0000 7f5e8bfff700 1 -- 172.21.15.12:0/3846989604 learned_addr learned my addr 172.21.15.12:0/3846989604 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:39:53.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.565+0000 7f5e8b7fe700 1 --2- 172.21.15.12:0/3846989604 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5e8c1017b0 0x7f5e8c199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:53.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.565+0000 7f5e90c1b700 1 --2- 172.21.15.12:0/3846989604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e8c105ec0 0x7f5e8c1a4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:53.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.566+0000 7f5e8bfff700 1 -- 172.21.15.12:0/3846989604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e8c105ec0 msgr2=0x7f5e8c1a4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:53.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.566+0000 7f5e8bfff700 1 --2- 172.21.15.12:0/3846989604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e8c105ec0 0x7f5e8c1a4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:53.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.566+0000 7f5e8bfff700 1 -- 172.21.15.12:0/3846989604 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5e8c1017b0 msgr2=0x7f5e8c199610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:53.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.566+0000 7f5e8bfff700 1 --2- 172.21.15.12:0/3846989604 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5e8c1017b0 0x7f5e8c199610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:53.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.566+0000 7f5e8bfff700 1 -- 172.21.15.12:0/3846989604 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e800095f0 con 0x7f5e8c100e00 2024-02-18T11:39:53.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.566+0000 7f5e8bfff700 1 --2- 172.21.15.12:0/3846989604 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e8c100e00 0x7f5e8c1990d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c00c920 tx=0x7f5e7c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:39:53.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.566+0000 7f5e897fa700 1 -- 172.21.15.12:0/3846989604 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7c004a30 con 0x7f5e8c100e00 2024-02-18T11:39:53.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.566+0000 7f5e9267e700 1 -- 172.21.15.12:0/3846989604 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e8c1a4ae0 con 0x7f5e8c100e00 2024-02-18T11:39:53.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.566+0000 7f5e897fa700 1 -- 172.21.15.12:0/3846989604 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5e7c005500 con 0x7f5e8c100e00 2024-02-18T11:39:53.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.566+0000 7f5e897fa700 1 -- 172.21.15.12:0/3846989604 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7c00ec20 con 0x7f5e8c100e00 2024-02-18T11:39:53.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.566+0000 7f5e9267e700 1 -- 172.21.15.12:0/3846989604 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e8c1a5000 con 0x7f5e8c100e00 2024-02-18T11:39:53.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.567+0000 7f5e9267e700 1 -- 172.21.15.12:0/3846989604 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e8c109170 con 0x7f5e8c100e00 2024-02-18T11:39:53.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.568+0000 7f5e897fa700 1 -- 172.21.15.12:0/3846989604 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 137296+0+0 (secure 0 0 0) 0x7f5e7c004c20 con 0x7f5e8c100e00 2024-02-18T11:39:53.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.569+0000 7f5e897fa700 1 --2- 172.21.15.12:0/3846989604 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f5e740a2870 0x7f5e740a4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:53.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.569+0000 7f5e8b7fe700 1 --2- 172.21.15.12:0/3846989604 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f5e740a2870 0x7f5e740a4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:53.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.569+0000 7f5e897fa700 1 -- 172.21.15.12:0/3846989604 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5e7c0ce560 con 0x7f5e8c100e00 2024-02-18T11:39:53.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.570+0000 7f5e8b7fe700 1 --2- 172.21.15.12:0/3846989604 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f5e740a2870 0x7f5e740a4d30 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5e8c06af00 tx=0x7f5e80005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:39:53.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.574+0000 7f5e897fa700 1 -- 172.21.15.12:0/3846989604 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5e7c0971a0 con 0x7f5e8c100e00 2024-02-18T11:39:53.724 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:39:53 smithi012 ceph-mon[32460]: osdmap e89: 12 total, 12 up, 12 in 2024-02-18T11:39:53.724 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:39:53 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/2278773612' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-18T11:39:53.744 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:39:53 smithi158 ceph-mon[36626]: osdmap e89: 12 total, 12 up, 12 in 2024-02-18T11:39:53.744 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:39:53 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/2278773612' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-18T11:39:53.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.778+0000 7f5e9267e700 1 -- 172.21.15.12:0/3846989604 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f5e8c06a9a0 con 0x7f5e8c100e00 2024-02-18T11:39:53.773 INFO:teuthology.orchestra.run.smithi012.stdout:34359738449 2024-02-18T11:39:53.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.779+0000 7f5e897fa700 1 -- 172.21.15.12:0/3846989604 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5e7c01d020 con 0x7f5e8c100e00 2024-02-18T11:39:53.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.781+0000 7f5e9267e700 1 -- 172.21.15.12:0/3846989604 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f5e740a2870 msgr2=0x7f5e740a4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:53.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.781+0000 7f5e9267e700 1 --2- 172.21.15.12:0/3846989604 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f5e740a2870 0x7f5e740a4d30 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5e8c06af00 tx=0x7f5e80005fd0 comp rx=0 tx=0).stop 2024-02-18T11:39:53.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.781+0000 7f5e9267e700 1 -- 172.21.15.12:0/3846989604 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e8c100e00 msgr2=0x7f5e8c1990d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:53.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.781+0000 7f5e9267e700 1 --2- 172.21.15.12:0/3846989604 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e8c100e00 0x7f5e8c1990d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c00c920 tx=0x7f5e7c00cc30 comp rx=0 tx=0).stop 2024-02-18T11:39:53.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.781+0000 7f5e9267e700 1 -- 172.21.15.12:0/3846989604 shutdown_connections 2024-02-18T11:39:53.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.781+0000 7f5e9267e700 1 --2- 172.21.15.12:0/3846989604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e8c105ec0 0x7f5e8c1a4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:53.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.781+0000 7f5e9267e700 1 --2- 172.21.15.12:0/3846989604 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f5e740a2870 0x7f5e740a4d30 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:53.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.781+0000 7f5e9267e700 1 --2- 172.21.15.12:0/3846989604 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e8c100e00 0x7f5e8c1990d0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:53.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.781+0000 7f5e9267e700 1 --2- 172.21.15.12:0/3846989604 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5e8c1017b0 0x7f5e8c199610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:53.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.781+0000 7f5e9267e700 1 -- 172.21.15.12:0/3846989604 >> 172.21.15.12:0/3846989604 conn(0x7f5e8c077b20 msgr2=0x7f5e8c0feef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:39:53.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.781+0000 7f5e9267e700 1 -- 172.21.15.12:0/3846989604 shutdown_connections 2024-02-18T11:39:53.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.781+0000 7f5e9267e700 1 -- 172.21.15.12:0/3846989604 wait complete. 2024-02-18T11:39:53.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:39:53 smithi188 ceph-mon[36925]: osdmap e89: 12 total, 12 up, 12 in 2024-02-18T11:39:53.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:39:53 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/2278773612' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-18T11:39:53.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.978+0000 7fce2ea8c700 1 -- 172.21.15.12:0/1626282129 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fce28100bb0 msgr2=0x7fce28101010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:53.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.978+0000 7fce2ea8c700 1 --2- 172.21.15.12:0/1626282129 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fce28100bb0 0x7fce28101010 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fce14009990 tx=0x7fce14009ca0 comp rx=0 tx=0).stop 2024-02-18T11:39:53.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.979+0000 7fce2ea8c700 1 -- 172.21.15.12:0/1626282129 shutdown_connections 2024-02-18T11:39:53.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.979+0000 7fce2ea8c700 1 --2- 172.21.15.12:0/1626282129 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fce28101550 0x7fce2810da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:53.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.979+0000 7fce2ea8c700 1 --2- 172.21.15.12:0/1626282129 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce28106bf0 0x7fce28106fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:53.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.979+0000 7fce2ea8c700 1 --2- 172.21.15.12:0/1626282129 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fce28100bb0 0x7fce28101010 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:53.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.979+0000 7fce2ea8c700 1 -- 172.21.15.12:0/1626282129 >> 172.21.15.12:0/1626282129 conn(0x7fce28077ae0 msgr2=0x7fce28077ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:39:53.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.979+0000 7fce2ea8c700 1 -- 172.21.15.12:0/1626282129 shutdown_connections 2024-02-18T11:39:53.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.979+0000 7fce2ea8c700 1 -- 172.21.15.12:0/1626282129 wait complete. 2024-02-18T11:39:53.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.979+0000 7fce2ea8c700 1 Processor -- start 2024-02-18T11:39:53.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.979+0000 7fce2ea8c700 1 -- start start 2024-02-18T11:39:53.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.979+0000 7fce2ea8c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fce28100bb0 0x7fce28199010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:53.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.979+0000 7fce2ea8c700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fce28101550 0x7fce28199550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:53.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.979+0000 7fce2ea8c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce28106bf0 0x7fce281a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:53.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.979+0000 7fce2ea8c700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce28199e10 con 0x7fce28100bb0 2024-02-18T11:39:53.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.979+0000 7fce2ea8c700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce281a45d0 con 0x7fce28101550 2024-02-18T11:39:53.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.979+0000 7fce2ea8c700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce281a4710 con 0x7fce28106bf0 2024-02-18T11:39:53.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.980+0000 7fce2c828700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fce28100bb0 0x7fce28199010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:53.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.980+0000 7fce2c828700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fce28100bb0 0x7fce28199010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41032/0 (socket says 172.21.15.12:41032) 2024-02-18T11:39:53.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.980+0000 7fce2c828700 1 -- 172.21.15.12:0/276171666 learned_addr learned my addr 172.21.15.12:0/276171666 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:39:53.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.980+0000 7fce27fff700 1 --2- 172.21.15.12:0/276171666 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fce28101550 0x7fce28199550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:53.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.980+0000 7fce2d029700 1 --2- 172.21.15.12:0/276171666 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce28106bf0 0x7fce281a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:53.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.980+0000 7fce2c828700 1 -- 172.21.15.12:0/276171666 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce28106bf0 msgr2=0x7fce281a4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:53.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.980+0000 7fce2c828700 1 --2- 172.21.15.12:0/276171666 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce28106bf0 0x7fce281a4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:53.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.980+0000 7fce2c828700 1 -- 172.21.15.12:0/276171666 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fce28101550 msgr2=0x7fce28199550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:53.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.980+0000 7fce2c828700 1 --2- 172.21.15.12:0/276171666 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fce28101550 0x7fce28199550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:53.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.980+0000 7fce2c828700 1 -- 172.21.15.12:0/276171666 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce140095f0 con 0x7fce28100bb0 2024-02-18T11:39:53.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.980+0000 7fce2c828700 1 --2- 172.21.15.12:0/276171666 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fce28100bb0 0x7fce28199010 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fce1c00d980 tx=0x7fce1c00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:39:53.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.980+0000 7fce25ffb700 1 -- 172.21.15.12:0/276171666 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1c00de60 con 0x7fce28100bb0 2024-02-18T11:39:53.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.980+0000 7fce2ea8c700 1 -- 172.21.15.12:0/276171666 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce281a49f0 con 0x7fce28100bb0 2024-02-18T11:39:53.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.980+0000 7fce25ffb700 1 -- 172.21.15.12:0/276171666 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fce1c009e20 con 0x7fce28100bb0 2024-02-18T11:39:53.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.980+0000 7fce25ffb700 1 -- 172.21.15.12:0/276171666 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1c0054b0 con 0x7fce28100bb0 2024-02-18T11:39:53.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.981+0000 7fce2ea8c700 1 -- 172.21.15.12:0/276171666 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce281a4ef0 con 0x7fce28100bb0 2024-02-18T11:39:53.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.982+0000 7fce2ea8c700 1 -- 172.21.15.12:0/276171666 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce28109110 con 0x7fce28100bb0 2024-02-18T11:39:53.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.983+0000 7fce25ffb700 1 -- 172.21.15.12:0/276171666 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 137296+0+0 (secure 0 0 0) 0x7fce1c005610 con 0x7fce28100bb0 2024-02-18T11:39:53.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.983+0000 7fce25ffb700 1 --2- 172.21.15.12:0/276171666 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fce180a28f0 0x7fce180a4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:53.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.983+0000 7fce27fff700 1 --2- 172.21.15.12:0/276171666 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fce180a28f0 0x7fce180a4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:53.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.984+0000 7fce25ffb700 1 -- 172.21.15.12:0/276171666 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fce1c0ce320 con 0x7fce28100bb0 2024-02-18T11:39:53.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.984+0000 7fce27fff700 1 --2- 172.21.15.12:0/276171666 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fce180a28f0 0x7fce180a4db0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fce14006010 tx=0x7fce1401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:39:53.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:53.988+0000 7fce25ffb700 1 -- 172.21.15.12:0/276171666 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fce1c096f60 con 0x7fce28100bb0 2024-02-18T11:39:54.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:54.191+0000 7fce2ea8c700 1 -- 172.21.15.12:0/276171666 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7fce28069f70 con 0x7fce28100bb0 2024-02-18T11:39:54.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:54.192+0000 7fce25ffb700 1 -- 172.21.15.12:0/276171666 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7fce1c09a620 con 0x7fce28100bb0 2024-02-18T11:39:54.186 INFO:teuthology.orchestra.run.smithi012.stdout:317827579935 2024-02-18T11:39:54.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:54.194+0000 7fce2ea8c700 1 -- 172.21.15.12:0/276171666 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fce180a28f0 msgr2=0x7fce180a4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:54.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:54.194+0000 7fce2ea8c700 1 --2- 172.21.15.12:0/276171666 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fce180a28f0 0x7fce180a4db0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fce14006010 tx=0x7fce1401a040 comp rx=0 tx=0).stop 2024-02-18T11:39:54.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:54.194+0000 7fce2ea8c700 1 -- 172.21.15.12:0/276171666 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fce28100bb0 msgr2=0x7fce28199010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:54.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:54.194+0000 7fce2ea8c700 1 --2- 172.21.15.12:0/276171666 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fce28100bb0 0x7fce28199010 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fce1c00d980 tx=0x7fce1c00dd40 comp rx=0 tx=0).stop 2024-02-18T11:39:54.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:54.194+0000 7fce2ea8c700 1 -- 172.21.15.12:0/276171666 shutdown_connections 2024-02-18T11:39:54.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:54.194+0000 7fce2ea8c700 1 --2- 172.21.15.12:0/276171666 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce28106bf0 0x7fce281a4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:54.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:54.194+0000 7fce2ea8c700 1 --2- 172.21.15.12:0/276171666 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fce180a28f0 0x7fce180a4db0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:54.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:54.194+0000 7fce2ea8c700 1 --2- 172.21.15.12:0/276171666 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fce28100bb0 0x7fce28199010 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:54.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:54.194+0000 7fce2ea8c700 1 --2- 172.21.15.12:0/276171666 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fce28101550 0x7fce28199550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:54.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:54.194+0000 7fce2ea8c700 1 -- 172.21.15.12:0/276171666 >> 172.21.15.12:0/276171666 conn(0x7fce28077ae0 msgr2=0x7fce280ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:39:54.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:54.194+0000 7fce2ea8c700 1 -- 172.21.15.12:0/276171666 shutdown_connections 2024-02-18T11:39:54.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:54.194+0000 7fce2ea8c700 1 -- 172.21.15.12:0/276171666 wait complete. 2024-02-18T11:39:54.505 INFO:tasks.cephadm.ceph_manager.ceph:need seq 352187318292 got 352187318297 for osd.11 2024-02-18T11:39:54.505 DEBUG:teuthology.parallel:result is None 2024-02-18T11:39:54.744 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:39:54 smithi158 ceph-mon[36626]: pgmap v320: 97 pgs: 5 peering, 92 active+clean; 639 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s; 2 B/s, 0 objects/s recovering 2024-02-18T11:39:54.744 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:39:54 smithi158 ceph-mon[36626]: Health check failed: Reduced data availability: 1 pg inactive, 5 pgs peering (PG_AVAILABILITY) 2024-02-18T11:39:54.744 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:39:54 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/3846989604' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-18T11:39:54.745 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:39:54 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/276171666' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-02-18T11:39:54.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:39:54 smithi188 ceph-mon[36925]: pgmap v320: 97 pgs: 5 peering, 92 active+clean; 639 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s; 2 B/s, 0 objects/s recovering 2024-02-18T11:39:54.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:39:54 smithi188 ceph-mon[36925]: Health check failed: Reduced data availability: 1 pg inactive, 5 pgs peering (PG_AVAILABILITY) 2024-02-18T11:39:54.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:39:54 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/3846989604' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-18T11:39:54.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:39:54 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/276171666' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-02-18T11:39:54.896 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:39:54 smithi012 ceph-mon[32460]: pgmap v320: 97 pgs: 5 peering, 92 active+clean; 639 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s; 2 B/s, 0 objects/s recovering 2024-02-18T11:39:54.896 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:39:54 smithi012 ceph-mon[32460]: Health check failed: Reduced data availability: 1 pg inactive, 5 pgs peering (PG_AVAILABILITY) 2024-02-18T11:39:54.896 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:39:54 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/3846989604' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-18T11:39:54.896 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:39:54 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/276171666' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-02-18T11:39:55.216 INFO:tasks.cephadm.ceph_manager.ceph:need seq 150323855416 got 150323855421 for osd.4 2024-02-18T11:39:55.216 DEBUG:teuthology.parallel:result is None 2024-02-18T11:39:55.849 INFO:tasks.cephadm.ceph_manager.ceph:need seq 261993005092 got 261993005097 for osd.8 2024-02-18T11:39:55.849 DEBUG:teuthology.parallel:result is None 2024-02-18T11:39:56.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:39:56 smithi012 ceph-mon[32460]: pgmap v321: 97 pgs: 5 peering, 92 active+clean; 639 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 4 op/s; 2 B/s, 0 objects/s recovering 2024-02-18T11:39:56.744 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:39:56 smithi158 ceph-mon[36626]: pgmap v321: 97 pgs: 5 peering, 92 active+clean; 639 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 4 op/s; 2 B/s, 0 objects/s recovering 2024-02-18T11:39:56.762 INFO:tasks.cephadm.ceph_manager.ceph:need seq 317827579930 got 317827579935 for osd.10 2024-02-18T11:39:56.762 DEBUG:teuthology.parallel:result is None 2024-02-18T11:39:56.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:39:56 smithi188 ceph-mon[36925]: pgmap v321: 97 pgs: 5 peering, 92 active+clean; 639 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 4 op/s; 2 B/s, 0 objects/s recovering 2024-02-18T11:39:56.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:56.973+0000 7fd5a254a700 1 -- 172.21.15.12:0/3811286532 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd59c106d20 msgr2=0x7fd59c107100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:56.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:56.973+0000 7fd5a254a700 1 --2- 172.21.15.12:0/3811286532 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd59c106d20 0x7fd59c107100 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd58c009990 tx=0x7fd58c009ca0 comp rx=0 tx=0).stop 2024-02-18T11:39:56.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:56.973+0000 7fd5a254a700 1 -- 172.21.15.12:0/3811286532 shutdown_connections 2024-02-18T11:39:56.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:56.973+0000 7fd5a254a700 1 --2- 172.21.15.12:0/3811286532 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd59c100c60 0x7fd59c10ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:56.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:56.973+0000 7fd5a254a700 1 --2- 172.21.15.12:0/3811286532 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd59c106d20 0x7fd59c107100 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:56.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:56.973+0000 7fd5a254a700 1 --2- 172.21.15.12:0/3811286532 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd59c1002a0 0x7fd59c100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:56.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:56.973+0000 7fd5a254a700 1 -- 172.21.15.12:0/3811286532 >> 172.21.15.12:0/3811286532 conn(0x7fd59c0771b0 msgr2=0x7fd59c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:39:56.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:56.973+0000 7fd5a254a700 1 -- 172.21.15.12:0/3811286532 shutdown_connections 2024-02-18T11:39:56.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:56.973+0000 7fd5a254a700 1 -- 172.21.15.12:0/3811286532 wait complete. 2024-02-18T11:39:56.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:56.973+0000 7fd5a254a700 1 Processor -- start 2024-02-18T11:39:56.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:56.974+0000 7fd5a254a700 1 -- start start 2024-02-18T11:39:56.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:56.974+0000 7fd5a254a700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd59c1002a0 0x7fd59c198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:56.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:56.974+0000 7fd5a254a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd59c100c60 0x7fd59c198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:56.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:56.974+0000 7fd5a254a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd59c106d20 0x7fd59c19cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:56.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:56.974+0000 7fd5a254a700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd59c19d1e0 con 0x7fd59c106d20 2024-02-18T11:39:56.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:56.974+0000 7fd5a254a700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd59c19d350 con 0x7fd59c1002a0 2024-02-18T11:39:56.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:56.974+0000 7fd5a254a700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd59c19d510 con 0x7fd59c100c60 2024-02-18T11:39:56.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:56.974+0000 7fd5a0ae7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd59c106d20 0x7fd59c19cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:56.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:56.974+0000 7fd5a0ae7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd59c106d20 0x7fd59c19cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41058/0 (socket says 172.21.15.12:41058) 2024-02-18T11:39:56.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:56.974+0000 7fd5a0ae7700 1 -- 172.21.15.12:0/1980381405 learned_addr learned my addr 172.21.15.12:0/1980381405 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:39:56.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:56.974+0000 7fd59b7fe700 1 --2- 172.21.15.12:0/1980381405 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd59c100c60 0x7fd59c198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:56.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:56.974+0000 7fd59bfff700 1 --2- 172.21.15.12:0/1980381405 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd59c1002a0 0x7fd59c198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:56.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:56.975+0000 7fd5a0ae7700 1 -- 172.21.15.12:0/1980381405 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd59c100c60 msgr2=0x7fd59c198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:56.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:56.975+0000 7fd5a0ae7700 1 --2- 172.21.15.12:0/1980381405 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd59c100c60 0x7fd59c198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:56.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:56.975+0000 7fd5a0ae7700 1 -- 172.21.15.12:0/1980381405 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd59c1002a0 msgr2=0x7fd59c198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:56.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:56.975+0000 7fd5a0ae7700 1 --2- 172.21.15.12:0/1980381405 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd59c1002a0 0x7fd59c198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:56.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:56.975+0000 7fd5a0ae7700 1 -- 172.21.15.12:0/1980381405 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd58c0095f0 con 0x7fd59c106d20 2024-02-18T11:39:56.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:56.975+0000 7fd59b7fe700 1 --2- 172.21.15.12:0/1980381405 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd59c100c60 0x7fd59c198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-18T11:39:56.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:56.975+0000 7fd5a0ae7700 1 --2- 172.21.15.12:0/1980381405 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd59c106d20 0x7fd59c19cbd0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fd5940094f0 tx=0x7fd59400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:39:56.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:56.975+0000 7fd5997fa700 1 -- 172.21.15.12:0/1980381405 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd594004db0 con 0x7fd59c106d20 2024-02-18T11:39:56.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:56.975+0000 7fd5997fa700 1 -- 172.21.15.12:0/1980381405 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd59400f040 con 0x7fd59c106d20 2024-02-18T11:39:56.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:56.975+0000 7fd5a254a700 1 -- 172.21.15.12:0/1980381405 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd59c19d7f0 con 0x7fd59c106d20 2024-02-18T11:39:56.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:56.975+0000 7fd5997fa700 1 -- 172.21.15.12:0/1980381405 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5940059e0 con 0x7fd59c106d20 2024-02-18T11:39:56.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:56.975+0000 7fd5a254a700 1 -- 172.21.15.12:0/1980381405 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd59c1a5050 con 0x7fd59c106d20 2024-02-18T11:39:56.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:56.976+0000 7fd5a254a700 1 -- 172.21.15.12:0/1980381405 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd59c10a2f0 con 0x7fd59c106d20 2024-02-18T11:39:56.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:56.977+0000 7fd5997fa700 1 -- 172.21.15.12:0/1980381405 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 137296+0+0 (secure 0 0 0) 0x7fd594005020 con 0x7fd59c106d20 2024-02-18T11:39:56.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:56.978+0000 7fd5997fa700 1 --2- 172.21.15.12:0/1980381405 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fd5880a2870 0x7fd5880a4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:39:56.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:56.978+0000 7fd59bfff700 1 --2- 172.21.15.12:0/1980381405 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fd5880a2870 0x7fd5880a4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:39:56.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:56.978+0000 7fd5997fa700 1 -- 172.21.15.12:0/1980381405 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd5940cdf90 con 0x7fd59c106d20 2024-02-18T11:39:56.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:56.978+0000 7fd59bfff700 1 --2- 172.21.15.12:0/1980381405 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fd5880a2870 0x7fd5880a4d30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd58c005b00 tx=0x7fd58c005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:39:56.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:56.982+0000 7fd5997fa700 1 -- 172.21.15.12:0/1980381405 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd59404fd20 con 0x7fd59c106d20 2024-02-18T11:39:57.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:57.187+0000 7fd5a254a700 1 -- 172.21.15.12:0/1980381405 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fd59c0689d0 con 0x7fd59c106d20 2024-02-18T11:39:57.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:57.187+0000 7fd5997fa700 1 -- 172.21.15.12:0/1980381405 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd594096bd0 con 0x7fd59c106d20 2024-02-18T11:39:57.182 INFO:teuthology.orchestra.run.smithi012.stdout:55834574923 2024-02-18T11:39:57.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:57.189+0000 7fd5a254a700 1 -- 172.21.15.12:0/1980381405 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fd5880a2870 msgr2=0x7fd5880a4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:57.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:57.189+0000 7fd5a254a700 1 --2- 172.21.15.12:0/1980381405 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fd5880a2870 0x7fd5880a4d30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd58c005b00 tx=0x7fd58c005a90 comp rx=0 tx=0).stop 2024-02-18T11:39:57.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:57.189+0000 7fd5a254a700 1 -- 172.21.15.12:0/1980381405 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd59c106d20 msgr2=0x7fd59c19cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:39:57.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:57.189+0000 7fd5a254a700 1 --2- 172.21.15.12:0/1980381405 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd59c106d20 0x7fd59c19cbd0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fd5940094f0 tx=0x7fd59400d3b0 comp rx=0 tx=0).stop 2024-02-18T11:39:57.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:57.189+0000 7fd5a254a700 1 -- 172.21.15.12:0/1980381405 shutdown_connections 2024-02-18T11:39:57.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:57.189+0000 7fd5a254a700 1 --2- 172.21.15.12:0/1980381405 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fd5880a2870 0x7fd5880a4d30 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:57.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:57.189+0000 7fd5a254a700 1 --2- 172.21.15.12:0/1980381405 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd59c106d20 0x7fd59c19cbd0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:57.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:57.189+0000 7fd5a254a700 1 --2- 172.21.15.12:0/1980381405 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd59c1002a0 0x7fd59c198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:57.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:57.189+0000 7fd5a254a700 1 --2- 172.21.15.12:0/1980381405 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd59c100c60 0x7fd59c198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:39:57.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:57.189+0000 7fd5a254a700 1 -- 172.21.15.12:0/1980381405 >> 172.21.15.12:0/1980381405 conn(0x7fd59c0771b0 msgr2=0x7fd59c104d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:39:57.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:57.189+0000 7fd5a254a700 1 -- 172.21.15.12:0/1980381405 shutdown_connections 2024-02-18T11:39:57.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:39:57.189+0000 7fd5a254a700 1 -- 172.21.15.12:0/1980381405 wait complete. 2024-02-18T11:39:57.401 INFO:tasks.cephadm.ceph_manager.ceph:need seq 227633266728 got 227633266734 for osd.7 2024-02-18T11:39:57.401 DEBUG:teuthology.parallel:result is None 2024-02-18T11:39:57.744 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:39:57 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/1980381405' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-18T11:39:57.745 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:39:57 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/1980381405' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-18T11:39:57.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:39:57 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/1980381405' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-18T11:39:58.050 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574915 got 55834574923 for osd.1 2024-02-18T11:39:58.051 DEBUG:teuthology.parallel:result is None 2024-02-18T11:39:58.466 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738444 got 34359738449 for osd.0 2024-02-18T11:39:58.466 DEBUG:teuthology.parallel:result is None 2024-02-18T11:39:58.466 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-02-18T11:39:58.466 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph pg dump --format=json 2024-02-18T11:39:58.709 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:39:58 smithi012 ceph-mon[32460]: pgmap v322: 97 pgs: 97 active+clean; 639 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s; 2 B/s, 0 objects/s recovering 2024-02-18T11:39:58.709 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:39:58 smithi012 ceph-mon[32460]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 5 pgs peering) 2024-02-18T11:39:58.709 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:39:58 smithi012 ceph-mon[32460]: Cluster is now healthy 2024-02-18T11:39:58.744 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:39:58 smithi158 ceph-mon[36626]: pgmap v322: 97 pgs: 97 active+clean; 639 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s; 2 B/s, 0 objects/s recovering 2024-02-18T11:39:58.744 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:39:58 smithi158 ceph-mon[36626]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 5 pgs peering) 2024-02-18T11:39:58.744 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:39:58 smithi158 ceph-mon[36626]: Cluster is now healthy 2024-02-18T11:39:58.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:39:58 smithi188 ceph-mon[36925]: pgmap v322: 97 pgs: 97 active+clean; 639 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s; 2 B/s, 0 objects/s recovering 2024-02-18T11:39:58.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:39:58 smithi188 ceph-mon[36925]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 5 pgs peering) 2024-02-18T11:39:58.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:39:58 smithi188 ceph-mon[36925]: Cluster is now healthy 2024-02-18T11:39:59.087 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T11:40:00.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:00.719+0000 7f09e5191700 1 -- 172.21.15.12:0/2045907100 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09e0106bf0 msgr2=0x7f09e0106fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:00.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:00.719+0000 7f09e5191700 1 --2- 172.21.15.12:0/2045907100 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09e0106bf0 0x7f09e0106fd0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f09d0009990 tx=0x7f09d0009ca0 comp rx=0 tx=0).stop 2024-02-18T11:40:00.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:00.719+0000 7f09e5191700 1 -- 172.21.15.12:0/2045907100 shutdown_connections 2024-02-18T11:40:00.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:00.719+0000 7f09e5191700 1 --2- 172.21.15.12:0/2045907100 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f09e0101550 0x7f09e010da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:00.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:00.719+0000 7f09e5191700 1 --2- 172.21.15.12:0/2045907100 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09e0106bf0 0x7f09e0106fd0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:00.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:00.719+0000 7f09e5191700 1 --2- 172.21.15.12:0/2045907100 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09e0100bb0 0x7f09e0101010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:00.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:00.719+0000 7f09e5191700 1 -- 172.21.15.12:0/2045907100 >> 172.21.15.12:0/2045907100 conn(0x7f09e0077ae0 msgr2=0x7f09e0077ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:00.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:00.719+0000 7f09e5191700 1 -- 172.21.15.12:0/2045907100 shutdown_connections 2024-02-18T11:40:00.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:00.719+0000 7f09e5191700 1 -- 172.21.15.12:0/2045907100 wait complete. 2024-02-18T11:40:00.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:00.720+0000 7f09e5191700 1 Processor -- start 2024-02-18T11:40:00.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:00.720+0000 7f09e5191700 1 -- start start 2024-02-18T11:40:00.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:00.720+0000 7f09e5191700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09e0100bb0 0x7f09e0196de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:00.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:00.720+0000 7f09e5191700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f09e0101550 0x7f09e0197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:00.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:00.720+0000 7f09e5191700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09e0106bf0 0x7f09e0197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:00.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:00.720+0000 7f09e5191700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09e01a6520 con 0x7f09e0100bb0 2024-02-18T11:40:00.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:00.720+0000 7f09e5191700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09e01a6690 con 0x7f09e0101550 2024-02-18T11:40:00.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:00.720+0000 7f09e5191700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09e01a6850 con 0x7f09e0106bf0 2024-02-18T11:40:00.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:00.720+0000 7f09ded9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09e0100bb0 0x7f09e0196de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:00.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:00.720+0000 7f09ded9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09e0100bb0 0x7f09e0196de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41082/0 (socket says 172.21.15.12:41082) 2024-02-18T11:40:00.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:00.720+0000 7f09ded9d700 1 -- 172.21.15.12:0/1928722835 learned_addr learned my addr 172.21.15.12:0/1928722835 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:40:00.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:00.720+0000 7f09d7fff700 1 --2- 172.21.15.12:0/1928722835 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f09e0101550 0x7f09e0197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:00.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:00.720+0000 7f09df59e700 1 --2- 172.21.15.12:0/1928722835 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09e0106bf0 0x7f09e0197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:00.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:00.720+0000 7f09ded9d700 1 -- 172.21.15.12:0/1928722835 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09e0106bf0 msgr2=0x7f09e0197860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:00.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:00.720+0000 7f09ded9d700 1 --2- 172.21.15.12:0/1928722835 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09e0106bf0 0x7f09e0197860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:00.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:00.720+0000 7f09ded9d700 1 -- 172.21.15.12:0/1928722835 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f09e0101550 msgr2=0x7f09e0197320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:00.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:00.720+0000 7f09ded9d700 1 --2- 172.21.15.12:0/1928722835 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f09e0101550 0x7f09e0197320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:00.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:00.720+0000 7f09ded9d700 1 -- 172.21.15.12:0/1928722835 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09d00095f0 con 0x7f09e0100bb0 2024-02-18T11:40:00.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:00.720+0000 7f09ded9d700 1 --2- 172.21.15.12:0/1928722835 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09e0100bb0 0x7f09e0196de0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f09d00098e0 tx=0x7f09d000f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:00.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:00.721+0000 7f09dcd99700 1 -- 172.21.15.12:0/1928722835 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d001d070 con 0x7f09e0100bb0 2024-02-18T11:40:00.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:00.721+0000 7f09dcd99700 1 -- 172.21.15.12:0/1928722835 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f09d000fc70 con 0x7f09e0100bb0 2024-02-18T11:40:00.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:00.721+0000 7f09e5191700 1 -- 172.21.15.12:0/1928722835 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09e01a6ad0 con 0x7f09e0100bb0 2024-02-18T11:40:00.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:00.721+0000 7f09dcd99700 1 -- 172.21.15.12:0/1928722835 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d00176e0 con 0x7f09e0100bb0 2024-02-18T11:40:00.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:00.721+0000 7f09e5191700 1 -- 172.21.15.12:0/1928722835 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09e01a6ee0 con 0x7f09e0100bb0 2024-02-18T11:40:00.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:00.722+0000 7f09e5191700 1 -- 172.21.15.12:0/1928722835 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09e0109110 con 0x7f09e0100bb0 2024-02-18T11:40:00.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:00.723+0000 7f09dcd99700 1 -- 172.21.15.12:0/1928722835 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 137296+0+0 (secure 0 0 0) 0x7f09d00229e0 con 0x7f09e0100bb0 2024-02-18T11:40:00.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:00.723+0000 7f09dcd99700 1 --2- 172.21.15.12:0/1928722835 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f09cc0a2870 0x7f09cc0a4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:00.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:00.723+0000 7f09d7fff700 1 --2- 172.21.15.12:0/1928722835 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f09cc0a2870 0x7f09cc0a4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:00.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:00.723+0000 7f09dcd99700 1 -- 172.21.15.12:0/1928722835 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f09d00cfc10 con 0x7f09e0100bb0 2024-02-18T11:40:00.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:00.724+0000 7f09d7fff700 1 --2- 172.21.15.12:0/1928722835 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f09cc0a2870 0x7f09cc0a4d30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f09c8005ea0 tx=0x7f09c8005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:00.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:00.727+0000 7f09dcd99700 1 -- 172.21.15.12:0/1928722835 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f09d00988e0 con 0x7f09e0100bb0 2024-02-18T11:40:00.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:40:00 smithi188 ceph-mon[36925]: pgmap v323: 97 pgs: 97 active+clean; 639 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s; 2 B/s, 0 objects/s recovering 2024-02-18T11:40:00.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:40:00 smithi188 ceph-mon[36925]: overall HEALTH_OK 2024-02-18T11:40:00.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:40:00 smithi012 ceph-mon[32460]: pgmap v323: 97 pgs: 97 active+clean; 639 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s; 2 B/s, 0 objects/s recovering 2024-02-18T11:40:00.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:40:00 smithi012 ceph-mon[32460]: overall HEALTH_OK 2024-02-18T11:40:00.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:00.966+0000 7f09e5191700 1 -- 172.21.15.12:0/1928722835 --> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f09e0110ca0 con 0x7f09cc0a2870 2024-02-18T11:40:00.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:00.974+0000 7f09dcd99700 1 -- 172.21.15.12:0/1928722835 <== mgr.14152 v2:172.21.15.12:6800/1153736685 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+258710 (secure 0 0 0) 0x7f09e0110ca0 con 0x7f09cc0a2870 2024-02-18T11:40:00.972 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:40:00.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:00.978+0000 7f09e5191700 1 -- 172.21.15.12:0/1928722835 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f09cc0a2870 msgr2=0x7f09cc0a4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:00.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:00.978+0000 7f09e5191700 1 --2- 172.21.15.12:0/1928722835 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f09cc0a2870 0x7f09cc0a4d30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f09c8005ea0 tx=0x7f09c8005e30 comp rx=0 tx=0).stop 2024-02-18T11:40:00.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:00.978+0000 7f09e5191700 1 -- 172.21.15.12:0/1928722835 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09e0100bb0 msgr2=0x7f09e0196de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:00.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:00.978+0000 7f09e5191700 1 --2- 172.21.15.12:0/1928722835 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09e0100bb0 0x7f09e0196de0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f09d00098e0 tx=0x7f09d000f740 comp rx=0 tx=0).stop 2024-02-18T11:40:00.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:00.978+0000 7f09e5191700 1 -- 172.21.15.12:0/1928722835 shutdown_connections 2024-02-18T11:40:00.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:00.978+0000 7f09e5191700 1 --2- 172.21.15.12:0/1928722835 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09e0106bf0 0x7f09e0197860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:00.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:00.978+0000 7f09e5191700 1 --2- 172.21.15.12:0/1928722835 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f09cc0a2870 0x7f09cc0a4d30 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:00.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:00.978+0000 7f09e5191700 1 --2- 172.21.15.12:0/1928722835 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09e0100bb0 0x7f09e0196de0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:00.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:00.978+0000 7f09e5191700 1 --2- 172.21.15.12:0/1928722835 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f09e0101550 0x7f09e0197320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:00.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:00.978+0000 7f09e5191700 1 -- 172.21.15.12:0/1928722835 >> 172.21.15.12:0/1928722835 conn(0x7f09e0077ae0 msgr2=0x7f09e00ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:00.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:00.978+0000 7f09e5191700 1 -- 172.21.15.12:0/1928722835 shutdown_connections 2024-02-18T11:40:00.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:00.978+0000 7f09e5191700 1 -- 172.21.15.12:0/1928722835 wait complete. 2024-02-18T11:40:00.975 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-02-18T11:40:00.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:40:00 smithi158 ceph-mon[36626]: pgmap v323: 97 pgs: 97 active+clean; 639 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s; 2 B/s, 0 objects/s recovering 2024-02-18T11:40:00.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:40:00 smithi158 ceph-mon[36626]: overall HEALTH_OK 2024-02-18T11:40:01.649 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":323,"stamp":"2024-02-18T11:39:59.033330+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654626,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":545,"num_read_kb":385,"num_write":340,"num_write_kb":3049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1180758,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":275,"ondisk_log_size":275,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":391704,"kb_used_data":13356,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124550120,"statfs":{"total":1151940427776,"available":1151539322880,"internally_reserved":0,"allocated":13676544,"data_stored":7511526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":387233082},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":44,"num_read_kb":19,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001936"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.496585+0000","last_change":"2024-02-18T11:38:47.847524+0000","last_active":"2024-02-18T11:39:52.496585+0000","last_peered":"2024-02-18T11:39:52.496585+0000","last_clean":"2024-02-18T11:39:52.496585+0000","last_became_active":"2024-02-18T11:38:47.846756+0000","last_became_peered":"2024-02-18T11:38:47.846756+0000","last_unstale":"2024-02-18T11:39:52.496585+0000","last_undegraded":"2024-02-18T11:39:52.496585+0000","last_fullsized":"2024-02-18T11:39:52.496585+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:23:02.715418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.492264+0000","last_change":"2024-02-18T11:38:47.847391+0000","last_active":"2024-02-18T11:39:52.492264+0000","last_peered":"2024-02-18T11:39:52.492264+0000","last_clean":"2024-02-18T11:39:52.492264+0000","last_became_active":"2024-02-18T11:38:47.846558+0000","last_became_peered":"2024-02-18T11:38:47.846558+0000","last_unstale":"2024-02-18T11:39:52.492264+0000","last_undegraded":"2024-02-18T11:39:52.492264+0000","last_fullsized":"2024-02-18T11:39:52.492264+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:37:34.940328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.493048+0000","last_change":"2024-02-18T11:38:47.850143+0000","last_active":"2024-02-18T11:39:52.493048+0000","last_peered":"2024-02-18T11:39:52.493048+0000","last_clean":"2024-02-18T11:39:52.493048+0000","last_became_active":"2024-02-18T11:38:47.845844+0000","last_became_peered":"2024-02-18T11:38:47.845844+0000","last_unstale":"2024-02-18T11:39:52.493048+0000","last_undegraded":"2024-02-18T11:39:52.493048+0000","last_fullsized":"2024-02-18T11:39:52.493048+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T12:45:15.638456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.498814+0000","last_change":"2024-02-18T11:38:47.841317+0000","last_active":"2024-02-18T11:39:52.498814+0000","last_peered":"2024-02-18T11:39:52.498814+0000","last_clean":"2024-02-18T11:39:52.498814+0000","last_became_active":"2024-02-18T11:38:47.840982+0000","last_became_peered":"2024-02-18T11:38:47.840982+0000","last_unstale":"2024-02-18T11:39:52.498814+0000","last_undegraded":"2024-02-18T11:39:52.498814+0000","last_fullsized":"2024-02-18T11:39:52.498814+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:53:40.115908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.500194+0000","last_change":"2024-02-18T11:38:47.851422+0000","last_active":"2024-02-18T11:39:52.500194+0000","last_peered":"2024-02-18T11:39:52.500194+0000","last_clean":"2024-02-18T11:39:52.500194+0000","last_became_active":"2024-02-18T11:38:47.850479+0000","last_became_peered":"2024-02-18T11:38:47.850479+0000","last_unstale":"2024-02-18T11:39:52.500194+0000","last_undegraded":"2024-02-18T11:39:52.500194+0000","last_fullsized":"2024-02-18T11:39:52.500194+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:09:21.200282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.629275+0000","last_change":"2024-02-18T11:38:47.837468+0000","last_active":"2024-02-18T11:39:52.629275+0000","last_peered":"2024-02-18T11:39:52.629275+0000","last_clean":"2024-02-18T11:39:52.629275+0000","last_became_active":"2024-02-18T11:38:47.837129+0000","last_became_peered":"2024-02-18T11:38:47.837129+0000","last_unstale":"2024-02-18T11:39:52.629275+0000","last_undegraded":"2024-02-18T11:39:52.629275+0000","last_fullsized":"2024-02-18T11:39:52.629275+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:12:01.954496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.488243+0000","last_change":"2024-02-18T11:38:47.849467+0000","last_active":"2024-02-18T11:39:52.488243+0000","last_peered":"2024-02-18T11:39:52.488243+0000","last_clean":"2024-02-18T11:39:52.488243+0000","last_became_active":"2024-02-18T11:38:47.848938+0000","last_became_peered":"2024-02-18T11:38:47.848938+0000","last_unstale":"2024-02-18T11:39:52.488243+0000","last_undegraded":"2024-02-18T11:39:52.488243+0000","last_fullsized":"2024-02-18T11:39:52.488243+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:39:16.680834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.507552+0000","last_change":"2024-02-18T11:38:47.830143+0000","last_active":"2024-02-18T11:39:52.507552+0000","last_peered":"2024-02-18T11:39:52.507552+0000","last_clean":"2024-02-18T11:39:52.507552+0000","last_became_active":"2024-02-18T11:38:47.829765+0000","last_became_peered":"2024-02-18T11:38:47.829765+0000","last_unstale":"2024-02-18T11:39:52.507552+0000","last_undegraded":"2024-02-18T11:39:52.507552+0000","last_fullsized":"2024-02-18T11:39:52.507552+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:29:11.825847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.500879+0000","last_change":"2024-02-18T11:39:52.500879+0000","last_active":"2024-02-18T11:39:52.500879+0000","last_peered":"2024-02-18T11:39:52.500879+0000","last_clean":"2024-02-18T11:39:52.500879+0000","last_became_active":"2024-02-18T11:39:52.500547+0000","last_became_peered":"2024-02-18T11:39:52.500547+0000","last_unstale":"2024-02-18T11:39:52.500879+0000","last_undegraded":"2024-02-18T11:39:52.500879+0000","last_fullsized":"2024-02-18T11:39:52.500879+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:22:40.792845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,11],"acting":[1,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.629615+0000","last_change":"2024-02-18T11:38:47.836478+0000","last_active":"2024-02-18T11:39:52.629615+0000","last_peered":"2024-02-18T11:39:52.629615+0000","last_clean":"2024-02-18T11:39:52.629615+0000","last_became_active":"2024-02-18T11:38:47.836161+0000","last_became_peered":"2024-02-18T11:38:47.836161+0000","last_unstale":"2024-02-18T11:39:52.629615+0000","last_undegraded":"2024-02-18T11:39:52.629615+0000","last_fullsized":"2024-02-18T11:39:52.629615+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:28:13.828518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.488765+0000","last_change":"2024-02-18T11:38:47.849435+0000","last_active":"2024-02-18T11:39:52.488765+0000","last_peered":"2024-02-18T11:39:52.488765+0000","last_clean":"2024-02-18T11:39:52.488765+0000","last_became_active":"2024-02-18T11:38:47.848947+0000","last_became_peered":"2024-02-18T11:38:47.848947+0000","last_unstale":"2024-02-18T11:39:52.488765+0000","last_undegraded":"2024-02-18T11:39:52.488765+0000","last_fullsized":"2024-02-18T11:39:52.488765+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T11:48:14.101732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.498682+0000","last_change":"2024-02-18T11:38:47.840366+0000","last_active":"2024-02-18T11:39:52.498682+0000","last_peered":"2024-02-18T11:39:52.498682+0000","last_clean":"2024-02-18T11:39:52.498682+0000","last_became_active":"2024-02-18T11:38:47.840044+0000","last_became_peered":"2024-02-18T11:38:47.840044+0000","last_unstale":"2024-02-18T11:39:52.498682+0000","last_undegraded":"2024-02-18T11:39:52.498682+0000","last_fullsized":"2024-02-18T11:39:52.498682+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:31:05.184418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.491677+0000","last_change":"2024-02-18T11:38:47.844433+0000","last_active":"2024-02-18T11:39:52.491677+0000","last_peered":"2024-02-18T11:39:52.491677+0000","last_clean":"2024-02-18T11:39:52.491677+0000","last_became_active":"2024-02-18T11:38:47.844048+0000","last_became_peered":"2024-02-18T11:38:47.844048+0000","last_unstale":"2024-02-18T11:39:52.491677+0000","last_undegraded":"2024-02-18T11:39:52.491677+0000","last_fullsized":"2024-02-18T11:39:52.491677+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:08:27.698272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.492551+0000","last_change":"2024-02-18T11:38:47.842819+0000","last_active":"2024-02-18T11:39:52.492551+0000","last_peered":"2024-02-18T11:39:52.492551+0000","last_clean":"2024-02-18T11:39:52.492551+0000","last_became_active":"2024-02-18T11:38:47.842489+0000","last_became_peered":"2024-02-18T11:38:47.842489+0000","last_unstale":"2024-02-18T11:39:52.492551+0000","last_undegraded":"2024-02-18T11:39:52.492551+0000","last_fullsized":"2024-02-18T11:39:52.492551+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:03:21.808910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.494547+0000","last_change":"2024-02-18T11:39:52.494547+0000","last_active":"2024-02-18T11:39:52.494547+0000","last_peered":"2024-02-18T11:39:52.494547+0000","last_clean":"2024-02-18T11:39:52.494547+0000","last_became_active":"2024-02-18T11:39:52.494161+0000","last_became_peered":"2024-02-18T11:39:52.494161+0000","last_unstale":"2024-02-18T11:39:52.494547+0000","last_undegraded":"2024-02-18T11:39:52.494547+0000","last_fullsized":"2024-02-18T11:39:52.494547+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T12:52:53.831143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.490320+0000","last_change":"2024-02-18T11:38:47.838729+0000","last_active":"2024-02-18T11:39:52.490320+0000","last_peered":"2024-02-18T11:39:52.490320+0000","last_clean":"2024-02-18T11:39:52.490320+0000","last_became_active":"2024-02-18T11:38:47.838267+0000","last_became_peered":"2024-02-18T11:38:47.838267+0000","last_unstale":"2024-02-18T11:39:52.490320+0000","last_undegraded":"2024-02-18T11:39:52.490320+0000","last_fullsized":"2024-02-18T11:39:52.490320+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:08:37.797780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.496470+0000","last_change":"2024-02-18T11:38:47.847817+0000","last_active":"2024-02-18T11:39:52.496470+0000","last_peered":"2024-02-18T11:39:52.496470+0000","last_clean":"2024-02-18T11:39:52.496470+0000","last_became_active":"2024-02-18T11:38:47.847200+0000","last_became_peered":"2024-02-18T11:38:47.847200+0000","last_unstale":"2024-02-18T11:39:52.496470+0000","last_undegraded":"2024-02-18T11:39:52.496470+0000","last_fullsized":"2024-02-18T11:39:52.496470+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:22:12.425623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.488774+0000","last_change":"2024-02-18T11:38:47.843954+0000","last_active":"2024-02-18T11:39:52.488774+0000","last_peered":"2024-02-18T11:39:52.488774+0000","last_clean":"2024-02-18T11:39:52.488774+0000","last_became_active":"2024-02-18T11:38:47.840084+0000","last_became_peered":"2024-02-18T11:38:47.840084+0000","last_unstale":"2024-02-18T11:39:52.488774+0000","last_undegraded":"2024-02-18T11:39:52.488774+0000","last_fullsized":"2024-02-18T11:39:52.488774+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:21:13.124696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.493319+0000","last_change":"2024-02-18T11:38:47.834262+0000","last_active":"2024-02-18T11:39:52.493319+0000","last_peered":"2024-02-18T11:39:52.493319+0000","last_clean":"2024-02-18T11:39:52.493319+0000","last_became_active":"2024-02-18T11:38:47.833948+0000","last_became_peered":"2024-02-18T11:38:47.833948+0000","last_unstale":"2024-02-18T11:39:52.493319+0000","last_undegraded":"2024-02-18T11:39:52.493319+0000","last_fullsized":"2024-02-18T11:39:52.493319+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T21:10:16.673734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.488965+0000","last_change":"2024-02-18T11:38:46.533805+0000","last_active":"2024-02-18T11:39:52.488965+0000","last_peered":"2024-02-18T11:39:52.488965+0000","last_clean":"2024-02-18T11:39:52.488965+0000","last_became_active":"2024-02-18T11:38:46.533503+0000","last_became_peered":"2024-02-18T11:38:46.533503+0000","last_unstale":"2024-02-18T11:39:52.488965+0000","last_undegraded":"2024-02-18T11:39:52.488965+0000","last_fullsized":"2024-02-18T11:39:52.488965+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:05:28.087740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.508228+0000","last_change":"2024-02-18T11:38:47.833802+0000","last_active":"2024-02-18T11:39:52.508228+0000","last_peered":"2024-02-18T11:39:52.508228+0000","last_clean":"2024-02-18T11:39:52.508228+0000","last_became_active":"2024-02-18T11:38:47.833441+0000","last_became_peered":"2024-02-18T11:38:47.833441+0000","last_unstale":"2024-02-18T11:39:52.508228+0000","last_undegraded":"2024-02-18T11:39:52.508228+0000","last_fullsized":"2024-02-18T11:39:52.508228+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:17:03.294191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.508336+0000","last_change":"2024-02-18T11:38:46.540718+0000","last_active":"2024-02-18T11:39:52.508336+0000","last_peered":"2024-02-18T11:39:52.508336+0000","last_clean":"2024-02-18T11:39:52.508336+0000","last_became_active":"2024-02-18T11:38:46.540413+0000","last_became_peered":"2024-02-18T11:38:46.540413+0000","last_unstale":"2024-02-18T11:39:52.508336+0000","last_undegraded":"2024-02-18T11:39:52.508336+0000","last_fullsized":"2024-02-18T11:39:52.508336+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:17:53.287376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.492598+0000","last_change":"2024-02-18T11:38:47.846400+0000","last_active":"2024-02-18T11:39:52.492598+0000","last_peered":"2024-02-18T11:39:52.492598+0000","last_clean":"2024-02-18T11:39:52.492598+0000","last_became_active":"2024-02-18T11:38:47.845934+0000","last_became_peered":"2024-02-18T11:38:47.845934+0000","last_unstale":"2024-02-18T11:39:52.492598+0000","last_undegraded":"2024-02-18T11:39:52.492598+0000","last_fullsized":"2024-02-18T11:39:52.492598+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:20:12.093697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"87'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.501031+0000","last_change":"2024-02-18T11:38:46.537340+0000","last_active":"2024-02-18T11:39:52.501031+0000","last_peered":"2024-02-18T11:39:52.501031+0000","last_clean":"2024-02-18T11:39:52.501031+0000","last_became_active":"2024-02-18T11:38:46.537019+0000","last_became_peered":"2024-02-18T11:38:46.537019+0000","last_unstale":"2024-02-18T11:39:52.501031+0000","last_undegraded":"2024-02-18T11:39:52.501031+0000","last_fullsized":"2024-02-18T11:39:52.501031+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T23:00:50.052654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.490785+0000","last_change":"2024-02-18T11:38:47.852656+0000","last_active":"2024-02-18T11:39:52.490785+0000","last_peered":"2024-02-18T11:39:52.490785+0000","last_clean":"2024-02-18T11:39:52.490785+0000","last_became_active":"2024-02-18T11:38:47.852279+0000","last_became_peered":"2024-02-18T11:38:47.852279+0000","last_unstale":"2024-02-18T11:39:52.490785+0000","last_undegraded":"2024-02-18T11:39:52.490785+0000","last_fullsized":"2024-02-18T11:39:52.490785+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:51:48.911504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.500473+0000","last_change":"2024-02-18T11:38:46.533091+0000","last_active":"2024-02-18T11:39:52.500473+0000","last_peered":"2024-02-18T11:39:52.500473+0000","last_clean":"2024-02-18T11:39:52.500473+0000","last_became_active":"2024-02-18T11:38:46.532504+0000","last_became_peered":"2024-02-18T11:38:46.532504+0000","last_unstale":"2024-02-18T11:39:52.500473+0000","last_undegraded":"2024-02-18T11:39:52.500473+0000","last_fullsized":"2024-02-18T11:39:52.500473+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:12:00.529365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.499324+0000","last_change":"2024-02-18T11:38:47.836578+0000","last_active":"2024-02-18T11:39:52.499324+0000","last_peered":"2024-02-18T11:39:52.499324+0000","last_clean":"2024-02-18T11:39:52.499324+0000","last_became_active":"2024-02-18T11:38:47.836217+0000","last_became_peered":"2024-02-18T11:38:47.836217+0000","last_unstale":"2024-02-18T11:39:52.499324+0000","last_undegraded":"2024-02-18T11:39:52.499324+0000","last_fullsized":"2024-02-18T11:39:52.499324+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T21:53:39.011822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"87'6","reported_seq":94,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:55.634439+0000","last_change":"2024-02-18T11:38:46.532196+0000","last_active":"2024-02-18T11:39:55.634439+0000","last_peered":"2024-02-18T11:39:55.634439+0000","last_clean":"2024-02-18T11:39:55.634439+0000","last_became_active":"2024-02-18T11:38:46.531868+0000","last_became_peered":"2024-02-18T11:38:46.531868+0000","last_unstale":"2024-02-18T11:39:55.634439+0000","last_undegraded":"2024-02-18T11:39:55.634439+0000","last_fullsized":"2024-02-18T11:39:55.634439+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:52:07.396279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":69,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.500073+0000","last_change":"2024-02-18T11:38:47.837329+0000","last_active":"2024-02-18T11:39:52.500073+0000","last_peered":"2024-02-18T11:39:52.500073+0000","last_clean":"2024-02-18T11:39:52.500073+0000","last_became_active":"2024-02-18T11:38:47.836989+0000","last_became_peered":"2024-02-18T11:38:47.836989+0000","last_unstale":"2024-02-18T11:39:52.500073+0000","last_undegraded":"2024-02-18T11:39:52.500073+0000","last_fullsized":"2024-02-18T11:39:52.500073+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:21:16.801916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"87'3","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.629204+0000","last_change":"2024-02-18T11:38:46.533244+0000","last_active":"2024-02-18T11:39:52.629204+0000","last_peered":"2024-02-18T11:39:52.629204+0000","last_clean":"2024-02-18T11:39:52.629204+0000","last_became_active":"2024-02-18T11:38:46.532326+0000","last_became_peered":"2024-02-18T11:38:46.532326+0000","last_unstale":"2024-02-18T11:39:52.629204+0000","last_undegraded":"2024-02-18T11:39:52.629204+0000","last_fullsized":"2024-02-18T11:39:52.629204+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:32:32.979056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.499349+0000","last_change":"2024-02-18T11:38:47.824560+0000","last_active":"2024-02-18T11:39:52.499349+0000","last_peered":"2024-02-18T11:39:52.499349+0000","last_clean":"2024-02-18T11:39:52.499349+0000","last_became_active":"2024-02-18T11:38:47.824095+0000","last_became_peered":"2024-02-18T11:38:47.824095+0000","last_unstale":"2024-02-18T11:39:52.499349+0000","last_undegraded":"2024-02-18T11:39:52.499349+0000","last_fullsized":"2024-02-18T11:39:52.499349+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:27:09.870585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"87'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.491527+0000","last_change":"2024-02-18T11:38:46.526024+0000","last_active":"2024-02-18T11:39:52.491527+0000","last_peered":"2024-02-18T11:39:52.491527+0000","last_clean":"2024-02-18T11:39:52.491527+0000","last_became_active":"2024-02-18T11:38:46.525735+0000","last_became_peered":"2024-02-18T11:38:46.525735+0000","last_unstale":"2024-02-18T11:39:52.491527+0000","last_undegraded":"2024-02-18T11:39:52.491527+0000","last_fullsized":"2024-02-18T11:39:52.491527+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:25:10.659210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.507643+0000","last_change":"2024-02-18T11:38:47.841743+0000","last_active":"2024-02-18T11:39:52.507643+0000","last_peered":"2024-02-18T11:39:52.507643+0000","last_clean":"2024-02-18T11:39:52.507643+0000","last_became_active":"2024-02-18T11:38:47.840848+0000","last_became_peered":"2024-02-18T11:38:47.840848+0000","last_unstale":"2024-02-18T11:39:52.507643+0000","last_undegraded":"2024-02-18T11:39:52.507643+0000","last_fullsized":"2024-02-18T11:39:52.507643+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:50:37.787782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"87'1","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.498571+0000","last_change":"2024-02-18T11:38:46.531355+0000","last_active":"2024-02-18T11:39:52.498571+0000","last_peered":"2024-02-18T11:39:52.498571+0000","last_clean":"2024-02-18T11:39:52.498571+0000","last_became_active":"2024-02-18T11:38:46.530601+0000","last_became_peered":"2024-02-18T11:38:46.530601+0000","last_unstale":"2024-02-18T11:39:52.498571+0000","last_undegraded":"2024-02-18T11:39:52.498571+0000","last_fullsized":"2024-02-18T11:39:52.498571+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:55:29.670274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.488009+0000","last_change":"2024-02-18T11:38:47.844506+0000","last_active":"2024-02-18T11:39:52.488009+0000","last_peered":"2024-02-18T11:39:52.488009+0000","last_clean":"2024-02-18T11:39:52.488009+0000","last_became_active":"2024-02-18T11:38:47.844158+0000","last_became_peered":"2024-02-18T11:38:47.844158+0000","last_unstale":"2024-02-18T11:39:52.488009+0000","last_undegraded":"2024-02-18T11:39:52.488009+0000","last_fullsized":"2024-02-18T11:39:52.488009+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T23:13:17.360077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"87'3","reported_seq":87,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:54.632589+0000","last_change":"2024-02-18T11:38:46.536313+0000","last_active":"2024-02-18T11:39:54.632589+0000","last_peered":"2024-02-18T11:39:54.632589+0000","last_clean":"2024-02-18T11:39:54.632589+0000","last_became_active":"2024-02-18T11:38:46.535910+0000","last_became_peered":"2024-02-18T11:38:46.535910+0000","last_unstale":"2024-02-18T11:39:54.632589+0000","last_undegraded":"2024-02-18T11:39:54.632589+0000","last_fullsized":"2024-02-18T11:39:54.632589+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:56:16.606415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":65,"num_read_kb":65,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.490880+0000","last_change":"2024-02-18T11:38:47.836131+0000","last_active":"2024-02-18T11:39:52.490880+0000","last_peered":"2024-02-18T11:39:52.490880+0000","last_clean":"2024-02-18T11:39:52.490880+0000","last_became_active":"2024-02-18T11:38:47.835702+0000","last_became_peered":"2024-02-18T11:38:47.835702+0000","last_unstale":"2024-02-18T11:39:52.490880+0000","last_undegraded":"2024-02-18T11:39:52.490880+0000","last_fullsized":"2024-02-18T11:39:52.490880+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:58:11.622972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"87'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.488516+0000","last_change":"2024-02-18T11:38:46.530031+0000","last_active":"2024-02-18T11:39:52.488516+0000","last_peered":"2024-02-18T11:39:52.488516+0000","last_clean":"2024-02-18T11:39:52.488516+0000","last_became_active":"2024-02-18T11:38:46.529674+0000","last_became_peered":"2024-02-18T11:38:46.529674+0000","last_unstale":"2024-02-18T11:39:52.488516+0000","last_undegraded":"2024-02-18T11:39:52.488516+0000","last_fullsized":"2024-02-18T11:39:52.488516+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T21:13:53.955970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.492495+0000","last_change":"2024-02-18T11:38:47.852340+0000","last_active":"2024-02-18T11:39:52.492495+0000","last_peered":"2024-02-18T11:39:52.492495+0000","last_clean":"2024-02-18T11:39:52.492495+0000","last_became_active":"2024-02-18T11:38:47.847978+0000","last_became_peered":"2024-02-18T11:38:47.847978+0000","last_unstale":"2024-02-18T11:39:52.492495+0000","last_undegraded":"2024-02-18T11:39:52.492495+0000","last_fullsized":"2024-02-18T11:39:52.492495+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:47:09.667619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.499446+0000","last_change":"2024-02-18T11:38:46.533511+0000","last_active":"2024-02-18T11:39:52.499446+0000","last_peered":"2024-02-18T11:39:52.499446+0000","last_clean":"2024-02-18T11:39:52.499446+0000","last_became_active":"2024-02-18T11:38:46.533162+0000","last_became_peered":"2024-02-18T11:38:46.533162+0000","last_unstale":"2024-02-18T11:39:52.499446+0000","last_undegraded":"2024-02-18T11:39:52.499446+0000","last_fullsized":"2024-02-18T11:39:52.499446+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:00:49.307489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.490667+0000","last_change":"2024-02-18T11:38:47.843835+0000","last_active":"2024-02-18T11:39:52.490667+0000","last_peered":"2024-02-18T11:39:52.490667+0000","last_clean":"2024-02-18T11:39:52.490667+0000","last_became_active":"2024-02-18T11:38:47.843517+0000","last_became_peered":"2024-02-18T11:38:47.843517+0000","last_unstale":"2024-02-18T11:39:52.490667+0000","last_undegraded":"2024-02-18T11:39:52.490667+0000","last_fullsized":"2024-02-18T11:39:52.490667+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:54:02.259854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'29","reported_seq":186,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:55.633646+0000","last_change":"2024-02-18T11:38:46.536900+0000","last_active":"2024-02-18T11:39:55.633646+0000","last_peered":"2024-02-18T11:39:55.633646+0000","last_clean":"2024-02-18T11:39:55.633646+0000","last_became_active":"2024-02-18T11:38:46.536563+0000","last_became_peered":"2024-02-18T11:38:46.536563+0000","last_unstale":"2024-02-18T11:39:55.633646+0000","last_undegraded":"2024-02-18T11:39:55.633646+0000","last_fullsized":"2024-02-18T11:39:55.633646+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T12:56:57.959482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60023,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":138,"num_read_kb":127,"num_write":30,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.500408+0000","last_change":"2024-02-18T11:38:47.842452+0000","last_active":"2024-02-18T11:39:52.500408+0000","last_peered":"2024-02-18T11:39:52.500408+0000","last_clean":"2024-02-18T11:39:52.500408+0000","last_became_active":"2024-02-18T11:38:47.842105+0000","last_became_peered":"2024-02-18T11:38:47.842105+0000","last_unstale":"2024-02-18T11:39:52.500408+0000","last_undegraded":"2024-02-18T11:39:52.500408+0000","last_fullsized":"2024-02-18T11:39:52.500408+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:10:05.131880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"87'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.493002+0000","last_change":"2024-02-18T11:38:46.534220+0000","last_active":"2024-02-18T11:39:52.493002+0000","last_peered":"2024-02-18T11:39:52.493002+0000","last_clean":"2024-02-18T11:39:52.493002+0000","last_became_active":"2024-02-18T11:38:46.532929+0000","last_became_peered":"2024-02-18T11:38:46.532929+0000","last_unstale":"2024-02-18T11:39:52.493002+0000","last_undegraded":"2024-02-18T11:39:52.493002+0000","last_fullsized":"2024-02-18T11:39:52.493002+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:41:42.674196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.491511+0000","last_change":"2024-02-18T11:38:47.848434+0000","last_active":"2024-02-18T11:39:52.491511+0000","last_peered":"2024-02-18T11:39:52.491511+0000","last_clean":"2024-02-18T11:39:52.491511+0000","last_became_active":"2024-02-18T11:38:47.848092+0000","last_became_peered":"2024-02-18T11:38:47.848092+0000","last_unstale":"2024-02-18T11:39:52.491511+0000","last_undegraded":"2024-02-18T11:39:52.491511+0000","last_fullsized":"2024-02-18T11:39:52.491511+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:23:35.292982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"87'3","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.499075+0000","last_change":"2024-02-18T11:38:46.531579+0000","last_active":"2024-02-18T11:39:52.499075+0000","last_peered":"2024-02-18T11:39:52.499075+0000","last_clean":"2024-02-18T11:39:52.499075+0000","last_became_active":"2024-02-18T11:38:46.531236+0000","last_became_peered":"2024-02-18T11:38:46.531236+0000","last_unstale":"2024-02-18T11:39:52.499075+0000","last_undegraded":"2024-02-18T11:39:52.499075+0000","last_fullsized":"2024-02-18T11:39:52.499075+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:54:48.886442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.493252+0000","last_change":"2024-02-18T11:39:52.493252+0000","last_active":"2024-02-18T11:39:52.493252+0000","last_peered":"2024-02-18T11:39:52.493252+0000","last_clean":"2024-02-18T11:39:52.493252+0000","last_became_active":"2024-02-18T11:39:52.492827+0000","last_became_peered":"2024-02-18T11:39:52.492827+0000","last_unstale":"2024-02-18T11:39:52.493252+0000","last_undegraded":"2024-02-18T11:39:52.493252+0000","last_fullsized":"2024-02-18T11:39:52.493252+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:24:18.392029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'3","reported_seq":90,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:55.632423+0000","last_change":"2024-02-18T11:38:46.528086+0000","last_active":"2024-02-18T11:39:55.632423+0000","last_peered":"2024-02-18T11:39:55.632423+0000","last_clean":"2024-02-18T11:39:55.632423+0000","last_became_active":"2024-02-18T11:38:46.527753+0000","last_became_peered":"2024-02-18T11:38:46.527753+0000","last_unstale":"2024-02-18T11:39:55.632423+0000","last_undegraded":"2024-02-18T11:39:55.632423+0000","last_fullsized":"2024-02-18T11:39:55.632423+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:48:09.524280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":69,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.488168+0000","last_change":"2024-02-18T11:38:47.849139+0000","last_active":"2024-02-18T11:39:52.488168+0000","last_peered":"2024-02-18T11:39:52.488168+0000","last_clean":"2024-02-18T11:39:52.488168+0000","last_became_active":"2024-02-18T11:38:47.848752+0000","last_became_peered":"2024-02-18T11:38:47.848752+0000","last_unstale":"2024-02-18T11:39:52.488168+0000","last_undegraded":"2024-02-18T11:39:52.488168+0000","last_fullsized":"2024-02-18T11:39:52.488168+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:00:56.046670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"87'15","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.499251+0000","last_change":"2024-02-18T11:38:46.540471+0000","last_active":"2024-02-18T11:39:52.499251+0000","last_peered":"2024-02-18T11:39:52.499251+0000","last_clean":"2024-02-18T11:39:52.499251+0000","last_became_active":"2024-02-18T11:38:46.540169+0000","last_became_peered":"2024-02-18T11:38:46.540169+0000","last_unstale":"2024-02-18T11:39:52.499251+0000","last_undegraded":"2024-02-18T11:39:52.499251+0000","last_fullsized":"2024-02-18T11:39:52.499251+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:29:46.001510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.498896+0000","last_change":"2024-02-18T11:38:47.838077+0000","last_active":"2024-02-18T11:39:52.498896+0000","last_peered":"2024-02-18T11:39:52.498896+0000","last_clean":"2024-02-18T11:39:52.498896+0000","last_became_active":"2024-02-18T11:38:47.837137+0000","last_became_peered":"2024-02-18T11:38:47.837137+0000","last_unstale":"2024-02-18T11:39:52.498896+0000","last_undegraded":"2024-02-18T11:39:52.498896+0000","last_fullsized":"2024-02-18T11:39:52.498896+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:06:15.443851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.506447+0000","last_change":"2024-02-18T11:39:52.505994+0000","last_active":"2024-02-18T11:39:52.506447+0000","last_peered":"2024-02-18T11:39:52.506447+0000","last_clean":"2024-02-18T11:39:52.506447+0000","last_became_active":"2024-02-18T11:39:52.501544+0000","last_became_peered":"2024-02-18T11:39:52.501544+0000","last_unstale":"2024-02-18T11:39:52.506447+0000","last_undegraded":"2024-02-18T11:39:52.506447+0000","last_fullsized":"2024-02-18T11:39:52.506447+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:51:13.850168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.496442+0000","last_change":"2024-02-18T11:38:47.835481+0000","last_active":"2024-02-18T11:39:52.496442+0000","last_peered":"2024-02-18T11:39:52.496442+0000","last_clean":"2024-02-18T11:39:52.496442+0000","last_became_active":"2024-02-18T11:38:47.835071+0000","last_became_peered":"2024-02-18T11:38:47.835071+0000","last_unstale":"2024-02-18T11:39:52.496442+0000","last_undegraded":"2024-02-18T11:39:52.496442+0000","last_fullsized":"2024-02-18T11:39:52.496442+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T12:10:42.098646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.497775+0000","last_change":"2024-02-18T11:39:52.497775+0000","last_active":"2024-02-18T11:39:52.497775+0000","last_peered":"2024-02-18T11:39:52.497775+0000","last_clean":"2024-02-18T11:39:52.497775+0000","last_became_active":"2024-02-18T11:39:52.497326+0000","last_became_peered":"2024-02-18T11:39:52.497326+0000","last_unstale":"2024-02-18T11:39:52.497775+0000","last_undegraded":"2024-02-18T11:39:52.497775+0000","last_fullsized":"2024-02-18T11:39:52.497775+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:01:39.098340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'1","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.629525+0000","last_change":"2024-02-18T11:38:47.842020+0000","last_active":"2024-02-18T11:39:52.629525+0000","last_peered":"2024-02-18T11:39:52.629525+0000","last_clean":"2024-02-18T11:39:52.629525+0000","last_became_active":"2024-02-18T11:38:47.841656+0000","last_became_peered":"2024-02-18T11:38:47.841656+0000","last_unstale":"2024-02-18T11:39:52.629525+0000","last_undegraded":"2024-02-18T11:39:52.629525+0000","last_fullsized":"2024-02-18T11:39:52.629525+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:12:50.508340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"87'1","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.488501+0000","last_change":"2024-02-18T11:38:46.531867+0000","last_active":"2024-02-18T11:39:52.488501+0000","last_peered":"2024-02-18T11:39:52.488501+0000","last_clean":"2024-02-18T11:39:52.488501+0000","last_became_active":"2024-02-18T11:38:46.531051+0000","last_became_peered":"2024-02-18T11:38:46.531051+0000","last_unstale":"2024-02-18T11:39:52.488501+0000","last_undegraded":"2024-02-18T11:39:52.488501+0000","last_fullsized":"2024-02-18T11:39:52.488501+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:15:26.808764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.629482+0000","last_change":"2024-02-18T11:38:47.835160+0000","last_active":"2024-02-18T11:39:52.629482+0000","last_peered":"2024-02-18T11:39:52.629482+0000","last_clean":"2024-02-18T11:39:52.629482+0000","last_became_active":"2024-02-18T11:38:47.834813+0000","last_became_peered":"2024-02-18T11:38:47.834813+0000","last_unstale":"2024-02-18T11:39:52.629482+0000","last_undegraded":"2024-02-18T11:39:52.629482+0000","last_fullsized":"2024-02-18T11:39:52.629482+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:30:59.674796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.498735+0000","last_change":"2024-02-18T11:38:47.838197+0000","last_active":"2024-02-18T11:39:52.498735+0000","last_peered":"2024-02-18T11:39:52.498735+0000","last_clean":"2024-02-18T11:39:52.498735+0000","last_became_active":"2024-02-18T11:38:47.837393+0000","last_became_peered":"2024-02-18T11:38:47.837393+0000","last_unstale":"2024-02-18T11:39:52.498735+0000","last_undegraded":"2024-02-18T11:39:52.498735+0000","last_fullsized":"2024-02-18T11:39:52.498735+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:13:19.095082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'4","reported_seq":89,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:55.631771+0000","last_change":"2024-02-18T11:38:46.538682+0000","last_active":"2024-02-18T11:39:55.631771+0000","last_peered":"2024-02-18T11:39:55.631771+0000","last_clean":"2024-02-18T11:39:55.631771+0000","last_became_active":"2024-02-18T11:38:46.538356+0000","last_became_peered":"2024-02-18T11:38:46.538356+0000","last_unstale":"2024-02-18T11:39:55.631771+0000","last_undegraded":"2024-02-18T11:39:55.631771+0000","last_fullsized":"2024-02-18T11:39:55.631771+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:41:02.449905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":66,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.500472+0000","last_change":"2024-02-18T11:39:52.500472+0000","last_active":"2024-02-18T11:39:52.500472+0000","last_peered":"2024-02-18T11:39:52.500472+0000","last_clean":"2024-02-18T11:39:52.500472+0000","last_became_active":"2024-02-18T11:39:52.500154+0000","last_became_peered":"2024-02-18T11:39:52.500154+0000","last_unstale":"2024-02-18T11:39:52.500472+0000","last_undegraded":"2024-02-18T11:39:52.500472+0000","last_fullsized":"2024-02-18T11:39:52.500472+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:47:23.278384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"87'3","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.498713+0000","last_change":"2024-02-18T11:38:46.531504+0000","last_active":"2024-02-18T11:39:52.498713+0000","last_peered":"2024-02-18T11:39:52.498713+0000","last_clean":"2024-02-18T11:39:52.498713+0000","last_became_active":"2024-02-18T11:38:46.531173+0000","last_became_peered":"2024-02-18T11:38:46.531173+0000","last_unstale":"2024-02-18T11:39:52.498713+0000","last_undegraded":"2024-02-18T11:39:52.498713+0000","last_fullsized":"2024-02-18T11:39:52.498713+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:32:40.632054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":345,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.488254+0000","last_change":"2024-02-18T11:37:34.534077+0000","last_active":"2024-02-18T11:39:52.488254+0000","last_peered":"2024-02-18T11:39:52.488254+0000","last_clean":"2024-02-18T11:39:52.488254+0000","last_became_active":"2024-02-18T11:37:34.533557+0000","last_became_peered":"2024-02-18T11:37:34.533557+0000","last_unstale":"2024-02-18T11:39:52.488254+0000","last_undegraded":"2024-02-18T11:39:52.488254+0000","last_fullsized":"2024-02-18T11:39:52.488254+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:34:15.604912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:34:15.604912+0000","last_clean_scrub_stamp":"2024-02-18T11:34:15.604912+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T23:01:35.216671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.491283+0000","last_change":"2024-02-18T11:38:47.838123+0000","last_active":"2024-02-18T11:39:52.491283+0000","last_peered":"2024-02-18T11:39:52.491283+0000","last_clean":"2024-02-18T11:39:52.491283+0000","last_became_active":"2024-02-18T11:38:47.837713+0000","last_became_peered":"2024-02-18T11:38:47.837713+0000","last_unstale":"2024-02-18T11:39:52.491283+0000","last_undegraded":"2024-02-18T11:39:52.491283+0000","last_fullsized":"2024-02-18T11:39:52.491283+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T21:47:44.609399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'3","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.488599+0000","last_change":"2024-02-18T11:38:46.529508+0000","last_active":"2024-02-18T11:39:52.488599+0000","last_peered":"2024-02-18T11:39:52.488599+0000","last_clean":"2024-02-18T11:39:52.488599+0000","last_became_active":"2024-02-18T11:38:46.529165+0000","last_became_peered":"2024-02-18T11:38:46.529165+0000","last_unstale":"2024-02-18T11:39:52.488599+0000","last_undegraded":"2024-02-18T11:39:52.488599+0000","last_fullsized":"2024-02-18T11:39:52.488599+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:10:12.774664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"87'1","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.500817+0000","last_change":"2024-02-18T11:38:46.542236+0000","last_active":"2024-02-18T11:39:52.500817+0000","last_peered":"2024-02-18T11:39:52.500817+0000","last_clean":"2024-02-18T11:39:52.500817+0000","last_became_active":"2024-02-18T11:38:46.541901+0000","last_became_peered":"2024-02-18T11:38:46.541901+0000","last_unstale":"2024-02-18T11:39:52.500817+0000","last_undegraded":"2024-02-18T11:39:52.500817+0000","last_fullsized":"2024-02-18T11:39:52.500817+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:40:28.699330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.492068+0000","last_change":"2024-02-18T11:38:47.841210+0000","last_active":"2024-02-18T11:39:52.492068+0000","last_peered":"2024-02-18T11:39:52.492068+0000","last_clean":"2024-02-18T11:39:52.492068+0000","last_became_active":"2024-02-18T11:38:47.838873+0000","last_became_peered":"2024-02-18T11:38:47.838873+0000","last_unstale":"2024-02-18T11:39:52.492068+0000","last_undegraded":"2024-02-18T11:39:52.492068+0000","last_fullsized":"2024-02-18T11:39:52.492068+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T21:22:11.304237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.488856+0000","last_change":"2024-02-18T11:38:46.533820+0000","last_active":"2024-02-18T11:39:52.488856+0000","last_peered":"2024-02-18T11:39:52.488856+0000","last_clean":"2024-02-18T11:39:52.488856+0000","last_became_active":"2024-02-18T11:38:46.533342+0000","last_became_peered":"2024-02-18T11:38:46.533342+0000","last_unstale":"2024-02-18T11:39:52.488856+0000","last_undegraded":"2024-02-18T11:39:52.488856+0000","last_fullsized":"2024-02-18T11:39:52.488856+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:27:34.814511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.508332+0000","last_change":"2024-02-18T11:38:47.837034+0000","last_active":"2024-02-18T11:39:52.508332+0000","last_peered":"2024-02-18T11:39:52.508332+0000","last_clean":"2024-02-18T11:39:52.508332+0000","last_became_active":"2024-02-18T11:38:47.836340+0000","last_became_peered":"2024-02-18T11:38:47.836340+0000","last_unstale":"2024-02-18T11:39:52.508332+0000","last_undegraded":"2024-02-18T11:39:52.508332+0000","last_fullsized":"2024-02-18T11:39:52.508332+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:44:10.812220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.498656+0000","last_change":"2024-02-18T11:38:46.529389+0000","last_active":"2024-02-18T11:39:52.498656+0000","last_peered":"2024-02-18T11:39:52.498656+0000","last_clean":"2024-02-18T11:39:52.498656+0000","last_became_active":"2024-02-18T11:38:46.529076+0000","last_became_peered":"2024-02-18T11:38:46.529076+0000","last_unstale":"2024-02-18T11:39:52.498656+0000","last_undegraded":"2024-02-18T11:39:52.498656+0000","last_fullsized":"2024-02-18T11:39:52.498656+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T21:01:05.553901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.492174+0000","last_change":"2024-02-18T11:38:47.847677+0000","last_active":"2024-02-18T11:39:52.492174+0000","last_peered":"2024-02-18T11:39:52.492174+0000","last_clean":"2024-02-18T11:39:52.492174+0000","last_became_active":"2024-02-18T11:38:47.847051+0000","last_became_peered":"2024-02-18T11:38:47.847051+0000","last_unstale":"2024-02-18T11:39:52.492174+0000","last_undegraded":"2024-02-18T11:39:52.492174+0000","last_fullsized":"2024-02-18T11:39:52.492174+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:25:53.398173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.490396+0000","last_change":"2024-02-18T11:38:46.525030+0000","last_active":"2024-02-18T11:39:52.490396+0000","last_peered":"2024-02-18T11:39:52.490396+0000","last_clean":"2024-02-18T11:39:52.490396+0000","last_became_active":"2024-02-18T11:38:46.524405+0000","last_became_peered":"2024-02-18T11:38:46.524405+0000","last_unstale":"2024-02-18T11:39:52.490396+0000","last_undegraded":"2024-02-18T11:39:52.490396+0000","last_fullsized":"2024-02-18T11:39:52.490396+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T12:33:41.751625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.499092+0000","last_change":"2024-02-18T11:38:47.836583+0000","last_active":"2024-02-18T11:39:52.499092+0000","last_peered":"2024-02-18T11:39:52.499092+0000","last_clean":"2024-02-18T11:39:52.499092+0000","last_became_active":"2024-02-18T11:38:47.836138+0000","last_became_peered":"2024-02-18T11:38:47.836138+0000","last_unstale":"2024-02-18T11:39:52.499092+0000","last_undegraded":"2024-02-18T11:39:52.499092+0000","last_fullsized":"2024-02-18T11:39:52.499092+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:34:04.002137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"87'3","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.507827+0000","last_change":"2024-02-18T11:38:46.538553+0000","last_active":"2024-02-18T11:39:52.507827+0000","last_peered":"2024-02-18T11:39:52.507827+0000","last_clean":"2024-02-18T11:39:52.507827+0000","last_became_active":"2024-02-18T11:38:46.538216+0000","last_became_peered":"2024-02-18T11:38:46.538216+0000","last_unstale":"2024-02-18T11:39:52.507827+0000","last_undegraded":"2024-02-18T11:39:52.507827+0000","last_fullsized":"2024-02-18T11:39:52.507827+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:31:32.713327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.496065+0000","last_change":"2024-02-18T11:38:47.840452+0000","last_active":"2024-02-18T11:39:52.496065+0000","last_peered":"2024-02-18T11:39:52.496065+0000","last_clean":"2024-02-18T11:39:52.496065+0000","last_became_active":"2024-02-18T11:38:47.840071+0000","last_became_peered":"2024-02-18T11:38:47.840071+0000","last_unstale":"2024-02-18T11:39:52.496065+0000","last_undegraded":"2024-02-18T11:39:52.496065+0000","last_fullsized":"2024-02-18T11:39:52.496065+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:07:22.746854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"87'1","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.492907+0000","last_change":"2024-02-18T11:38:46.535460+0000","last_active":"2024-02-18T11:39:52.492907+0000","last_peered":"2024-02-18T11:39:52.492907+0000","last_clean":"2024-02-18T11:39:52.492907+0000","last_became_active":"2024-02-18T11:38:46.534626+0000","last_became_peered":"2024-02-18T11:38:46.534626+0000","last_unstale":"2024-02-18T11:39:52.492907+0000","last_undegraded":"2024-02-18T11:39:52.492907+0000","last_fullsized":"2024-02-18T11:39:52.492907+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:56:18.434798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.508274+0000","last_change":"2024-02-18T11:38:47.838616+0000","last_active":"2024-02-18T11:39:52.508274+0000","last_peered":"2024-02-18T11:39:52.508274+0000","last_clean":"2024-02-18T11:39:52.508274+0000","last_became_active":"2024-02-18T11:38:47.838301+0000","last_became_peered":"2024-02-18T11:38:47.838301+0000","last_unstale":"2024-02-18T11:39:52.508274+0000","last_undegraded":"2024-02-18T11:39:52.508274+0000","last_fullsized":"2024-02-18T11:39:52.508274+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T11:43:37.731817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.500339+0000","last_change":"2024-02-18T11:38:46.536544+0000","last_active":"2024-02-18T11:39:52.500339+0000","last_peered":"2024-02-18T11:39:52.500339+0000","last_clean":"2024-02-18T11:39:52.500339+0000","last_became_active":"2024-02-18T11:38:46.536215+0000","last_became_peered":"2024-02-18T11:38:46.536215+0000","last_unstale":"2024-02-18T11:39:52.500339+0000","last_undegraded":"2024-02-18T11:39:52.500339+0000","last_fullsized":"2024-02-18T11:39:52.500339+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:12:44.496950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.488718+0000","last_change":"2024-02-18T11:38:47.847872+0000","last_active":"2024-02-18T11:39:52.488718+0000","last_peered":"2024-02-18T11:39:52.488718+0000","last_clean":"2024-02-18T11:39:52.488718+0000","last_became_active":"2024-02-18T11:38:47.847537+0000","last_became_peered":"2024-02-18T11:38:47.847537+0000","last_unstale":"2024-02-18T11:39:52.488718+0000","last_undegraded":"2024-02-18T11:39:52.488718+0000","last_fullsized":"2024-02-18T11:39:52.488718+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T12:30:46.754937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"87'3","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.493409+0000","last_change":"2024-02-18T11:38:46.534677+0000","last_active":"2024-02-18T11:39:52.493409+0000","last_peered":"2024-02-18T11:39:52.493409+0000","last_clean":"2024-02-18T11:39:52.493409+0000","last_became_active":"2024-02-18T11:38:46.527789+0000","last_became_peered":"2024-02-18T11:38:46.527789+0000","last_unstale":"2024-02-18T11:39:52.493409+0000","last_undegraded":"2024-02-18T11:39:52.493409+0000","last_fullsized":"2024-02-18T11:39:52.493409+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:13:15.365557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.488754+0000","last_change":"2024-02-18T11:38:47.849874+0000","last_active":"2024-02-18T11:39:52.488754+0000","last_peered":"2024-02-18T11:39:52.488754+0000","last_clean":"2024-02-18T11:39:52.488754+0000","last_became_active":"2024-02-18T11:38:47.849318+0000","last_became_peered":"2024-02-18T11:38:47.849318+0000","last_unstale":"2024-02-18T11:39:52.488754+0000","last_undegraded":"2024-02-18T11:39:52.488754+0000","last_fullsized":"2024-02-18T11:39:52.488754+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:22:31.974649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'1","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.499208+0000","last_change":"2024-02-18T11:38:46.531691+0000","last_active":"2024-02-18T11:39:52.499208+0000","last_peered":"2024-02-18T11:39:52.499208+0000","last_clean":"2024-02-18T11:39:52.499208+0000","last_became_active":"2024-02-18T11:38:46.530987+0000","last_became_peered":"2024-02-18T11:38:46.530987+0000","last_unstale":"2024-02-18T11:39:52.499208+0000","last_undegraded":"2024-02-18T11:39:52.499208+0000","last_fullsized":"2024-02-18T11:39:52.499208+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:15:46.228599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'1","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.488492+0000","last_change":"2024-02-18T11:38:47.850273+0000","last_active":"2024-02-18T11:39:52.488492+0000","last_peered":"2024-02-18T11:39:52.488492+0000","last_clean":"2024-02-18T11:39:52.488492+0000","last_became_active":"2024-02-18T11:38:47.849110+0000","last_became_peered":"2024-02-18T11:38:47.849110+0000","last_unstale":"2024-02-18T11:39:52.488492+0000","last_undegraded":"2024-02-18T11:39:52.488492+0000","last_fullsized":"2024-02-18T11:39:52.488492+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T23:18:55.096768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.491794+0000","last_change":"2024-02-18T11:38:46.537641+0000","last_active":"2024-02-18T11:39:52.491794+0000","last_peered":"2024-02-18T11:39:52.491794+0000","last_clean":"2024-02-18T11:39:52.491794+0000","last_became_active":"2024-02-18T11:38:46.537311+0000","last_became_peered":"2024-02-18T11:38:46.537311+0000","last_unstale":"2024-02-18T11:39:52.491794+0000","last_undegraded":"2024-02-18T11:39:52.491794+0000","last_fullsized":"2024-02-18T11:39:52.491794+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:13:25.720250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.488697+0000","last_change":"2024-02-18T11:38:47.833177+0000","last_active":"2024-02-18T11:39:52.488697+0000","last_peered":"2024-02-18T11:39:52.488697+0000","last_clean":"2024-02-18T11:39:52.488697+0000","last_became_active":"2024-02-18T11:38:47.832839+0000","last_became_peered":"2024-02-18T11:38:47.832839+0000","last_unstale":"2024-02-18T11:39:52.488697+0000","last_undegraded":"2024-02-18T11:39:52.488697+0000","last_fullsized":"2024-02-18T11:39:52.488697+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:12:28.448505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.496193+0000","last_change":"2024-02-18T11:38:47.841062+0000","last_active":"2024-02-18T11:39:52.496193+0000","last_peered":"2024-02-18T11:39:52.496193+0000","last_clean":"2024-02-18T11:39:52.496193+0000","last_became_active":"2024-02-18T11:38:47.840730+0000","last_became_peered":"2024-02-18T11:38:47.840730+0000","last_unstale":"2024-02-18T11:39:52.496193+0000","last_undegraded":"2024-02-18T11:39:52.496193+0000","last_fullsized":"2024-02-18T11:39:52.496193+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:18:13.417676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.493337+0000","last_change":"2024-02-18T11:38:47.839488+0000","last_active":"2024-02-18T11:39:52.493337+0000","last_peered":"2024-02-18T11:39:52.493337+0000","last_clean":"2024-02-18T11:39:52.493337+0000","last_became_active":"2024-02-18T11:38:47.839134+0000","last_became_peered":"2024-02-18T11:38:47.839134+0000","last_unstale":"2024-02-18T11:39:52.493337+0000","last_undegraded":"2024-02-18T11:39:52.493337+0000","last_fullsized":"2024-02-18T11:39:52.493337+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:51:11.149307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.492959+0000","last_change":"2024-02-18T11:38:47.845397+0000","last_active":"2024-02-18T11:39:52.492959+0000","last_peered":"2024-02-18T11:39:52.492959+0000","last_clean":"2024-02-18T11:39:52.492959+0000","last_became_active":"2024-02-18T11:38:47.845027+0000","last_became_peered":"2024-02-18T11:38:47.845027+0000","last_unstale":"2024-02-18T11:39:52.492959+0000","last_undegraded":"2024-02-18T11:39:52.492959+0000","last_fullsized":"2024-02-18T11:39:52.492959+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:26:44.260093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.492867+0000","last_change":"2024-02-18T11:38:47.839238+0000","last_active":"2024-02-18T11:39:52.492867+0000","last_peered":"2024-02-18T11:39:52.492867+0000","last_clean":"2024-02-18T11:39:52.492867+0000","last_became_active":"2024-02-18T11:38:47.838928+0000","last_became_peered":"2024-02-18T11:38:47.838928+0000","last_unstale":"2024-02-18T11:39:52.492867+0000","last_undegraded":"2024-02-18T11:39:52.492867+0000","last_fullsized":"2024-02-18T11:39:52.492867+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:32:10.867251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.495865+0000","last_change":"2024-02-18T11:39:52.495865+0000","last_active":"2024-02-18T11:39:52.495865+0000","last_peered":"2024-02-18T11:39:52.495865+0000","last_clean":"2024-02-18T11:39:52.495865+0000","last_became_active":"2024-02-18T11:39:52.495484+0000","last_became_peered":"2024-02-18T11:39:52.495484+0000","last_unstale":"2024-02-18T11:39:52.495865+0000","last_undegraded":"2024-02-18T11:39:52.495865+0000","last_fullsized":"2024-02-18T11:39:52.495865+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:13:29.153602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.496463+0000","last_change":"2024-02-18T11:38:47.839821+0000","last_active":"2024-02-18T11:39:52.496463+0000","last_peered":"2024-02-18T11:39:52.496463+0000","last_clean":"2024-02-18T11:39:52.496463+0000","last_became_active":"2024-02-18T11:38:47.839431+0000","last_became_peered":"2024-02-18T11:38:47.839431+0000","last_unstale":"2024-02-18T11:39:52.496463+0000","last_undegraded":"2024-02-18T11:39:52.496463+0000","last_fullsized":"2024-02-18T11:39:52.496463+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:43:07.002223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.493560+0000","last_change":"2024-02-18T11:38:47.843937+0000","last_active":"2024-02-18T11:39:52.493560+0000","last_peered":"2024-02-18T11:39:52.493560+0000","last_clean":"2024-02-18T11:39:52.493560+0000","last_became_active":"2024-02-18T11:38:47.843603+0000","last_became_peered":"2024-02-18T11:38:47.843603+0000","last_unstale":"2024-02-18T11:39:52.493560+0000","last_undegraded":"2024-02-18T11:39:52.493560+0000","last_fullsized":"2024-02-18T11:39:52.493560+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:45:42.228604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.508376+0000","last_change":"2024-02-18T11:38:47.836702+0000","last_active":"2024-02-18T11:39:52.508376+0000","last_peered":"2024-02-18T11:39:52.508376+0000","last_clean":"2024-02-18T11:39:52.508376+0000","last_became_active":"2024-02-18T11:38:47.834843+0000","last_became_peered":"2024-02-18T11:38:47.834843+0000","last_unstale":"2024-02-18T11:39:52.508376+0000","last_undegraded":"2024-02-18T11:39:52.508376+0000","last_fullsized":"2024-02-18T11:39:52.508376+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T12:32:31.342561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.498701+0000","last_change":"2024-02-18T11:38:47.837429+0000","last_active":"2024-02-18T11:39:52.498701+0000","last_peered":"2024-02-18T11:39:52.498701+0000","last_clean":"2024-02-18T11:39:52.498701+0000","last_became_active":"2024-02-18T11:38:47.832983+0000","last_became_peered":"2024-02-18T11:38:47.832983+0000","last_unstale":"2024-02-18T11:39:52.498701+0000","last_undegraded":"2024-02-18T11:39:52.498701+0000","last_fullsized":"2024-02-18T11:39:52.498701+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:37:07.582626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.507679+0000","last_change":"2024-02-18T11:38:47.837041+0000","last_active":"2024-02-18T11:39:52.507679+0000","last_peered":"2024-02-18T11:39:52.507679+0000","last_clean":"2024-02-18T11:39:52.507679+0000","last_became_active":"2024-02-18T11:38:47.836730+0000","last_became_peered":"2024-02-18T11:38:47.836730+0000","last_unstale":"2024-02-18T11:39:52.507679+0000","last_undegraded":"2024-02-18T11:39:52.507679+0000","last_fullsized":"2024-02-18T11:39:52.507679+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:44:21.998090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.501957+0000","last_change":"2024-02-18T11:39:52.501957+0000","last_active":"2024-02-18T11:39:52.501957+0000","last_peered":"2024-02-18T11:39:52.501957+0000","last_clean":"2024-02-18T11:39:52.501957+0000","last_became_active":"2024-02-18T11:39:52.501561+0000","last_became_peered":"2024-02-18T11:39:52.501561+0000","last_unstale":"2024-02-18T11:39:52.501957+0000","last_undegraded":"2024-02-18T11:39:52.501957+0000","last_fullsized":"2024-02-18T11:39:52.501957+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:49:37.243875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,8],"acting":[1,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.491728+0000","last_change":"2024-02-18T11:38:47.841231+0000","last_active":"2024-02-18T11:39:52.491728+0000","last_peered":"2024-02-18T11:39:52.491728+0000","last_clean":"2024-02-18T11:39:52.491728+0000","last_became_active":"2024-02-18T11:38:47.840860+0000","last_became_peered":"2024-02-18T11:38:47.840860+0000","last_unstale":"2024-02-18T11:39:52.491728+0000","last_undegraded":"2024-02-18T11:39:52.491728+0000","last_fullsized":"2024-02-18T11:39:52.491728+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:29:00.036507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.496302+0000","last_change":"2024-02-18T11:38:47.839901+0000","last_active":"2024-02-18T11:39:52.496302+0000","last_peered":"2024-02-18T11:39:52.496302+0000","last_clean":"2024-02-18T11:39:52.496302+0000","last_became_active":"2024-02-18T11:38:47.839532+0000","last_became_peered":"2024-02-18T11:38:47.839532+0000","last_unstale":"2024-02-18T11:39:52.496302+0000","last_undegraded":"2024-02-18T11:39:52.496302+0000","last_fullsized":"2024-02-18T11:39:52.496302+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T21:32:14.369846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64024,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":419,"num_read_kb":276,"num_write":157,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":192072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136,"ondisk_log_size":136,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318300,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":962560,"data_stored":462908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579936,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":41120,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704032,"statfs":{"total":95995035648,"available":95952928768,"internally_reserved":0,"allocated":1654784,"data_stored":1114390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":40434112},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743461,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":974848,"data_stored":463122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005098,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":970752,"data_stored":463020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266736,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":41032,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704120,"statfs":{"total":95995035648,"available":95953018880,"internally_reserved":0,"allocated":1568768,"data_stored":1054972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430261,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":962560,"data_stored":462511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574924,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":522327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738450,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":983040,"data_stored":463782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411400,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1048576,"data_stored":524068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117060,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":978944,"data_stored":462822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":35,"seq":150323855423,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36288,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708864,"statfs":{"total":95995035648,"available":95957876736,"internally_reserved":0,"allocated":966656,"data_stored":462948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691898,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32728,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712424,"statfs":{"total":95995035648,"available":95961522176,"internally_reserved":0,"allocated":1581056,"data_stored":1054656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":60023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":61764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-18T11:40:01.654 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph pg dump --format=json 2024-02-18T11:40:01.906 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T11:40:02.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:40:01 smithi158 ceph-mon[36626]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T11:40:02.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:40:01 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:40:02.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:40:01 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:40:02.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:40:01 smithi158 ceph-mon[36626]: pgmap v324: 97 pgs: 97 active+clean; 639 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s; 2 B/s, 0 objects/s recovering 2024-02-18T11:40:02.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:40:01 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:40:02.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:40:01 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:40:02.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:40:01 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:40:02.251 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:40:01 smithi012 ceph-mon[32460]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T11:40:02.252 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:40:01 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:40:02.252 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:40:01 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:40:02.252 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:40:01 smithi012 ceph-mon[32460]: pgmap v324: 97 pgs: 97 active+clean; 639 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s; 2 B/s, 0 objects/s recovering 2024-02-18T11:40:02.252 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:40:01 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:40:02.252 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:40:01 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:40:02.252 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:40:01 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:40:02.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:40:01 smithi188 ceph-mon[36925]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T11:40:02.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:40:01 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:40:02.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:40:01 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:40:02.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:40:01 smithi188 ceph-mon[36925]: pgmap v324: 97 pgs: 97 active+clean; 639 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s; 2 B/s, 0 objects/s recovering 2024-02-18T11:40:02.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:40:01 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:40:02.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:40:01 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:40:02.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:40:01 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:40:03.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:03.477+0000 7f9b4bb37700 1 -- 172.21.15.12:0/1275961807 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b441015a0 msgr2=0x7f9b4410d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:03.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:03.477+0000 7f9b4bb37700 1 --2- 172.21.15.12:0/1275961807 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b441015a0 0x7f9b4410d9f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f9b3400b0d0 tx=0x7f9b3400b3e0 comp rx=0 tx=0).stop 2024-02-18T11:40:03.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:03.477+0000 7f9b4bb37700 1 -- 172.21.15.12:0/1275961807 shutdown_connections 2024-02-18T11:40:03.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:03.477+0000 7f9b4bb37700 1 --2- 172.21.15.12:0/1275961807 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b441015a0 0x7f9b4410d9f0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:03.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:03.477+0000 7f9b4bb37700 1 --2- 172.21.15.12:0/1275961807 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9b44106c20 0x7f9b44107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:03.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:03.477+0000 7f9b4bb37700 1 --2- 172.21.15.12:0/1275961807 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9b44100be0 0x7f9b44101060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:03.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:03.477+0000 7f9b4bb37700 1 -- 172.21.15.12:0/1275961807 >> 172.21.15.12:0/1275961807 conn(0x7f9b44076c00 msgr2=0x7f9b44077010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:03.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:03.477+0000 7f9b4bb37700 1 -- 172.21.15.12:0/1275961807 shutdown_connections 2024-02-18T11:40:03.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:03.477+0000 7f9b4bb37700 1 -- 172.21.15.12:0/1275961807 wait complete. 2024-02-18T11:40:03.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:03.478+0000 7f9b4bb37700 1 Processor -- start 2024-02-18T11:40:03.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:03.478+0000 7f9b4bb37700 1 -- start start 2024-02-18T11:40:03.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:03.478+0000 7f9b4bb37700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9b44100be0 0x7f9b44198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:03.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:03.478+0000 7f9b4bb37700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9b441015a0 0x7f9b44198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:03.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:03.478+0000 7f9b4bb37700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b44106c20 0x7f9b4419cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:03.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:03.478+0000 7f9b4bb37700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b4419d0f0 con 0x7f9b44106c20 2024-02-18T11:40:03.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:03.478+0000 7f9b4bb37700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b4419d260 con 0x7f9b441015a0 2024-02-18T11:40:03.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:03.478+0000 7f9b4bb37700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b4419d420 con 0x7f9b44100be0 2024-02-18T11:40:03.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:03.478+0000 7f9b4a0d4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b44106c20 0x7f9b4419cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:03.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:03.478+0000 7f9b4a0d4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b44106c20 0x7f9b4419cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39852/0 (socket says 172.21.15.12:39852) 2024-02-18T11:40:03.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:03.478+0000 7f9b4a0d4700 1 -- 172.21.15.12:0/1617398510 learned_addr learned my addr 172.21.15.12:0/1617398510 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:40:03.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:03.478+0000 7f9b498d3700 1 --2- 172.21.15.12:0/1617398510 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9b44100be0 0x7f9b44198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:03.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:03.478+0000 7f9b490d2700 1 --2- 172.21.15.12:0/1617398510 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9b441015a0 0x7f9b44198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:03.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:03.478+0000 7f9b4a0d4700 1 -- 172.21.15.12:0/1617398510 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9b44100be0 msgr2=0x7f9b44198210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:03.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:03.478+0000 7f9b4a0d4700 1 --2- 172.21.15.12:0/1617398510 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9b44100be0 0x7f9b44198210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:03.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:03.478+0000 7f9b4a0d4700 1 -- 172.21.15.12:0/1617398510 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9b441015a0 msgr2=0x7f9b44198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:03.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:03.478+0000 7f9b4a0d4700 1 --2- 172.21.15.12:0/1617398510 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9b441015a0 0x7f9b44198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:03.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:03.478+0000 7f9b4a0d4700 1 -- 172.21.15.12:0/1617398510 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b34009d00 con 0x7f9b44106c20 2024-02-18T11:40:03.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:03.478+0000 7f9b490d2700 1 --2- 172.21.15.12:0/1617398510 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9b441015a0 0x7f9b44198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:40:03.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:03.478+0000 7f9b4a0d4700 1 --2- 172.21.15.12:0/1617398510 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b44106c20 0x7f9b4419cae0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9b3400d010 tx=0x7f9b34007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:03.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:03.479+0000 7f9b42ffd700 1 -- 172.21.15.12:0/1617398510 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b3400e040 con 0x7f9b44106c20 2024-02-18T11:40:03.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:03.479+0000 7f9b4bb37700 1 -- 172.21.15.12:0/1617398510 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b4419d6a0 con 0x7f9b44106c20 2024-02-18T11:40:03.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:03.479+0000 7f9b42ffd700 1 -- 172.21.15.12:0/1617398510 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9b3401dd70 con 0x7f9b44106c20 2024-02-18T11:40:03.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:03.479+0000 7f9b42ffd700 1 -- 172.21.15.12:0/1617398510 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b340123f0 con 0x7f9b44106c20 2024-02-18T11:40:03.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:03.479+0000 7f9b4bb37700 1 -- 172.21.15.12:0/1617398510 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b441a4f60 con 0x7f9b44106c20 2024-02-18T11:40:03.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:03.480+0000 7f9b4bb37700 1 -- 172.21.15.12:0/1617398510 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b44068970 con 0x7f9b44106c20 2024-02-18T11:40:03.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:03.481+0000 7f9b42ffd700 1 -- 172.21.15.12:0/1617398510 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 137296+0+0 (secure 0 0 0) 0x7f9b3401d890 con 0x7f9b44106c20 2024-02-18T11:40:03.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:03.481+0000 7f9b42ffd700 1 --2- 172.21.15.12:0/1617398510 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f9b2c0a2870 0x7f9b2c0a4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:03.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:03.481+0000 7f9b498d3700 1 --2- 172.21.15.12:0/1617398510 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f9b2c0a2870 0x7f9b2c0a4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:03.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:03.481+0000 7f9b42ffd700 1 -- 172.21.15.12:0/1617398510 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9b340d0a90 con 0x7f9b44106c20 2024-02-18T11:40:03.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:03.482+0000 7f9b498d3700 1 --2- 172.21.15.12:0/1617398510 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f9b2c0a2870 0x7f9b2c0a4d30 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c005950 tx=0x7f9b3c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:03.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:03.485+0000 7f9b42ffd700 1 -- 172.21.15.12:0/1617398510 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9b34099760 con 0x7f9b44106c20 2024-02-18T11:40:03.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:03.681+0000 7f9b4bb37700 1 -- 172.21.15.12:0/1617398510 --> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9b44110c50 con 0x7f9b2c0a2870 2024-02-18T11:40:03.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:03.688+0000 7f9b42ffd700 1 -- 172.21.15.12:0/1617398510 <== mgr.14152 v2:172.21.15.12:6800/1153736685 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+258711 (secure 0 0 0) 0x7f9b44110c50 con 0x7f9b2c0a2870 2024-02-18T11:40:03.686 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:40:03.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:03.692+0000 7f9b4bb37700 1 -- 172.21.15.12:0/1617398510 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f9b2c0a2870 msgr2=0x7f9b2c0a4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:03.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:03.692+0000 7f9b4bb37700 1 --2- 172.21.15.12:0/1617398510 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f9b2c0a2870 0x7f9b2c0a4d30 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c005950 tx=0x7f9b3c00b410 comp rx=0 tx=0).stop 2024-02-18T11:40:03.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:03.692+0000 7f9b4bb37700 1 -- 172.21.15.12:0/1617398510 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b44106c20 msgr2=0x7f9b4419cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:03.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:03.692+0000 7f9b4bb37700 1 --2- 172.21.15.12:0/1617398510 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b44106c20 0x7f9b4419cae0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9b3400d010 tx=0x7f9b34007870 comp rx=0 tx=0).stop 2024-02-18T11:40:03.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:03.692+0000 7f9b4bb37700 1 -- 172.21.15.12:0/1617398510 shutdown_connections 2024-02-18T11:40:03.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:03.692+0000 7f9b4bb37700 1 --2- 172.21.15.12:0/1617398510 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f9b2c0a2870 0x7f9b2c0a4d30 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:03.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:03.692+0000 7f9b4bb37700 1 --2- 172.21.15.12:0/1617398510 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b44106c20 0x7f9b4419cae0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:03.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:03.692+0000 7f9b4bb37700 1 --2- 172.21.15.12:0/1617398510 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9b44100be0 0x7f9b44198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:03.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:03.692+0000 7f9b4bb37700 1 --2- 172.21.15.12:0/1617398510 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9b441015a0 0x7f9b44198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:03.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:03.692+0000 7f9b4bb37700 1 -- 172.21.15.12:0/1617398510 >> 172.21.15.12:0/1617398510 conn(0x7f9b44076c00 msgr2=0x7f9b440fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:03.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:03.693+0000 7f9b4bb37700 1 -- 172.21.15.12:0/1617398510 shutdown_connections 2024-02-18T11:40:03.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:03.693+0000 7f9b4bb37700 1 -- 172.21.15.12:0/1617398510 wait complete. 2024-02-18T11:40:03.689 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-02-18T11:40:04.271 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:40:04 smithi012 ceph-mon[32460]: pgmap v325: 97 pgs: 97 active+clean; 639 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 2 B/s, 0 objects/s recovering 2024-02-18T11:40:04.271 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:40:04 smithi012 ceph-mon[32460]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T11:40:04.298 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":325,"stamp":"2024-02-18T11:40:03.034179+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654626,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":575,"num_read_kb":400,"num_write":340,"num_write_kb":3049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1180758,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":275,"ondisk_log_size":275,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":391704,"kb_used_data":13356,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124550120,"statfs":{"total":1151940427776,"available":1151539322880,"internally_reserved":0,"allocated":13676544,"data_stored":7511526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":387233082},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":59,"num_read_kb":29,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.547411"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.496585+0000","last_change":"2024-02-18T11:38:47.847524+0000","last_active":"2024-02-18T11:39:52.496585+0000","last_peered":"2024-02-18T11:39:52.496585+0000","last_clean":"2024-02-18T11:39:52.496585+0000","last_became_active":"2024-02-18T11:38:47.846756+0000","last_became_peered":"2024-02-18T11:38:47.846756+0000","last_unstale":"2024-02-18T11:39:52.496585+0000","last_undegraded":"2024-02-18T11:39:52.496585+0000","last_fullsized":"2024-02-18T11:39:52.496585+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:23:02.715418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.492264+0000","last_change":"2024-02-18T11:38:47.847391+0000","last_active":"2024-02-18T11:39:52.492264+0000","last_peered":"2024-02-18T11:39:52.492264+0000","last_clean":"2024-02-18T11:39:52.492264+0000","last_became_active":"2024-02-18T11:38:47.846558+0000","last_became_peered":"2024-02-18T11:38:47.846558+0000","last_unstale":"2024-02-18T11:39:52.492264+0000","last_undegraded":"2024-02-18T11:39:52.492264+0000","last_fullsized":"2024-02-18T11:39:52.492264+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:37:34.940328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.493048+0000","last_change":"2024-02-18T11:38:47.850143+0000","last_active":"2024-02-18T11:39:52.493048+0000","last_peered":"2024-02-18T11:39:52.493048+0000","last_clean":"2024-02-18T11:39:52.493048+0000","last_became_active":"2024-02-18T11:38:47.845844+0000","last_became_peered":"2024-02-18T11:38:47.845844+0000","last_unstale":"2024-02-18T11:39:52.493048+0000","last_undegraded":"2024-02-18T11:39:52.493048+0000","last_fullsized":"2024-02-18T11:39:52.493048+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T12:45:15.638456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.498814+0000","last_change":"2024-02-18T11:38:47.841317+0000","last_active":"2024-02-18T11:39:52.498814+0000","last_peered":"2024-02-18T11:39:52.498814+0000","last_clean":"2024-02-18T11:39:52.498814+0000","last_became_active":"2024-02-18T11:38:47.840982+0000","last_became_peered":"2024-02-18T11:38:47.840982+0000","last_unstale":"2024-02-18T11:39:52.498814+0000","last_undegraded":"2024-02-18T11:39:52.498814+0000","last_fullsized":"2024-02-18T11:39:52.498814+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:53:40.115908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.500194+0000","last_change":"2024-02-18T11:38:47.851422+0000","last_active":"2024-02-18T11:39:52.500194+0000","last_peered":"2024-02-18T11:39:52.500194+0000","last_clean":"2024-02-18T11:39:52.500194+0000","last_became_active":"2024-02-18T11:38:47.850479+0000","last_became_peered":"2024-02-18T11:38:47.850479+0000","last_unstale":"2024-02-18T11:39:52.500194+0000","last_undegraded":"2024-02-18T11:39:52.500194+0000","last_fullsized":"2024-02-18T11:39:52.500194+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:09:21.200282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.629275+0000","last_change":"2024-02-18T11:38:47.837468+0000","last_active":"2024-02-18T11:39:52.629275+0000","last_peered":"2024-02-18T11:39:52.629275+0000","last_clean":"2024-02-18T11:39:52.629275+0000","last_became_active":"2024-02-18T11:38:47.837129+0000","last_became_peered":"2024-02-18T11:38:47.837129+0000","last_unstale":"2024-02-18T11:39:52.629275+0000","last_undegraded":"2024-02-18T11:39:52.629275+0000","last_fullsized":"2024-02-18T11:39:52.629275+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:12:01.954496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.488243+0000","last_change":"2024-02-18T11:38:47.849467+0000","last_active":"2024-02-18T11:39:52.488243+0000","last_peered":"2024-02-18T11:39:52.488243+0000","last_clean":"2024-02-18T11:39:52.488243+0000","last_became_active":"2024-02-18T11:38:47.848938+0000","last_became_peered":"2024-02-18T11:38:47.848938+0000","last_unstale":"2024-02-18T11:39:52.488243+0000","last_undegraded":"2024-02-18T11:39:52.488243+0000","last_fullsized":"2024-02-18T11:39:52.488243+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:39:16.680834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.507552+0000","last_change":"2024-02-18T11:38:47.830143+0000","last_active":"2024-02-18T11:39:52.507552+0000","last_peered":"2024-02-18T11:39:52.507552+0000","last_clean":"2024-02-18T11:39:52.507552+0000","last_became_active":"2024-02-18T11:38:47.829765+0000","last_became_peered":"2024-02-18T11:38:47.829765+0000","last_unstale":"2024-02-18T11:39:52.507552+0000","last_undegraded":"2024-02-18T11:39:52.507552+0000","last_fullsized":"2024-02-18T11:39:52.507552+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:29:11.825847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.500879+0000","last_change":"2024-02-18T11:39:52.500879+0000","last_active":"2024-02-18T11:39:52.500879+0000","last_peered":"2024-02-18T11:39:52.500879+0000","last_clean":"2024-02-18T11:39:52.500879+0000","last_became_active":"2024-02-18T11:39:52.500547+0000","last_became_peered":"2024-02-18T11:39:52.500547+0000","last_unstale":"2024-02-18T11:39:52.500879+0000","last_undegraded":"2024-02-18T11:39:52.500879+0000","last_fullsized":"2024-02-18T11:39:52.500879+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:22:40.792845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,11],"acting":[1,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.629615+0000","last_change":"2024-02-18T11:38:47.836478+0000","last_active":"2024-02-18T11:39:52.629615+0000","last_peered":"2024-02-18T11:39:52.629615+0000","last_clean":"2024-02-18T11:39:52.629615+0000","last_became_active":"2024-02-18T11:38:47.836161+0000","last_became_peered":"2024-02-18T11:38:47.836161+0000","last_unstale":"2024-02-18T11:39:52.629615+0000","last_undegraded":"2024-02-18T11:39:52.629615+0000","last_fullsized":"2024-02-18T11:39:52.629615+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:28:13.828518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.488765+0000","last_change":"2024-02-18T11:38:47.849435+0000","last_active":"2024-02-18T11:39:52.488765+0000","last_peered":"2024-02-18T11:39:52.488765+0000","last_clean":"2024-02-18T11:39:52.488765+0000","last_became_active":"2024-02-18T11:38:47.848947+0000","last_became_peered":"2024-02-18T11:38:47.848947+0000","last_unstale":"2024-02-18T11:39:52.488765+0000","last_undegraded":"2024-02-18T11:39:52.488765+0000","last_fullsized":"2024-02-18T11:39:52.488765+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T11:48:14.101732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.498682+0000","last_change":"2024-02-18T11:38:47.840366+0000","last_active":"2024-02-18T11:39:52.498682+0000","last_peered":"2024-02-18T11:39:52.498682+0000","last_clean":"2024-02-18T11:39:52.498682+0000","last_became_active":"2024-02-18T11:38:47.840044+0000","last_became_peered":"2024-02-18T11:38:47.840044+0000","last_unstale":"2024-02-18T11:39:52.498682+0000","last_undegraded":"2024-02-18T11:39:52.498682+0000","last_fullsized":"2024-02-18T11:39:52.498682+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:31:05.184418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.491677+0000","last_change":"2024-02-18T11:38:47.844433+0000","last_active":"2024-02-18T11:39:52.491677+0000","last_peered":"2024-02-18T11:39:52.491677+0000","last_clean":"2024-02-18T11:39:52.491677+0000","last_became_active":"2024-02-18T11:38:47.844048+0000","last_became_peered":"2024-02-18T11:38:47.844048+0000","last_unstale":"2024-02-18T11:39:52.491677+0000","last_undegraded":"2024-02-18T11:39:52.491677+0000","last_fullsized":"2024-02-18T11:39:52.491677+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:08:27.698272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.492551+0000","last_change":"2024-02-18T11:38:47.842819+0000","last_active":"2024-02-18T11:39:52.492551+0000","last_peered":"2024-02-18T11:39:52.492551+0000","last_clean":"2024-02-18T11:39:52.492551+0000","last_became_active":"2024-02-18T11:38:47.842489+0000","last_became_peered":"2024-02-18T11:38:47.842489+0000","last_unstale":"2024-02-18T11:39:52.492551+0000","last_undegraded":"2024-02-18T11:39:52.492551+0000","last_fullsized":"2024-02-18T11:39:52.492551+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:03:21.808910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.494547+0000","last_change":"2024-02-18T11:39:52.494547+0000","last_active":"2024-02-18T11:39:52.494547+0000","last_peered":"2024-02-18T11:39:52.494547+0000","last_clean":"2024-02-18T11:39:52.494547+0000","last_became_active":"2024-02-18T11:39:52.494161+0000","last_became_peered":"2024-02-18T11:39:52.494161+0000","last_unstale":"2024-02-18T11:39:52.494547+0000","last_undegraded":"2024-02-18T11:39:52.494547+0000","last_fullsized":"2024-02-18T11:39:52.494547+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T12:52:53.831143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.490320+0000","last_change":"2024-02-18T11:38:47.838729+0000","last_active":"2024-02-18T11:39:52.490320+0000","last_peered":"2024-02-18T11:39:52.490320+0000","last_clean":"2024-02-18T11:39:52.490320+0000","last_became_active":"2024-02-18T11:38:47.838267+0000","last_became_peered":"2024-02-18T11:38:47.838267+0000","last_unstale":"2024-02-18T11:39:52.490320+0000","last_undegraded":"2024-02-18T11:39:52.490320+0000","last_fullsized":"2024-02-18T11:39:52.490320+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:08:37.797780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.496470+0000","last_change":"2024-02-18T11:38:47.847817+0000","last_active":"2024-02-18T11:39:52.496470+0000","last_peered":"2024-02-18T11:39:52.496470+0000","last_clean":"2024-02-18T11:39:52.496470+0000","last_became_active":"2024-02-18T11:38:47.847200+0000","last_became_peered":"2024-02-18T11:38:47.847200+0000","last_unstale":"2024-02-18T11:39:52.496470+0000","last_undegraded":"2024-02-18T11:39:52.496470+0000","last_fullsized":"2024-02-18T11:39:52.496470+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:22:12.425623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.488774+0000","last_change":"2024-02-18T11:38:47.843954+0000","last_active":"2024-02-18T11:39:52.488774+0000","last_peered":"2024-02-18T11:39:52.488774+0000","last_clean":"2024-02-18T11:39:52.488774+0000","last_became_active":"2024-02-18T11:38:47.840084+0000","last_became_peered":"2024-02-18T11:38:47.840084+0000","last_unstale":"2024-02-18T11:39:52.488774+0000","last_undegraded":"2024-02-18T11:39:52.488774+0000","last_fullsized":"2024-02-18T11:39:52.488774+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:21:13.124696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.493319+0000","last_change":"2024-02-18T11:38:47.834262+0000","last_active":"2024-02-18T11:39:52.493319+0000","last_peered":"2024-02-18T11:39:52.493319+0000","last_clean":"2024-02-18T11:39:52.493319+0000","last_became_active":"2024-02-18T11:38:47.833948+0000","last_became_peered":"2024-02-18T11:38:47.833948+0000","last_unstale":"2024-02-18T11:39:52.493319+0000","last_undegraded":"2024-02-18T11:39:52.493319+0000","last_fullsized":"2024-02-18T11:39:52.493319+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T21:10:16.673734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.488965+0000","last_change":"2024-02-18T11:38:46.533805+0000","last_active":"2024-02-18T11:39:52.488965+0000","last_peered":"2024-02-18T11:39:52.488965+0000","last_clean":"2024-02-18T11:39:52.488965+0000","last_became_active":"2024-02-18T11:38:46.533503+0000","last_became_peered":"2024-02-18T11:38:46.533503+0000","last_unstale":"2024-02-18T11:39:52.488965+0000","last_undegraded":"2024-02-18T11:39:52.488965+0000","last_fullsized":"2024-02-18T11:39:52.488965+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:05:28.087740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.508228+0000","last_change":"2024-02-18T11:38:47.833802+0000","last_active":"2024-02-18T11:39:52.508228+0000","last_peered":"2024-02-18T11:39:52.508228+0000","last_clean":"2024-02-18T11:39:52.508228+0000","last_became_active":"2024-02-18T11:38:47.833441+0000","last_became_peered":"2024-02-18T11:38:47.833441+0000","last_unstale":"2024-02-18T11:39:52.508228+0000","last_undegraded":"2024-02-18T11:39:52.508228+0000","last_fullsized":"2024-02-18T11:39:52.508228+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:17:03.294191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.508336+0000","last_change":"2024-02-18T11:38:46.540718+0000","last_active":"2024-02-18T11:39:52.508336+0000","last_peered":"2024-02-18T11:39:52.508336+0000","last_clean":"2024-02-18T11:39:52.508336+0000","last_became_active":"2024-02-18T11:38:46.540413+0000","last_became_peered":"2024-02-18T11:38:46.540413+0000","last_unstale":"2024-02-18T11:39:52.508336+0000","last_undegraded":"2024-02-18T11:39:52.508336+0000","last_fullsized":"2024-02-18T11:39:52.508336+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:17:53.287376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.492598+0000","last_change":"2024-02-18T11:38:47.846400+0000","last_active":"2024-02-18T11:39:52.492598+0000","last_peered":"2024-02-18T11:39:52.492598+0000","last_clean":"2024-02-18T11:39:52.492598+0000","last_became_active":"2024-02-18T11:38:47.845934+0000","last_became_peered":"2024-02-18T11:38:47.845934+0000","last_unstale":"2024-02-18T11:39:52.492598+0000","last_undegraded":"2024-02-18T11:39:52.492598+0000","last_fullsized":"2024-02-18T11:39:52.492598+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:20:12.093697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"87'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.501031+0000","last_change":"2024-02-18T11:38:46.537340+0000","last_active":"2024-02-18T11:39:52.501031+0000","last_peered":"2024-02-18T11:39:52.501031+0000","last_clean":"2024-02-18T11:39:52.501031+0000","last_became_active":"2024-02-18T11:38:46.537019+0000","last_became_peered":"2024-02-18T11:38:46.537019+0000","last_unstale":"2024-02-18T11:39:52.501031+0000","last_undegraded":"2024-02-18T11:39:52.501031+0000","last_fullsized":"2024-02-18T11:39:52.501031+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T23:00:50.052654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.490785+0000","last_change":"2024-02-18T11:38:47.852656+0000","last_active":"2024-02-18T11:39:52.490785+0000","last_peered":"2024-02-18T11:39:52.490785+0000","last_clean":"2024-02-18T11:39:52.490785+0000","last_became_active":"2024-02-18T11:38:47.852279+0000","last_became_peered":"2024-02-18T11:38:47.852279+0000","last_unstale":"2024-02-18T11:39:52.490785+0000","last_undegraded":"2024-02-18T11:39:52.490785+0000","last_fullsized":"2024-02-18T11:39:52.490785+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:51:48.911504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.500473+0000","last_change":"2024-02-18T11:38:46.533091+0000","last_active":"2024-02-18T11:39:52.500473+0000","last_peered":"2024-02-18T11:39:52.500473+0000","last_clean":"2024-02-18T11:39:52.500473+0000","last_became_active":"2024-02-18T11:38:46.532504+0000","last_became_peered":"2024-02-18T11:38:46.532504+0000","last_unstale":"2024-02-18T11:39:52.500473+0000","last_undegraded":"2024-02-18T11:39:52.500473+0000","last_fullsized":"2024-02-18T11:39:52.500473+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:12:00.529365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.499324+0000","last_change":"2024-02-18T11:38:47.836578+0000","last_active":"2024-02-18T11:39:52.499324+0000","last_peered":"2024-02-18T11:39:52.499324+0000","last_clean":"2024-02-18T11:39:52.499324+0000","last_became_active":"2024-02-18T11:38:47.836217+0000","last_became_peered":"2024-02-18T11:38:47.836217+0000","last_unstale":"2024-02-18T11:39:52.499324+0000","last_undegraded":"2024-02-18T11:39:52.499324+0000","last_fullsized":"2024-02-18T11:39:52.499324+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T21:53:39.011822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"87'6","reported_seq":99,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:40:00.640732+0000","last_change":"2024-02-18T11:38:46.532196+0000","last_active":"2024-02-18T11:40:00.640732+0000","last_peered":"2024-02-18T11:40:00.640732+0000","last_clean":"2024-02-18T11:40:00.640732+0000","last_became_active":"2024-02-18T11:38:46.531868+0000","last_became_peered":"2024-02-18T11:38:46.531868+0000","last_unstale":"2024-02-18T11:40:00.640732+0000","last_undegraded":"2024-02-18T11:40:00.640732+0000","last_fullsized":"2024-02-18T11:40:00.640732+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:52:07.396279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.500073+0000","last_change":"2024-02-18T11:38:47.837329+0000","last_active":"2024-02-18T11:39:52.500073+0000","last_peered":"2024-02-18T11:39:52.500073+0000","last_clean":"2024-02-18T11:39:52.500073+0000","last_became_active":"2024-02-18T11:38:47.836989+0000","last_became_peered":"2024-02-18T11:38:47.836989+0000","last_unstale":"2024-02-18T11:39:52.500073+0000","last_undegraded":"2024-02-18T11:39:52.500073+0000","last_fullsized":"2024-02-18T11:39:52.500073+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:21:16.801916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"87'3","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.629204+0000","last_change":"2024-02-18T11:38:46.533244+0000","last_active":"2024-02-18T11:39:52.629204+0000","last_peered":"2024-02-18T11:39:52.629204+0000","last_clean":"2024-02-18T11:39:52.629204+0000","last_became_active":"2024-02-18T11:38:46.532326+0000","last_became_peered":"2024-02-18T11:38:46.532326+0000","last_unstale":"2024-02-18T11:39:52.629204+0000","last_undegraded":"2024-02-18T11:39:52.629204+0000","last_fullsized":"2024-02-18T11:39:52.629204+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:32:32.979056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.499349+0000","last_change":"2024-02-18T11:38:47.824560+0000","last_active":"2024-02-18T11:39:52.499349+0000","last_peered":"2024-02-18T11:39:52.499349+0000","last_clean":"2024-02-18T11:39:52.499349+0000","last_became_active":"2024-02-18T11:38:47.824095+0000","last_became_peered":"2024-02-18T11:38:47.824095+0000","last_unstale":"2024-02-18T11:39:52.499349+0000","last_undegraded":"2024-02-18T11:39:52.499349+0000","last_fullsized":"2024-02-18T11:39:52.499349+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:27:09.870585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"87'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.491527+0000","last_change":"2024-02-18T11:38:46.526024+0000","last_active":"2024-02-18T11:39:52.491527+0000","last_peered":"2024-02-18T11:39:52.491527+0000","last_clean":"2024-02-18T11:39:52.491527+0000","last_became_active":"2024-02-18T11:38:46.525735+0000","last_became_peered":"2024-02-18T11:38:46.525735+0000","last_unstale":"2024-02-18T11:39:52.491527+0000","last_undegraded":"2024-02-18T11:39:52.491527+0000","last_fullsized":"2024-02-18T11:39:52.491527+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:25:10.659210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.507643+0000","last_change":"2024-02-18T11:38:47.841743+0000","last_active":"2024-02-18T11:39:52.507643+0000","last_peered":"2024-02-18T11:39:52.507643+0000","last_clean":"2024-02-18T11:39:52.507643+0000","last_became_active":"2024-02-18T11:38:47.840848+0000","last_became_peered":"2024-02-18T11:38:47.840848+0000","last_unstale":"2024-02-18T11:39:52.507643+0000","last_undegraded":"2024-02-18T11:39:52.507643+0000","last_fullsized":"2024-02-18T11:39:52.507643+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:50:37.787782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"87'1","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.498571+0000","last_change":"2024-02-18T11:38:46.531355+0000","last_active":"2024-02-18T11:39:52.498571+0000","last_peered":"2024-02-18T11:39:52.498571+0000","last_clean":"2024-02-18T11:39:52.498571+0000","last_became_active":"2024-02-18T11:38:46.530601+0000","last_became_peered":"2024-02-18T11:38:46.530601+0000","last_unstale":"2024-02-18T11:39:52.498571+0000","last_undegraded":"2024-02-18T11:39:52.498571+0000","last_fullsized":"2024-02-18T11:39:52.498571+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:55:29.670274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.488009+0000","last_change":"2024-02-18T11:38:47.844506+0000","last_active":"2024-02-18T11:39:52.488009+0000","last_peered":"2024-02-18T11:39:52.488009+0000","last_clean":"2024-02-18T11:39:52.488009+0000","last_became_active":"2024-02-18T11:38:47.844158+0000","last_became_peered":"2024-02-18T11:38:47.844158+0000","last_unstale":"2024-02-18T11:39:52.488009+0000","last_undegraded":"2024-02-18T11:39:52.488009+0000","last_fullsized":"2024-02-18T11:39:52.488009+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T23:13:17.360077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"87'3","reported_seq":92,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:59.638788+0000","last_change":"2024-02-18T11:38:46.536313+0000","last_active":"2024-02-18T11:39:59.638788+0000","last_peered":"2024-02-18T11:39:59.638788+0000","last_clean":"2024-02-18T11:39:59.638788+0000","last_became_active":"2024-02-18T11:38:46.535910+0000","last_became_peered":"2024-02-18T11:38:46.535910+0000","last_unstale":"2024-02-18T11:39:59.638788+0000","last_undegraded":"2024-02-18T11:39:59.638788+0000","last_fullsized":"2024-02-18T11:39:59.638788+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:56:16.606415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":70,"num_read_kb":70,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.490880+0000","last_change":"2024-02-18T11:38:47.836131+0000","last_active":"2024-02-18T11:39:52.490880+0000","last_peered":"2024-02-18T11:39:52.490880+0000","last_clean":"2024-02-18T11:39:52.490880+0000","last_became_active":"2024-02-18T11:38:47.835702+0000","last_became_peered":"2024-02-18T11:38:47.835702+0000","last_unstale":"2024-02-18T11:39:52.490880+0000","last_undegraded":"2024-02-18T11:39:52.490880+0000","last_fullsized":"2024-02-18T11:39:52.490880+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:58:11.622972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"87'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.488516+0000","last_change":"2024-02-18T11:38:46.530031+0000","last_active":"2024-02-18T11:39:52.488516+0000","last_peered":"2024-02-18T11:39:52.488516+0000","last_clean":"2024-02-18T11:39:52.488516+0000","last_became_active":"2024-02-18T11:38:46.529674+0000","last_became_peered":"2024-02-18T11:38:46.529674+0000","last_unstale":"2024-02-18T11:39:52.488516+0000","last_undegraded":"2024-02-18T11:39:52.488516+0000","last_fullsized":"2024-02-18T11:39:52.488516+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T21:13:53.955970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.492495+0000","last_change":"2024-02-18T11:38:47.852340+0000","last_active":"2024-02-18T11:39:52.492495+0000","last_peered":"2024-02-18T11:39:52.492495+0000","last_clean":"2024-02-18T11:39:52.492495+0000","last_became_active":"2024-02-18T11:38:47.847978+0000","last_became_peered":"2024-02-18T11:38:47.847978+0000","last_unstale":"2024-02-18T11:39:52.492495+0000","last_undegraded":"2024-02-18T11:39:52.492495+0000","last_fullsized":"2024-02-18T11:39:52.492495+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:47:09.667619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.499446+0000","last_change":"2024-02-18T11:38:46.533511+0000","last_active":"2024-02-18T11:39:52.499446+0000","last_peered":"2024-02-18T11:39:52.499446+0000","last_clean":"2024-02-18T11:39:52.499446+0000","last_became_active":"2024-02-18T11:38:46.533162+0000","last_became_peered":"2024-02-18T11:38:46.533162+0000","last_unstale":"2024-02-18T11:39:52.499446+0000","last_undegraded":"2024-02-18T11:39:52.499446+0000","last_fullsized":"2024-02-18T11:39:52.499446+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:00:49.307489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.490667+0000","last_change":"2024-02-18T11:38:47.843835+0000","last_active":"2024-02-18T11:39:52.490667+0000","last_peered":"2024-02-18T11:39:52.490667+0000","last_clean":"2024-02-18T11:39:52.490667+0000","last_became_active":"2024-02-18T11:38:47.843517+0000","last_became_peered":"2024-02-18T11:38:47.843517+0000","last_unstale":"2024-02-18T11:39:52.490667+0000","last_undegraded":"2024-02-18T11:39:52.490667+0000","last_fullsized":"2024-02-18T11:39:52.490667+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:54:02.259854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'29","reported_seq":196,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:40:00.639866+0000","last_change":"2024-02-18T11:38:46.536900+0000","last_active":"2024-02-18T11:40:00.639866+0000","last_peered":"2024-02-18T11:40:00.639866+0000","last_clean":"2024-02-18T11:40:00.639866+0000","last_became_active":"2024-02-18T11:38:46.536563+0000","last_became_peered":"2024-02-18T11:38:46.536563+0000","last_unstale":"2024-02-18T11:40:00.639866+0000","last_undegraded":"2024-02-18T11:40:00.639866+0000","last_fullsized":"2024-02-18T11:40:00.639866+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T12:56:57.959482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60023,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":148,"num_read_kb":132,"num_write":30,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.500408+0000","last_change":"2024-02-18T11:38:47.842452+0000","last_active":"2024-02-18T11:39:52.500408+0000","last_peered":"2024-02-18T11:39:52.500408+0000","last_clean":"2024-02-18T11:39:52.500408+0000","last_became_active":"2024-02-18T11:38:47.842105+0000","last_became_peered":"2024-02-18T11:38:47.842105+0000","last_unstale":"2024-02-18T11:39:52.500408+0000","last_undegraded":"2024-02-18T11:39:52.500408+0000","last_fullsized":"2024-02-18T11:39:52.500408+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:10:05.131880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"87'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.493002+0000","last_change":"2024-02-18T11:38:46.534220+0000","last_active":"2024-02-18T11:39:52.493002+0000","last_peered":"2024-02-18T11:39:52.493002+0000","last_clean":"2024-02-18T11:39:52.493002+0000","last_became_active":"2024-02-18T11:38:46.532929+0000","last_became_peered":"2024-02-18T11:38:46.532929+0000","last_unstale":"2024-02-18T11:39:52.493002+0000","last_undegraded":"2024-02-18T11:39:52.493002+0000","last_fullsized":"2024-02-18T11:39:52.493002+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:41:42.674196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.491511+0000","last_change":"2024-02-18T11:38:47.848434+0000","last_active":"2024-02-18T11:39:52.491511+0000","last_peered":"2024-02-18T11:39:52.491511+0000","last_clean":"2024-02-18T11:39:52.491511+0000","last_became_active":"2024-02-18T11:38:47.848092+0000","last_became_peered":"2024-02-18T11:38:47.848092+0000","last_unstale":"2024-02-18T11:39:52.491511+0000","last_undegraded":"2024-02-18T11:39:52.491511+0000","last_fullsized":"2024-02-18T11:39:52.491511+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:23:35.292982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"87'3","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.499075+0000","last_change":"2024-02-18T11:38:46.531579+0000","last_active":"2024-02-18T11:39:52.499075+0000","last_peered":"2024-02-18T11:39:52.499075+0000","last_clean":"2024-02-18T11:39:52.499075+0000","last_became_active":"2024-02-18T11:38:46.531236+0000","last_became_peered":"2024-02-18T11:38:46.531236+0000","last_unstale":"2024-02-18T11:39:52.499075+0000","last_undegraded":"2024-02-18T11:39:52.499075+0000","last_fullsized":"2024-02-18T11:39:52.499075+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:54:48.886442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.493252+0000","last_change":"2024-02-18T11:39:52.493252+0000","last_active":"2024-02-18T11:39:52.493252+0000","last_peered":"2024-02-18T11:39:52.493252+0000","last_clean":"2024-02-18T11:39:52.493252+0000","last_became_active":"2024-02-18T11:39:52.492827+0000","last_became_peered":"2024-02-18T11:39:52.492827+0000","last_unstale":"2024-02-18T11:39:52.493252+0000","last_undegraded":"2024-02-18T11:39:52.493252+0000","last_fullsized":"2024-02-18T11:39:52.493252+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:24:18.392029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'3","reported_seq":95,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:40:00.640100+0000","last_change":"2024-02-18T11:38:46.528086+0000","last_active":"2024-02-18T11:40:00.640100+0000","last_peered":"2024-02-18T11:40:00.640100+0000","last_clean":"2024-02-18T11:40:00.640100+0000","last_became_active":"2024-02-18T11:38:46.527753+0000","last_became_peered":"2024-02-18T11:38:46.527753+0000","last_unstale":"2024-02-18T11:40:00.640100+0000","last_undegraded":"2024-02-18T11:40:00.640100+0000","last_fullsized":"2024-02-18T11:40:00.640100+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:48:09.524280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.488168+0000","last_change":"2024-02-18T11:38:47.849139+0000","last_active":"2024-02-18T11:39:52.488168+0000","last_peered":"2024-02-18T11:39:52.488168+0000","last_clean":"2024-02-18T11:39:52.488168+0000","last_became_active":"2024-02-18T11:38:47.848752+0000","last_became_peered":"2024-02-18T11:38:47.848752+0000","last_unstale":"2024-02-18T11:39:52.488168+0000","last_undegraded":"2024-02-18T11:39:52.488168+0000","last_fullsized":"2024-02-18T11:39:52.488168+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:00:56.046670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"87'15","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.499251+0000","last_change":"2024-02-18T11:38:46.540471+0000","last_active":"2024-02-18T11:39:52.499251+0000","last_peered":"2024-02-18T11:39:52.499251+0000","last_clean":"2024-02-18T11:39:52.499251+0000","last_became_active":"2024-02-18T11:38:46.540169+0000","last_became_peered":"2024-02-18T11:38:46.540169+0000","last_unstale":"2024-02-18T11:39:52.499251+0000","last_undegraded":"2024-02-18T11:39:52.499251+0000","last_fullsized":"2024-02-18T11:39:52.499251+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:29:46.001510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.498896+0000","last_change":"2024-02-18T11:38:47.838077+0000","last_active":"2024-02-18T11:39:52.498896+0000","last_peered":"2024-02-18T11:39:52.498896+0000","last_clean":"2024-02-18T11:39:52.498896+0000","last_became_active":"2024-02-18T11:38:47.837137+0000","last_became_peered":"2024-02-18T11:38:47.837137+0000","last_unstale":"2024-02-18T11:39:52.498896+0000","last_undegraded":"2024-02-18T11:39:52.498896+0000","last_fullsized":"2024-02-18T11:39:52.498896+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:06:15.443851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.506447+0000","last_change":"2024-02-18T11:39:52.505994+0000","last_active":"2024-02-18T11:39:52.506447+0000","last_peered":"2024-02-18T11:39:52.506447+0000","last_clean":"2024-02-18T11:39:52.506447+0000","last_became_active":"2024-02-18T11:39:52.501544+0000","last_became_peered":"2024-02-18T11:39:52.501544+0000","last_unstale":"2024-02-18T11:39:52.506447+0000","last_undegraded":"2024-02-18T11:39:52.506447+0000","last_fullsized":"2024-02-18T11:39:52.506447+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:51:13.850168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.496442+0000","last_change":"2024-02-18T11:38:47.835481+0000","last_active":"2024-02-18T11:39:52.496442+0000","last_peered":"2024-02-18T11:39:52.496442+0000","last_clean":"2024-02-18T11:39:52.496442+0000","last_became_active":"2024-02-18T11:38:47.835071+0000","last_became_peered":"2024-02-18T11:38:47.835071+0000","last_unstale":"2024-02-18T11:39:52.496442+0000","last_undegraded":"2024-02-18T11:39:52.496442+0000","last_fullsized":"2024-02-18T11:39:52.496442+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T12:10:42.098646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.497775+0000","last_change":"2024-02-18T11:39:52.497775+0000","last_active":"2024-02-18T11:39:52.497775+0000","last_peered":"2024-02-18T11:39:52.497775+0000","last_clean":"2024-02-18T11:39:52.497775+0000","last_became_active":"2024-02-18T11:39:52.497326+0000","last_became_peered":"2024-02-18T11:39:52.497326+0000","last_unstale":"2024-02-18T11:39:52.497775+0000","last_undegraded":"2024-02-18T11:39:52.497775+0000","last_fullsized":"2024-02-18T11:39:52.497775+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:01:39.098340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'1","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.629525+0000","last_change":"2024-02-18T11:38:47.842020+0000","last_active":"2024-02-18T11:39:52.629525+0000","last_peered":"2024-02-18T11:39:52.629525+0000","last_clean":"2024-02-18T11:39:52.629525+0000","last_became_active":"2024-02-18T11:38:47.841656+0000","last_became_peered":"2024-02-18T11:38:47.841656+0000","last_unstale":"2024-02-18T11:39:52.629525+0000","last_undegraded":"2024-02-18T11:39:52.629525+0000","last_fullsized":"2024-02-18T11:39:52.629525+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:12:50.508340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"87'1","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.488501+0000","last_change":"2024-02-18T11:38:46.531867+0000","last_active":"2024-02-18T11:39:52.488501+0000","last_peered":"2024-02-18T11:39:52.488501+0000","last_clean":"2024-02-18T11:39:52.488501+0000","last_became_active":"2024-02-18T11:38:46.531051+0000","last_became_peered":"2024-02-18T11:38:46.531051+0000","last_unstale":"2024-02-18T11:39:52.488501+0000","last_undegraded":"2024-02-18T11:39:52.488501+0000","last_fullsized":"2024-02-18T11:39:52.488501+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:15:26.808764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.629482+0000","last_change":"2024-02-18T11:38:47.835160+0000","last_active":"2024-02-18T11:39:52.629482+0000","last_peered":"2024-02-18T11:39:52.629482+0000","last_clean":"2024-02-18T11:39:52.629482+0000","last_became_active":"2024-02-18T11:38:47.834813+0000","last_became_peered":"2024-02-18T11:38:47.834813+0000","last_unstale":"2024-02-18T11:39:52.629482+0000","last_undegraded":"2024-02-18T11:39:52.629482+0000","last_fullsized":"2024-02-18T11:39:52.629482+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:30:59.674796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.498735+0000","last_change":"2024-02-18T11:38:47.838197+0000","last_active":"2024-02-18T11:39:52.498735+0000","last_peered":"2024-02-18T11:39:52.498735+0000","last_clean":"2024-02-18T11:39:52.498735+0000","last_became_active":"2024-02-18T11:38:47.837393+0000","last_became_peered":"2024-02-18T11:38:47.837393+0000","last_unstale":"2024-02-18T11:39:52.498735+0000","last_undegraded":"2024-02-18T11:39:52.498735+0000","last_fullsized":"2024-02-18T11:39:52.498735+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:13:19.095082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'4","reported_seq":94,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:40:00.639465+0000","last_change":"2024-02-18T11:38:46.538682+0000","last_active":"2024-02-18T11:40:00.639465+0000","last_peered":"2024-02-18T11:40:00.639465+0000","last_clean":"2024-02-18T11:40:00.639465+0000","last_became_active":"2024-02-18T11:38:46.538356+0000","last_became_peered":"2024-02-18T11:38:46.538356+0000","last_unstale":"2024-02-18T11:40:00.639465+0000","last_undegraded":"2024-02-18T11:40:00.639465+0000","last_fullsized":"2024-02-18T11:40:00.639465+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:41:02.449905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":71,"num_read_kb":71,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.500472+0000","last_change":"2024-02-18T11:39:52.500472+0000","last_active":"2024-02-18T11:39:52.500472+0000","last_peered":"2024-02-18T11:39:52.500472+0000","last_clean":"2024-02-18T11:39:52.500472+0000","last_became_active":"2024-02-18T11:39:52.500154+0000","last_became_peered":"2024-02-18T11:39:52.500154+0000","last_unstale":"2024-02-18T11:39:52.500472+0000","last_undegraded":"2024-02-18T11:39:52.500472+0000","last_fullsized":"2024-02-18T11:39:52.500472+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:47:23.278384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"87'3","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.498713+0000","last_change":"2024-02-18T11:38:46.531504+0000","last_active":"2024-02-18T11:39:52.498713+0000","last_peered":"2024-02-18T11:39:52.498713+0000","last_clean":"2024-02-18T11:39:52.498713+0000","last_became_active":"2024-02-18T11:38:46.531173+0000","last_became_peered":"2024-02-18T11:38:46.531173+0000","last_unstale":"2024-02-18T11:39:52.498713+0000","last_undegraded":"2024-02-18T11:39:52.498713+0000","last_fullsized":"2024-02-18T11:39:52.498713+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:32:40.632054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":345,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.488254+0000","last_change":"2024-02-18T11:37:34.534077+0000","last_active":"2024-02-18T11:39:52.488254+0000","last_peered":"2024-02-18T11:39:52.488254+0000","last_clean":"2024-02-18T11:39:52.488254+0000","last_became_active":"2024-02-18T11:37:34.533557+0000","last_became_peered":"2024-02-18T11:37:34.533557+0000","last_unstale":"2024-02-18T11:39:52.488254+0000","last_undegraded":"2024-02-18T11:39:52.488254+0000","last_fullsized":"2024-02-18T11:39:52.488254+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:34:15.604912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:34:15.604912+0000","last_clean_scrub_stamp":"2024-02-18T11:34:15.604912+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T23:01:35.216671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.491283+0000","last_change":"2024-02-18T11:38:47.838123+0000","last_active":"2024-02-18T11:39:52.491283+0000","last_peered":"2024-02-18T11:39:52.491283+0000","last_clean":"2024-02-18T11:39:52.491283+0000","last_became_active":"2024-02-18T11:38:47.837713+0000","last_became_peered":"2024-02-18T11:38:47.837713+0000","last_unstale":"2024-02-18T11:39:52.491283+0000","last_undegraded":"2024-02-18T11:39:52.491283+0000","last_fullsized":"2024-02-18T11:39:52.491283+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T21:47:44.609399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'3","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.488599+0000","last_change":"2024-02-18T11:38:46.529508+0000","last_active":"2024-02-18T11:39:52.488599+0000","last_peered":"2024-02-18T11:39:52.488599+0000","last_clean":"2024-02-18T11:39:52.488599+0000","last_became_active":"2024-02-18T11:38:46.529165+0000","last_became_peered":"2024-02-18T11:38:46.529165+0000","last_unstale":"2024-02-18T11:39:52.488599+0000","last_undegraded":"2024-02-18T11:39:52.488599+0000","last_fullsized":"2024-02-18T11:39:52.488599+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:10:12.774664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"87'1","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.500817+0000","last_change":"2024-02-18T11:38:46.542236+0000","last_active":"2024-02-18T11:39:52.500817+0000","last_peered":"2024-02-18T11:39:52.500817+0000","last_clean":"2024-02-18T11:39:52.500817+0000","last_became_active":"2024-02-18T11:38:46.541901+0000","last_became_peered":"2024-02-18T11:38:46.541901+0000","last_unstale":"2024-02-18T11:39:52.500817+0000","last_undegraded":"2024-02-18T11:39:52.500817+0000","last_fullsized":"2024-02-18T11:39:52.500817+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:40:28.699330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.492068+0000","last_change":"2024-02-18T11:38:47.841210+0000","last_active":"2024-02-18T11:39:52.492068+0000","last_peered":"2024-02-18T11:39:52.492068+0000","last_clean":"2024-02-18T11:39:52.492068+0000","last_became_active":"2024-02-18T11:38:47.838873+0000","last_became_peered":"2024-02-18T11:38:47.838873+0000","last_unstale":"2024-02-18T11:39:52.492068+0000","last_undegraded":"2024-02-18T11:39:52.492068+0000","last_fullsized":"2024-02-18T11:39:52.492068+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T21:22:11.304237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.488856+0000","last_change":"2024-02-18T11:38:46.533820+0000","last_active":"2024-02-18T11:39:52.488856+0000","last_peered":"2024-02-18T11:39:52.488856+0000","last_clean":"2024-02-18T11:39:52.488856+0000","last_became_active":"2024-02-18T11:38:46.533342+0000","last_became_peered":"2024-02-18T11:38:46.533342+0000","last_unstale":"2024-02-18T11:39:52.488856+0000","last_undegraded":"2024-02-18T11:39:52.488856+0000","last_fullsized":"2024-02-18T11:39:52.488856+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:27:34.814511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.508332+0000","last_change":"2024-02-18T11:38:47.837034+0000","last_active":"2024-02-18T11:39:52.508332+0000","last_peered":"2024-02-18T11:39:52.508332+0000","last_clean":"2024-02-18T11:39:52.508332+0000","last_became_active":"2024-02-18T11:38:47.836340+0000","last_became_peered":"2024-02-18T11:38:47.836340+0000","last_unstale":"2024-02-18T11:39:52.508332+0000","last_undegraded":"2024-02-18T11:39:52.508332+0000","last_fullsized":"2024-02-18T11:39:52.508332+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:44:10.812220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.498656+0000","last_change":"2024-02-18T11:38:46.529389+0000","last_active":"2024-02-18T11:39:52.498656+0000","last_peered":"2024-02-18T11:39:52.498656+0000","last_clean":"2024-02-18T11:39:52.498656+0000","last_became_active":"2024-02-18T11:38:46.529076+0000","last_became_peered":"2024-02-18T11:38:46.529076+0000","last_unstale":"2024-02-18T11:39:52.498656+0000","last_undegraded":"2024-02-18T11:39:52.498656+0000","last_fullsized":"2024-02-18T11:39:52.498656+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T21:01:05.553901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.492174+0000","last_change":"2024-02-18T11:38:47.847677+0000","last_active":"2024-02-18T11:39:52.492174+0000","last_peered":"2024-02-18T11:39:52.492174+0000","last_clean":"2024-02-18T11:39:52.492174+0000","last_became_active":"2024-02-18T11:38:47.847051+0000","last_became_peered":"2024-02-18T11:38:47.847051+0000","last_unstale":"2024-02-18T11:39:52.492174+0000","last_undegraded":"2024-02-18T11:39:52.492174+0000","last_fullsized":"2024-02-18T11:39:52.492174+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:25:53.398173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.490396+0000","last_change":"2024-02-18T11:38:46.525030+0000","last_active":"2024-02-18T11:39:52.490396+0000","last_peered":"2024-02-18T11:39:52.490396+0000","last_clean":"2024-02-18T11:39:52.490396+0000","last_became_active":"2024-02-18T11:38:46.524405+0000","last_became_peered":"2024-02-18T11:38:46.524405+0000","last_unstale":"2024-02-18T11:39:52.490396+0000","last_undegraded":"2024-02-18T11:39:52.490396+0000","last_fullsized":"2024-02-18T11:39:52.490396+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T12:33:41.751625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.499092+0000","last_change":"2024-02-18T11:38:47.836583+0000","last_active":"2024-02-18T11:39:52.499092+0000","last_peered":"2024-02-18T11:39:52.499092+0000","last_clean":"2024-02-18T11:39:52.499092+0000","last_became_active":"2024-02-18T11:38:47.836138+0000","last_became_peered":"2024-02-18T11:38:47.836138+0000","last_unstale":"2024-02-18T11:39:52.499092+0000","last_undegraded":"2024-02-18T11:39:52.499092+0000","last_fullsized":"2024-02-18T11:39:52.499092+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:34:04.002137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"87'3","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.507827+0000","last_change":"2024-02-18T11:38:46.538553+0000","last_active":"2024-02-18T11:39:52.507827+0000","last_peered":"2024-02-18T11:39:52.507827+0000","last_clean":"2024-02-18T11:39:52.507827+0000","last_became_active":"2024-02-18T11:38:46.538216+0000","last_became_peered":"2024-02-18T11:38:46.538216+0000","last_unstale":"2024-02-18T11:39:52.507827+0000","last_undegraded":"2024-02-18T11:39:52.507827+0000","last_fullsized":"2024-02-18T11:39:52.507827+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:31:32.713327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.496065+0000","last_change":"2024-02-18T11:38:47.840452+0000","last_active":"2024-02-18T11:39:52.496065+0000","last_peered":"2024-02-18T11:39:52.496065+0000","last_clean":"2024-02-18T11:39:52.496065+0000","last_became_active":"2024-02-18T11:38:47.840071+0000","last_became_peered":"2024-02-18T11:38:47.840071+0000","last_unstale":"2024-02-18T11:39:52.496065+0000","last_undegraded":"2024-02-18T11:39:52.496065+0000","last_fullsized":"2024-02-18T11:39:52.496065+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:07:22.746854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"87'1","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.492907+0000","last_change":"2024-02-18T11:38:46.535460+0000","last_active":"2024-02-18T11:39:52.492907+0000","last_peered":"2024-02-18T11:39:52.492907+0000","last_clean":"2024-02-18T11:39:52.492907+0000","last_became_active":"2024-02-18T11:38:46.534626+0000","last_became_peered":"2024-02-18T11:38:46.534626+0000","last_unstale":"2024-02-18T11:39:52.492907+0000","last_undegraded":"2024-02-18T11:39:52.492907+0000","last_fullsized":"2024-02-18T11:39:52.492907+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:56:18.434798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.508274+0000","last_change":"2024-02-18T11:38:47.838616+0000","last_active":"2024-02-18T11:39:52.508274+0000","last_peered":"2024-02-18T11:39:52.508274+0000","last_clean":"2024-02-18T11:39:52.508274+0000","last_became_active":"2024-02-18T11:38:47.838301+0000","last_became_peered":"2024-02-18T11:38:47.838301+0000","last_unstale":"2024-02-18T11:39:52.508274+0000","last_undegraded":"2024-02-18T11:39:52.508274+0000","last_fullsized":"2024-02-18T11:39:52.508274+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T11:43:37.731817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.500339+0000","last_change":"2024-02-18T11:38:46.536544+0000","last_active":"2024-02-18T11:39:52.500339+0000","last_peered":"2024-02-18T11:39:52.500339+0000","last_clean":"2024-02-18T11:39:52.500339+0000","last_became_active":"2024-02-18T11:38:46.536215+0000","last_became_peered":"2024-02-18T11:38:46.536215+0000","last_unstale":"2024-02-18T11:39:52.500339+0000","last_undegraded":"2024-02-18T11:39:52.500339+0000","last_fullsized":"2024-02-18T11:39:52.500339+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:12:44.496950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.488718+0000","last_change":"2024-02-18T11:38:47.847872+0000","last_active":"2024-02-18T11:39:52.488718+0000","last_peered":"2024-02-18T11:39:52.488718+0000","last_clean":"2024-02-18T11:39:52.488718+0000","last_became_active":"2024-02-18T11:38:47.847537+0000","last_became_peered":"2024-02-18T11:38:47.847537+0000","last_unstale":"2024-02-18T11:39:52.488718+0000","last_undegraded":"2024-02-18T11:39:52.488718+0000","last_fullsized":"2024-02-18T11:39:52.488718+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T12:30:46.754937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"87'3","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.493409+0000","last_change":"2024-02-18T11:38:46.534677+0000","last_active":"2024-02-18T11:39:52.493409+0000","last_peered":"2024-02-18T11:39:52.493409+0000","last_clean":"2024-02-18T11:39:52.493409+0000","last_became_active":"2024-02-18T11:38:46.527789+0000","last_became_peered":"2024-02-18T11:38:46.527789+0000","last_unstale":"2024-02-18T11:39:52.493409+0000","last_undegraded":"2024-02-18T11:39:52.493409+0000","last_fullsized":"2024-02-18T11:39:52.493409+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:13:15.365557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.488754+0000","last_change":"2024-02-18T11:38:47.849874+0000","last_active":"2024-02-18T11:39:52.488754+0000","last_peered":"2024-02-18T11:39:52.488754+0000","last_clean":"2024-02-18T11:39:52.488754+0000","last_became_active":"2024-02-18T11:38:47.849318+0000","last_became_peered":"2024-02-18T11:38:47.849318+0000","last_unstale":"2024-02-18T11:39:52.488754+0000","last_undegraded":"2024-02-18T11:39:52.488754+0000","last_fullsized":"2024-02-18T11:39:52.488754+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:22:31.974649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'1","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.499208+0000","last_change":"2024-02-18T11:38:46.531691+0000","last_active":"2024-02-18T11:39:52.499208+0000","last_peered":"2024-02-18T11:39:52.499208+0000","last_clean":"2024-02-18T11:39:52.499208+0000","last_became_active":"2024-02-18T11:38:46.530987+0000","last_became_peered":"2024-02-18T11:38:46.530987+0000","last_unstale":"2024-02-18T11:39:52.499208+0000","last_undegraded":"2024-02-18T11:39:52.499208+0000","last_fullsized":"2024-02-18T11:39:52.499208+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:15:46.228599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'1","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.488492+0000","last_change":"2024-02-18T11:38:47.850273+0000","last_active":"2024-02-18T11:39:52.488492+0000","last_peered":"2024-02-18T11:39:52.488492+0000","last_clean":"2024-02-18T11:39:52.488492+0000","last_became_active":"2024-02-18T11:38:47.849110+0000","last_became_peered":"2024-02-18T11:38:47.849110+0000","last_unstale":"2024-02-18T11:39:52.488492+0000","last_undegraded":"2024-02-18T11:39:52.488492+0000","last_fullsized":"2024-02-18T11:39:52.488492+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T23:18:55.096768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.491794+0000","last_change":"2024-02-18T11:38:46.537641+0000","last_active":"2024-02-18T11:39:52.491794+0000","last_peered":"2024-02-18T11:39:52.491794+0000","last_clean":"2024-02-18T11:39:52.491794+0000","last_became_active":"2024-02-18T11:38:46.537311+0000","last_became_peered":"2024-02-18T11:38:46.537311+0000","last_unstale":"2024-02-18T11:39:52.491794+0000","last_undegraded":"2024-02-18T11:39:52.491794+0000","last_fullsized":"2024-02-18T11:39:52.491794+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:45.510078+0000","last_clean_scrub_stamp":"2024-02-18T11:38:45.510078+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:13:25.720250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.488697+0000","last_change":"2024-02-18T11:38:47.833177+0000","last_active":"2024-02-18T11:39:52.488697+0000","last_peered":"2024-02-18T11:39:52.488697+0000","last_clean":"2024-02-18T11:39:52.488697+0000","last_became_active":"2024-02-18T11:38:47.832839+0000","last_became_peered":"2024-02-18T11:38:47.832839+0000","last_unstale":"2024-02-18T11:39:52.488697+0000","last_undegraded":"2024-02-18T11:39:52.488697+0000","last_fullsized":"2024-02-18T11:39:52.488697+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:12:28.448505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.496193+0000","last_change":"2024-02-18T11:38:47.841062+0000","last_active":"2024-02-18T11:39:52.496193+0000","last_peered":"2024-02-18T11:39:52.496193+0000","last_clean":"2024-02-18T11:39:52.496193+0000","last_became_active":"2024-02-18T11:38:47.840730+0000","last_became_peered":"2024-02-18T11:38:47.840730+0000","last_unstale":"2024-02-18T11:39:52.496193+0000","last_undegraded":"2024-02-18T11:39:52.496193+0000","last_fullsized":"2024-02-18T11:39:52.496193+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:18:13.417676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.493337+0000","last_change":"2024-02-18T11:38:47.839488+0000","last_active":"2024-02-18T11:39:52.493337+0000","last_peered":"2024-02-18T11:39:52.493337+0000","last_clean":"2024-02-18T11:39:52.493337+0000","last_became_active":"2024-02-18T11:38:47.839134+0000","last_became_peered":"2024-02-18T11:38:47.839134+0000","last_unstale":"2024-02-18T11:39:52.493337+0000","last_undegraded":"2024-02-18T11:39:52.493337+0000","last_fullsized":"2024-02-18T11:39:52.493337+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:51:11.149307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.492959+0000","last_change":"2024-02-18T11:38:47.845397+0000","last_active":"2024-02-18T11:39:52.492959+0000","last_peered":"2024-02-18T11:39:52.492959+0000","last_clean":"2024-02-18T11:39:52.492959+0000","last_became_active":"2024-02-18T11:38:47.845027+0000","last_became_peered":"2024-02-18T11:38:47.845027+0000","last_unstale":"2024-02-18T11:39:52.492959+0000","last_undegraded":"2024-02-18T11:39:52.492959+0000","last_fullsized":"2024-02-18T11:39:52.492959+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:26:44.260093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.492867+0000","last_change":"2024-02-18T11:38:47.839238+0000","last_active":"2024-02-18T11:39:52.492867+0000","last_peered":"2024-02-18T11:39:52.492867+0000","last_clean":"2024-02-18T11:39:52.492867+0000","last_became_active":"2024-02-18T11:38:47.838928+0000","last_became_peered":"2024-02-18T11:38:47.838928+0000","last_unstale":"2024-02-18T11:39:52.492867+0000","last_undegraded":"2024-02-18T11:39:52.492867+0000","last_fullsized":"2024-02-18T11:39:52.492867+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:32:10.867251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.495865+0000","last_change":"2024-02-18T11:39:52.495865+0000","last_active":"2024-02-18T11:39:52.495865+0000","last_peered":"2024-02-18T11:39:52.495865+0000","last_clean":"2024-02-18T11:39:52.495865+0000","last_became_active":"2024-02-18T11:39:52.495484+0000","last_became_peered":"2024-02-18T11:39:52.495484+0000","last_unstale":"2024-02-18T11:39:52.495865+0000","last_undegraded":"2024-02-18T11:39:52.495865+0000","last_fullsized":"2024-02-18T11:39:52.495865+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:13:29.153602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.496463+0000","last_change":"2024-02-18T11:38:47.839821+0000","last_active":"2024-02-18T11:39:52.496463+0000","last_peered":"2024-02-18T11:39:52.496463+0000","last_clean":"2024-02-18T11:39:52.496463+0000","last_became_active":"2024-02-18T11:38:47.839431+0000","last_became_peered":"2024-02-18T11:38:47.839431+0000","last_unstale":"2024-02-18T11:39:52.496463+0000","last_undegraded":"2024-02-18T11:39:52.496463+0000","last_fullsized":"2024-02-18T11:39:52.496463+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:43:07.002223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.493560+0000","last_change":"2024-02-18T11:38:47.843937+0000","last_active":"2024-02-18T11:39:52.493560+0000","last_peered":"2024-02-18T11:39:52.493560+0000","last_clean":"2024-02-18T11:39:52.493560+0000","last_became_active":"2024-02-18T11:38:47.843603+0000","last_became_peered":"2024-02-18T11:38:47.843603+0000","last_unstale":"2024-02-18T11:39:52.493560+0000","last_undegraded":"2024-02-18T11:39:52.493560+0000","last_fullsized":"2024-02-18T11:39:52.493560+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:45:42.228604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.508376+0000","last_change":"2024-02-18T11:38:47.836702+0000","last_active":"2024-02-18T11:39:52.508376+0000","last_peered":"2024-02-18T11:39:52.508376+0000","last_clean":"2024-02-18T11:39:52.508376+0000","last_became_active":"2024-02-18T11:38:47.834843+0000","last_became_peered":"2024-02-18T11:38:47.834843+0000","last_unstale":"2024-02-18T11:39:52.508376+0000","last_undegraded":"2024-02-18T11:39:52.508376+0000","last_fullsized":"2024-02-18T11:39:52.508376+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T12:32:31.342561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.498701+0000","last_change":"2024-02-18T11:38:47.837429+0000","last_active":"2024-02-18T11:39:52.498701+0000","last_peered":"2024-02-18T11:39:52.498701+0000","last_clean":"2024-02-18T11:39:52.498701+0000","last_became_active":"2024-02-18T11:38:47.832983+0000","last_became_peered":"2024-02-18T11:38:47.832983+0000","last_unstale":"2024-02-18T11:39:52.498701+0000","last_undegraded":"2024-02-18T11:39:52.498701+0000","last_fullsized":"2024-02-18T11:39:52.498701+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:37:07.582626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.507679+0000","last_change":"2024-02-18T11:38:47.837041+0000","last_active":"2024-02-18T11:39:52.507679+0000","last_peered":"2024-02-18T11:39:52.507679+0000","last_clean":"2024-02-18T11:39:52.507679+0000","last_became_active":"2024-02-18T11:38:47.836730+0000","last_became_peered":"2024-02-18T11:38:47.836730+0000","last_unstale":"2024-02-18T11:39:52.507679+0000","last_undegraded":"2024-02-18T11:39:52.507679+0000","last_fullsized":"2024-02-18T11:39:52.507679+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:44:21.998090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.501957+0000","last_change":"2024-02-18T11:39:52.501957+0000","last_active":"2024-02-18T11:39:52.501957+0000","last_peered":"2024-02-18T11:39:52.501957+0000","last_clean":"2024-02-18T11:39:52.501957+0000","last_became_active":"2024-02-18T11:39:52.501561+0000","last_became_peered":"2024-02-18T11:39:52.501561+0000","last_unstale":"2024-02-18T11:39:52.501957+0000","last_undegraded":"2024-02-18T11:39:52.501957+0000","last_fullsized":"2024-02-18T11:39:52.501957+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:49:37.243875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,8],"acting":[1,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.491728+0000","last_change":"2024-02-18T11:38:47.841231+0000","last_active":"2024-02-18T11:39:52.491728+0000","last_peered":"2024-02-18T11:39:52.491728+0000","last_clean":"2024-02-18T11:39:52.491728+0000","last_became_active":"2024-02-18T11:38:47.840860+0000","last_became_peered":"2024-02-18T11:38:47.840860+0000","last_unstale":"2024-02-18T11:39:52.491728+0000","last_undegraded":"2024-02-18T11:39:52.491728+0000","last_fullsized":"2024-02-18T11:39:52.491728+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:29:00.036507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T11:39:52.496302+0000","last_change":"2024-02-18T11:38:47.839901+0000","last_active":"2024-02-18T11:39:52.496302+0000","last_peered":"2024-02-18T11:39:52.496302+0000","last_clean":"2024-02-18T11:39:52.496302+0000","last_became_active":"2024-02-18T11:38:47.839532+0000","last_became_peered":"2024-02-18T11:38:47.839532+0000","last_unstale":"2024-02-18T11:39:52.496302+0000","last_undegraded":"2024-02-18T11:39:52.496302+0000","last_fullsized":"2024-02-18T11:39:52.496302+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:38:46.511832+0000","last_clean_scrub_stamp":"2024-02-18T11:38:46.511832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T21:32:14.369846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64024,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":449,"num_read_kb":291,"num_write":157,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":192072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136,"ondisk_log_size":136,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318301,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":962560,"data_stored":462908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579937,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":41120,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704032,"statfs":{"total":95995035648,"available":95952928768,"internally_reserved":0,"allocated":1654784,"data_stored":1114390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":40434112},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743462,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":974848,"data_stored":463122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005099,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":970752,"data_stored":463020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266737,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":41032,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704120,"statfs":{"total":95995035648,"available":95953018880,"internally_reserved":0,"allocated":1568768,"data_stored":1054972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430262,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":962560,"data_stored":462511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":522327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738451,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":983040,"data_stored":463782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411401,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1048576,"data_stored":524068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117060,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":978944,"data_stored":462822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":35,"seq":150323855424,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36288,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708864,"statfs":{"total":95995035648,"available":95957876736,"internally_reserved":0,"allocated":966656,"data_stored":462948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691899,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32728,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712424,"statfs":{"total":95995035648,"available":95961522176,"internally_reserved":0,"allocated":1581056,"data_stored":1054656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":60023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":61764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-18T11:40:04.302 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-02-18T11:40:04.302 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-02-18T11:40:04.302 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-02-18T11:40:04.302 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph health --format=json 2024-02-18T11:40:04.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:40:04 smithi188 ceph-mon[36925]: pgmap v325: 97 pgs: 97 active+clean; 639 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 2 B/s, 0 objects/s recovering 2024-02-18T11:40:04.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:40:04 smithi188 ceph-mon[36925]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T11:40:04.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:40:04 smithi158 ceph-mon[36626]: pgmap v325: 97 pgs: 97 active+clean; 639 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 2 B/s, 0 objects/s recovering 2024-02-18T11:40:04.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:40:04 smithi158 ceph-mon[36626]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T11:40:04.547 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T11:40:06.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:06.192+0000 7fb8c4b11700 1 -- 172.21.15.12:0/4244228433 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8c006a450 msgr2=0x7fb8c006a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:06.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:06.192+0000 7fb8c4b11700 1 --2- 172.21.15.12:0/4244228433 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8c006a450 0x7fb8c006a830 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb8b800b0a0 tx=0x7fb8b800b3b0 comp rx=0 tx=0).stop 2024-02-18T11:40:06.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:06.192+0000 7fb8c4b11700 1 -- 172.21.15.12:0/4244228433 shutdown_connections 2024-02-18T11:40:06.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:06.192+0000 7fb8c4b11700 1 --2- 172.21.15.12:0/4244228433 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8c0107030 0x7fb8c010ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:06.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:06.192+0000 7fb8c4b11700 1 --2- 172.21.15.12:0/4244228433 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8c006a450 0x7fb8c006a830 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:06.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:06.192+0000 7fb8c4b11700 1 --2- 172.21.15.12:0/4244228433 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb8c006ae00 0x7fb8c0106900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:06.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:06.192+0000 7fb8c4b11700 1 -- 172.21.15.12:0/4244228433 >> 172.21.15.12:0/4244228433 conn(0x7fb8c0076c00 msgr2=0x7fb8c0077010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:06.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:06.192+0000 7fb8c4b11700 1 -- 172.21.15.12:0/4244228433 shutdown_connections 2024-02-18T11:40:06.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:06.192+0000 7fb8c4b11700 1 -- 172.21.15.12:0/4244228433 wait complete. 2024-02-18T11:40:06.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:06.193+0000 7fb8c4b11700 1 Processor -- start 2024-02-18T11:40:06.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:06.193+0000 7fb8c4b11700 1 -- start start 2024-02-18T11:40:06.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:06.193+0000 7fb8c4b11700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8c006a450 0x7fb8c0100cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:06.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:06.193+0000 7fb8c4b11700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8c006ae00 0x7fb8c01011f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:06.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:06.193+0000 7fb8c4b11700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb8c0107030 0x7fb8c0101730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:06.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:06.193+0000 7fb8c4b11700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c01a6690 con 0x7fb8c006a450 2024-02-18T11:40:06.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:06.193+0000 7fb8c4b11700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c01a6800 con 0x7fb8c0107030 2024-02-18T11:40:06.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:06.193+0000 7fb8c4b11700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c01a69c0 con 0x7fb8c006ae00 2024-02-18T11:40:06.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:06.193+0000 7fb8be59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8c006a450 0x7fb8c0100cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:06.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:06.193+0000 7fb8be59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8c006a450 0x7fb8c0100cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39874/0 (socket says 172.21.15.12:39874) 2024-02-18T11:40:06.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:06.193+0000 7fb8be59c700 1 -- 172.21.15.12:0/3262075191 learned_addr learned my addr 172.21.15.12:0/3262075191 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:40:06.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:06.193+0000 7fb8bdd9b700 1 --2- 172.21.15.12:0/3262075191 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8c006ae00 0x7fb8c01011f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:06.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:06.193+0000 7fb8be59c700 1 -- 172.21.15.12:0/3262075191 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8c006ae00 msgr2=0x7fb8c01011f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:06.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:06.193+0000 7fb8be59c700 1 --2- 172.21.15.12:0/3262075191 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8c006ae00 0x7fb8c01011f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:06.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:06.193+0000 7fb8bed9d700 1 --2- 172.21.15.12:0/3262075191 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb8c0107030 0x7fb8c0101730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:06.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:06.193+0000 7fb8be59c700 1 -- 172.21.15.12:0/3262075191 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb8c0107030 msgr2=0x7fb8c0101730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:06.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:06.193+0000 7fb8be59c700 1 --2- 172.21.15.12:0/3262075191 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb8c0107030 0x7fb8c0101730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:06.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:06.193+0000 7fb8be59c700 1 -- 172.21.15.12:0/3262075191 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8b8009d00 con 0x7fb8c006a450 2024-02-18T11:40:06.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:06.193+0000 7fb8be59c700 1 --2- 172.21.15.12:0/3262075191 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8c006a450 0x7fb8c0100cb0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb8b8000f80 tx=0x7fb8b8007be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:06.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:06.194+0000 7fb8b77fe700 1 -- 172.21.15.12:0/3262075191 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b800e040 con 0x7fb8c006a450 2024-02-18T11:40:06.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:06.194+0000 7fb8c4b11700 1 -- 172.21.15.12:0/3262075191 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8c01a6c40 con 0x7fb8c006a450 2024-02-18T11:40:06.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:06.194+0000 7fb8b77fe700 1 -- 172.21.15.12:0/3262075191 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb8b8005550 con 0x7fb8c006a450 2024-02-18T11:40:06.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:06.194+0000 7fb8c4b11700 1 -- 172.21.15.12:0/3262075191 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8c01a70b0 con 0x7fb8c006a450 2024-02-18T11:40:06.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:06.194+0000 7fb8b77fe700 1 -- 172.21.15.12:0/3262075191 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b801db50 con 0x7fb8c006a450 2024-02-18T11:40:06.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:06.195+0000 7fb8c4b11700 1 -- 172.21.15.12:0/3262075191 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8c010a1e0 con 0x7fb8c006a450 2024-02-18T11:40:06.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:06.196+0000 7fb8b77fe700 1 -- 172.21.15.12:0/3262075191 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 137296+0+0 (secure 0 0 0) 0x7fb8b80056c0 con 0x7fb8c006a450 2024-02-18T11:40:06.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:06.196+0000 7fb8b77fe700 1 --2- 172.21.15.12:0/3262075191 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fb8ac0a2870 0x7fb8ac0a4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:06.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:06.197+0000 7fb8bdd9b700 1 --2- 172.21.15.12:0/3262075191 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fb8ac0a2870 0x7fb8ac0a4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:06.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:06.197+0000 7fb8b77fe700 1 -- 172.21.15.12:0/3262075191 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb8b80cfe40 con 0x7fb8c006a450 2024-02-18T11:40:06.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:06.197+0000 7fb8bdd9b700 1 --2- 172.21.15.12:0/3262075191 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fb8ac0a2870 0x7fb8ac0a4d30 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb8c0100870 tx=0x7fb8a8009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:06.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:06.200+0000 7fb8b77fe700 1 -- 172.21.15.12:0/3262075191 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb8b8098b40 con 0x7fb8c006a450 2024-02-18T11:40:06.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:40:06 smithi188 ceph-mon[36925]: pgmap v326: 97 pgs: 97 active+clean; 639 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-18T11:40:06.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:40:06 smithi012 ceph-mon[32460]: pgmap v326: 97 pgs: 97 active+clean; 639 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-18T11:40:06.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:06.447+0000 7fb8c4b11700 1 -- 172.21.15.12:0/3262075191 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fb8c0068970 con 0x7fb8c006a450 2024-02-18T11:40:06.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:06.447+0000 7fb8b77fe700 1 -- 172.21.15.12:0/3262075191 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fb8b809c200 con 0x7fb8c006a450 2024-02-18T11:40:06.441 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:40:06.441 INFO:teuthology.orchestra.run.smithi012.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-02-18T11:40:06.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:06.449+0000 7fb8c4b11700 1 -- 172.21.15.12:0/3262075191 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fb8ac0a2870 msgr2=0x7fb8ac0a4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:06.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:06.449+0000 7fb8c4b11700 1 --2- 172.21.15.12:0/3262075191 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fb8ac0a2870 0x7fb8ac0a4d30 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb8c0100870 tx=0x7fb8a8009190 comp rx=0 tx=0).stop 2024-02-18T11:40:06.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:06.449+0000 7fb8c4b11700 1 -- 172.21.15.12:0/3262075191 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8c006a450 msgr2=0x7fb8c0100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:06.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:06.449+0000 7fb8c4b11700 1 --2- 172.21.15.12:0/3262075191 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8c006a450 0x7fb8c0100cb0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb8b8000f80 tx=0x7fb8b8007be0 comp rx=0 tx=0).stop 2024-02-18T11:40:06.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:06.449+0000 7fb8c4b11700 1 -- 172.21.15.12:0/3262075191 shutdown_connections 2024-02-18T11:40:06.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:06.449+0000 7fb8c4b11700 1 --2- 172.21.15.12:0/3262075191 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb8c0107030 0x7fb8c0101730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:06.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:06.449+0000 7fb8c4b11700 1 --2- 172.21.15.12:0/3262075191 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fb8ac0a2870 0x7fb8ac0a4d30 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:06.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:06.449+0000 7fb8c4b11700 1 --2- 172.21.15.12:0/3262075191 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8c006a450 0x7fb8c0100cb0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:06.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:06.449+0000 7fb8c4b11700 1 --2- 172.21.15.12:0/3262075191 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8c006ae00 0x7fb8c01011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:06.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:06.449+0000 7fb8c4b11700 1 -- 172.21.15.12:0/3262075191 >> 172.21.15.12:0/3262075191 conn(0x7fb8c0076c00 msgr2=0x7fb8c00fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:06.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:06.449+0000 7fb8c4b11700 1 -- 172.21.15.12:0/3262075191 shutdown_connections 2024-02-18T11:40:06.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:06.449+0000 7fb8c4b11700 1 -- 172.21.15.12:0/3262075191 wait complete. 2024-02-18T11:40:06.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:40:06 smithi158 ceph-mon[36626]: pgmap v326: 97 pgs: 97 active+clean; 639 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-18T11:40:07.066 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-02-18T11:40:07.067 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-18T11:40:07.172 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:40:07 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/3262075191' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-18T11:40:07.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.182+0000 7f43ccd8c640 1 -- 172.21.15.12:0/3400799705 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f43c81012d0 msgr2=0x7f43c8101750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:07.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.182+0000 7f43ccd8c640 1 --2- 172.21.15.12:0/3400799705 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f43c81012d0 0x7f43c8101750 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f43b0009900 tx=0x7f43b002bc30 comp rx=0 tx=0).stop 2024-02-18T11:40:07.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.182+0000 7f43ccd8c640 1 -- 172.21.15.12:0/3400799705 shutdown_connections 2024-02-18T11:40:07.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.182+0000 7f43ccd8c640 1 --2- 172.21.15.12:0/3400799705 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43c8101c90 0x7f43c8108520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:07.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.182+0000 7f43ccd8c640 1 --2- 172.21.15.12:0/3400799705 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f43c81012d0 0x7f43c8101750 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:07.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.182+0000 7f43ccd8c640 1 --2- 172.21.15.12:0/3400799705 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f43c81000e0 0x7f43c8100500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:07.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.182+0000 7f43ccd8c640 1 -- 172.21.15.12:0/3400799705 >> 172.21.15.12:0/3400799705 conn(0x7f43c80fb650 msgr2=0x7f43c80fda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:07.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.182+0000 7f43ccd8c640 1 -- 172.21.15.12:0/3400799705 shutdown_connections 2024-02-18T11:40:07.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.182+0000 7f43ccd8c640 1 -- 172.21.15.12:0/3400799705 wait complete. 2024-02-18T11:40:07.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.183+0000 7f43ccd8c640 1 Processor -- start 2024-02-18T11:40:07.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.183+0000 7f43ccd8c640 1 -- start start 2024-02-18T11:40:07.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.183+0000 7f43ccd8c640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f43c81000e0 0x7f43c81973d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:07.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.183+0000 7f43ccd8c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43c81012d0 0x7f43c8197910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:07.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.183+0000 7f43ccd8c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f43c8101c90 0x7f43c819e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:07.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.183+0000 7f43ccd8c640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c819ee50 con 0x7f43c8101c90 2024-02-18T11:40:07.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.183+0000 7f43ccd8c640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c819efc0 con 0x7f43c81000e0 2024-02-18T11:40:07.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.183+0000 7f43ccd8c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c819f180 con 0x7f43c81012d0 2024-02-18T11:40:07.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.183+0000 7f43c6d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f43c8101c90 0x7f43c819e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:07.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.183+0000 7f43c6d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f43c8101c90 0x7f43c819e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39892/0 (socket says 172.21.15.12:39892) 2024-02-18T11:40:07.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.183+0000 7f43c6d9d640 1 -- 172.21.15.12:0/3185188805 learned_addr learned my addr 172.21.15.12:0/3185188805 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:40:07.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.183+0000 7f43bffff640 1 --2- 172.21.15.12:0/3185188805 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43c81012d0 0x7f43c8197910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:07.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.184+0000 7f43c659c640 1 --2- 172.21.15.12:0/3185188805 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f43c81000e0 0x7f43c81973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:07.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.184+0000 7f43c6d9d640 1 -- 172.21.15.12:0/3185188805 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43c81012d0 msgr2=0x7f43c8197910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:07.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.184+0000 7f43c6d9d640 1 --2- 172.21.15.12:0/3185188805 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43c81012d0 0x7f43c8197910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:07.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.184+0000 7f43c6d9d640 1 -- 172.21.15.12:0/3185188805 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f43c81000e0 msgr2=0x7f43c81973d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:07.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.184+0000 7f43c6d9d640 1 --2- 172.21.15.12:0/3185188805 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f43c81000e0 0x7f43c81973d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:07.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.184+0000 7f43c6d9d640 1 -- 172.21.15.12:0/3185188805 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b0009560 con 0x7f43c8101c90 2024-02-18T11:40:07.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.184+0000 7f43c6d9d640 1 --2- 172.21.15.12:0/3185188805 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f43c8101c90 0x7f43c819e910 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f43c000cad0 tx=0x7f43c000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:07.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.184+0000 7f43bf7fe640 1 -- 172.21.15.12:0/3185188805 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c000ae20 con 0x7f43c8101c90 2024-02-18T11:40:07.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.184+0000 7f43bf7fe640 1 -- 172.21.15.12:0/3185188805 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f43c0005020 con 0x7f43c8101c90 2024-02-18T11:40:07.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.184+0000 7f43bf7fe640 1 -- 172.21.15.12:0/3185188805 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c000f6b0 con 0x7f43c8101c90 2024-02-18T11:40:07.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.184+0000 7f43ccd8c640 1 -- 172.21.15.12:0/3185188805 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43c819f460 con 0x7f43c8101c90 2024-02-18T11:40:07.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.184+0000 7f43ccd8c640 1 -- 172.21.15.12:0/3185188805 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43c819f970 con 0x7f43c8101c90 2024-02-18T11:40:07.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.185+0000 7f43ccd8c640 1 -- 172.21.15.12:0/3185188805 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43c810bb50 con 0x7f43c8101c90 2024-02-18T11:40:07.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.186+0000 7f43bf7fe640 1 -- 172.21.15.12:0/3185188805 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 137296+0+0 (secure 0 0 0) 0x7f43c0005520 con 0x7f43c8101c90 2024-02-18T11:40:07.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.187+0000 7f43bf7fe640 1 --2- 172.21.15.12:0/3185188805 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f43b40a27e0 0x7f43b40a4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:07.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.187+0000 7f43c659c640 1 --2- 172.21.15.12:0/3185188805 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f43b40a27e0 0x7f43b40a4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:07.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.187+0000 7f43bf7fe640 1 -- 172.21.15.12:0/3185188805 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f43c00ceac0 con 0x7f43c8101c90 2024-02-18T11:40:07.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.187+0000 7f43c659c640 1 --2- 172.21.15.12:0/3185188805 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f43b40a27e0 0x7f43b40a4ca0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f43c8101130 tx=0x7f43b8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:07.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.190+0000 7f43bf7fe640 1 -- 172.21.15.12:0/3185188805 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f43c0097700 con 0x7f43c8101c90 2024-02-18T11:40:07.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:40:07 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/3262075191' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-18T11:40:07.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.374+0000 7f43ccd8c640 1 -- 172.21.15.12:0/3185188805 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f43c810bce0 con 0x7f43c8101c90 2024-02-18T11:40:07.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.376+0000 7f43bf7fe640 1 -- 172.21.15.12:0/3185188805 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+22662 (secure 0 0 0) 0x7f43c009adc0 con 0x7f43c8101c90 2024-02-18T11:40:07.370 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:40:07.370 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":89,"fsid":"0c8bb0de-ce51-11ee-95ba-87774f69a715","created":"2024-02-18T11:30:49.110059+0000","modified":"2024-02-18T11:39:52.478480+0000","last_up_change":"2024-02-18T11:37:57.512337+0000","last_in_change":"2024-02-18T11:37:36.567969+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-18T11:34:15.281138+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-18T11:38:44.802134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-18T11:38:46.081188+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"35f15ea6-deb8-48a1-a7dd-ab649ec13533","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":536522426},{"type":"v1","addr":"172.21.15.12:6803","nonce":536522426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":536522426},{"type":"v1","addr":"172.21.15.12:6805","nonce":536522426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":536522426},{"type":"v1","addr":"172.21.15.12:6809","nonce":536522426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":536522426},{"type":"v1","addr":"172.21.15.12:6807","nonce":536522426}]},"public_addr":"172.21.15.12:6803/536522426","cluster_addr":"172.21.15.12:6805/536522426","heartbeat_back_addr":"172.21.15.12:6809/536522426","heartbeat_front_addr":"172.21.15.12:6807/536522426","state":["exists","up"]},{"osd":1,"uuid":"2af937ef-d827-44e6-b805-70cffbfe1126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":4249111488},{"type":"v1","addr":"172.21.15.158:6801","nonce":4249111488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":4249111488},{"type":"v1","addr":"172.21.15.158:6803","nonce":4249111488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":4249111488},{"type":"v1","addr":"172.21.15.158:6807","nonce":4249111488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":4249111488},{"type":"v1","addr":"172.21.15.158:6805","nonce":4249111488}]},"public_addr":"172.21.15.158:6801/4249111488","cluster_addr":"172.21.15.158:6803/4249111488","heartbeat_back_addr":"172.21.15.158:6807/4249111488","heartbeat_front_addr":"172.21.15.158:6805/4249111488","state":["exists","up"]},{"osd":2,"uuid":"17b804dd-31df-45ae-a637-4a5f65bbaf26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":627976150},{"type":"v1","addr":"172.21.15.188:6801","nonce":627976150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":627976150},{"type":"v1","addr":"172.21.15.188:6803","nonce":627976150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":627976150},{"type":"v1","addr":"172.21.15.188:6807","nonce":627976150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":627976150},{"type":"v1","addr":"172.21.15.188:6805","nonce":627976150}]},"public_addr":"172.21.15.188:6801/627976150","cluster_addr":"172.21.15.188:6803/627976150","heartbeat_back_addr":"172.21.15.188:6807/627976150","heartbeat_front_addr":"172.21.15.188:6805/627976150","state":["exists","up"]},{"osd":3,"uuid":"3f9cc423-cdfa-4acf-aaa2-b58b32ccb9cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":3758512503},{"type":"v1","addr":"172.21.15.12:6811","nonce":3758512503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":3758512503},{"type":"v1","addr":"172.21.15.12:6813","nonce":3758512503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3758512503},{"type":"v1","addr":"172.21.15.12:6817","nonce":3758512503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":3758512503},{"type":"v1","addr":"172.21.15.12:6815","nonce":3758512503}]},"public_addr":"172.21.15.12:6811/3758512503","cluster_addr":"172.21.15.12:6813/3758512503","heartbeat_back_addr":"172.21.15.12:6817/3758512503","heartbeat_front_addr":"172.21.15.12:6815/3758512503","state":["exists","up"]},{"osd":4,"uuid":"a92e20ec-af1c-4ef4-a0cb-caba5fcc71aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":3892820997},{"type":"v1","addr":"172.21.15.158:6809","nonce":3892820997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":3892820997},{"type":"v1","addr":"172.21.15.158:6811","nonce":3892820997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":3892820997},{"type":"v1","addr":"172.21.15.158:6815","nonce":3892820997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":3892820997},{"type":"v1","addr":"172.21.15.158:6813","nonce":3892820997}]},"public_addr":"172.21.15.158:6809/3892820997","cluster_addr":"172.21.15.158:6811/3892820997","heartbeat_back_addr":"172.21.15.158:6815/3892820997","heartbeat_front_addr":"172.21.15.158:6813/3892820997","state":["exists","up"]},{"osd":5,"uuid":"d9c2e0c8-85f5-4ae2-bf3e-125c1be3a2db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3939495896},{"type":"v1","addr":"172.21.15.188:6809","nonce":3939495896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3939495896},{"type":"v1","addr":"172.21.15.188:6811","nonce":3939495896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3939495896},{"type":"v1","addr":"172.21.15.188:6815","nonce":3939495896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3939495896},{"type":"v1","addr":"172.21.15.188:6813","nonce":3939495896}]},"public_addr":"172.21.15.188:6809/3939495896","cluster_addr":"172.21.15.188:6811/3939495896","heartbeat_back_addr":"172.21.15.188:6815/3939495896","heartbeat_front_addr":"172.21.15.188:6813/3939495896","state":["exists","up"]},{"osd":6,"uuid":"4a523512-de6b-49f0-be81-00f2f00c1c10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":1206337767},{"type":"v1","addr":"172.21.15.12:6819","nonce":1206337767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":1206337767},{"type":"v1","addr":"172.21.15.12:6821","nonce":1206337767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":1206337767},{"type":"v1","addr":"172.21.15.12:6825","nonce":1206337767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":1206337767},{"type":"v1","addr":"172.21.15.12:6823","nonce":1206337767}]},"public_addr":"172.21.15.12:6819/1206337767","cluster_addr":"172.21.15.12:6821/1206337767","heartbeat_back_addr":"172.21.15.12:6825/1206337767","heartbeat_front_addr":"172.21.15.12:6823/1206337767","state":["exists","up"]},{"osd":7,"uuid":"363271a6-df93-44c9-b0e5-3d29768aa96c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":3283400032},{"type":"v1","addr":"172.21.15.158:6817","nonce":3283400032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":3283400032},{"type":"v1","addr":"172.21.15.158:6819","nonce":3283400032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":3283400032},{"type":"v1","addr":"172.21.15.158:6823","nonce":3283400032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":3283400032},{"type":"v1","addr":"172.21.15.158:6821","nonce":3283400032}]},"public_addr":"172.21.15.158:6817/3283400032","cluster_addr":"172.21.15.158:6819/3283400032","heartbeat_back_addr":"172.21.15.158:6823/3283400032","heartbeat_front_addr":"172.21.15.158:6821/3283400032","state":["exists","up"]},{"osd":8,"uuid":"9a089528-3e8e-45d3-9279-ddb54088b96a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2110170682},{"type":"v1","addr":"172.21.15.188:6817","nonce":2110170682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2110170682},{"type":"v1","addr":"172.21.15.188:6819","nonce":2110170682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":2110170682},{"type":"v1","addr":"172.21.15.188:6823","nonce":2110170682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2110170682},{"type":"v1","addr":"172.21.15.188:6821","nonce":2110170682}]},"public_addr":"172.21.15.188:6817/2110170682","cluster_addr":"172.21.15.188:6819/2110170682","heartbeat_back_addr":"172.21.15.188:6823/2110170682","heartbeat_front_addr":"172.21.15.188:6821/2110170682","state":["exists","up"]},{"osd":9,"uuid":"b12e2727-d649-4464-90e1-fafbc1754832","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1825928986},{"type":"v1","addr":"172.21.15.12:6827","nonce":1825928986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1825928986},{"type":"v1","addr":"172.21.15.12:6829","nonce":1825928986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":1825928986},{"type":"v1","addr":"172.21.15.12:6833","nonce":1825928986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1825928986},{"type":"v1","addr":"172.21.15.12:6831","nonce":1825928986}]},"public_addr":"172.21.15.12:6827/1825928986","cluster_addr":"172.21.15.12:6829/1825928986","heartbeat_back_addr":"172.21.15.12:6833/1825928986","heartbeat_front_addr":"172.21.15.12:6831/1825928986","state":["exists","up"]},{"osd":10,"uuid":"518b4cbf-9d90-4e9a-8ac9-b5cb9b3dee4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":3239052167},{"type":"v1","addr":"172.21.15.158:6825","nonce":3239052167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":3239052167},{"type":"v1","addr":"172.21.15.158:6827","nonce":3239052167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":3239052167},{"type":"v1","addr":"172.21.15.158:6831","nonce":3239052167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":3239052167},{"type":"v1","addr":"172.21.15.158:6829","nonce":3239052167}]},"public_addr":"172.21.15.158:6825/3239052167","cluster_addr":"172.21.15.158:6827/3239052167","heartbeat_back_addr":"172.21.15.158:6831/3239052167","heartbeat_front_addr":"172.21.15.158:6829/3239052167","state":["exists","up"]},{"osd":11,"uuid":"98aa3a0d-4b2f-46c8-9341-375d13794c3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1135773350},{"type":"v1","addr":"172.21.15.188:6825","nonce":1135773350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1135773350},{"type":"v1","addr":"172.21.15.188:6827","nonce":1135773350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1135773350},{"type":"v1","addr":"172.21.15.188:6831","nonce":1135773350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1135773350},{"type":"v1","addr":"172.21.15.188:6829","nonce":1135773350}]},"public_addr":"172.21.15.188:6825/1135773350","cluster_addr":"172.21.15.188:6827/1135773350","heartbeat_back_addr":"172.21.15.188:6831/1135773350","heartbeat_front_addr":"172.21.15.188:6829/1135773350","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:33:24.280879+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:33:47.859461+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:34:11.725283+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:34:35.707629+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:35:00.024197+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:35:24.499803+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:35:48.953870+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:36:13.642456+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:36:38.936457+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:37:04.021466+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:37:29.073178+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:37:54.863919+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.37","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:6801/4135274411":"2024-02-19T11:31:50.936928+0000","172.21.15.12:6800/4135274411":"2024-02-19T11:31:50.936928+0000","172.21.15.12:0/118428584":"2024-02-19T11:31:50.936928+0000","172.21.15.12:0/4276810915":"2024-02-19T11:31:50.936928+0000","172.21.15.12:0/3777545262":"2024-02-19T11:31:24.000830+0000","172.21.15.12:0/114707743":"2024-02-19T11:31:24.000830+0000","172.21.15.12:6800/3925204723":"2024-02-19T11:31:24.000830+0000","172.21.15.12:6801/3925204723":"2024-02-19T11:31:24.000830+0000","172.21.15.12:0/74881618":"2024-02-19T11:31:24.000830+0000","172.21.15.12:0/1614805375":"2024-02-19T11:31:50.936928+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-18T11:40:07.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.378+0000 7f43ccd8c640 1 -- 172.21.15.12:0/3185188805 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f43b40a27e0 msgr2=0x7f43b40a4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:07.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.378+0000 7f43ccd8c640 1 --2- 172.21.15.12:0/3185188805 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f43b40a27e0 0x7f43b40a4ca0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f43c8101130 tx=0x7f43b8008040 comp rx=0 tx=0).stop 2024-02-18T11:40:07.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.378+0000 7f43ccd8c640 1 -- 172.21.15.12:0/3185188805 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f43c8101c90 msgr2=0x7f43c819e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:07.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.378+0000 7f43ccd8c640 1 --2- 172.21.15.12:0/3185188805 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f43c8101c90 0x7f43c819e910 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f43c000cad0 tx=0x7f43c000cf90 comp rx=0 tx=0).stop 2024-02-18T11:40:07.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.378+0000 7f43ccd8c640 1 -- 172.21.15.12:0/3185188805 shutdown_connections 2024-02-18T11:40:07.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.378+0000 7f43ccd8c640 1 --2- 172.21.15.12:0/3185188805 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f43b40a27e0 0x7f43b40a4ca0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:07.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.378+0000 7f43ccd8c640 1 --2- 172.21.15.12:0/3185188805 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43c81012d0 0x7f43c8197910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:07.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.378+0000 7f43ccd8c640 1 --2- 172.21.15.12:0/3185188805 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f43c8101c90 0x7f43c819e910 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:07.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.378+0000 7f43ccd8c640 1 --2- 172.21.15.12:0/3185188805 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f43c81000e0 0x7f43c81973d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:07.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.378+0000 7f43ccd8c640 1 -- 172.21.15.12:0/3185188805 >> 172.21.15.12:0/3185188805 conn(0x7f43c80fb650 msgr2=0x7f43c80fd120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:07.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.378+0000 7f43ccd8c640 1 -- 172.21.15.12:0/3185188805 shutdown_connections 2024-02-18T11:40:07.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.378+0000 7f43ccd8c640 1 -- 172.21.15.12:0/3185188805 wait complete. 2024-02-18T11:40:07.384 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-18T11:34:15.281138+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-18T11:38:44.802134+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-18T11:38:46.081188+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-18T11:40:07.384 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-18T11:40:07.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.496+0000 7f7d26655640 1 -- 172.21.15.12:0/3793627545 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d200ff860 msgr2=0x7f7d200ffc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:07.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.496+0000 7f7d26655640 1 --2- 172.21.15.12:0/3793627545 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d200ff860 0x7f7d200ffc80 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7d10009900 tx=0x7f7d1002bc30 comp rx=0 tx=0).stop 2024-02-18T11:40:07.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.496+0000 7f7d26655640 1 -- 172.21.15.12:0/3793627545 shutdown_connections 2024-02-18T11:40:07.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.496+0000 7f7d26655640 1 --2- 172.21.15.12:0/3793627545 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d20100a50 0x7f7d20100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:07.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.496+0000 7f7d26655640 1 --2- 172.21.15.12:0/3793627545 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7d20101410 0x7f7d20107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:07.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.496+0000 7f7d26655640 1 --2- 172.21.15.12:0/3793627545 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d200ff860 0x7f7d200ffc80 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:07.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.496+0000 7f7d26655640 1 -- 172.21.15.12:0/3793627545 >> 172.21.15.12:0/3793627545 conn(0x7f7d200fadf0 msgr2=0x7f7d200fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:07.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.496+0000 7f7d26655640 1 -- 172.21.15.12:0/3793627545 shutdown_connections 2024-02-18T11:40:07.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.496+0000 7f7d26655640 1 -- 172.21.15.12:0/3793627545 wait complete. 2024-02-18T11:40:07.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.496+0000 7f7d26655640 1 Processor -- start 2024-02-18T11:40:07.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.496+0000 7f7d26655640 1 -- start start 2024-02-18T11:40:07.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.497+0000 7f7d26655640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d200ff860 0x7f7d201972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:07.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.497+0000 7f7d26655640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7d20100a50 0x7f7d20197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:07.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.497+0000 7f7d26655640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d20101410 0x7f7d2019e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:07.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.497+0000 7f7d26655640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2019edd0 con 0x7f7d200ff860 2024-02-18T11:40:07.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.497+0000 7f7d26655640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2019ef40 con 0x7f7d20100a50 2024-02-18T11:40:07.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.497+0000 7f7d26655640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2019f100 con 0x7f7d20101410 2024-02-18T11:40:07.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.497+0000 7f7d1ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d200ff860 0x7f7d201972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:07.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.497+0000 7f7d1ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d200ff860 0x7f7d201972e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39920/0 (socket says 172.21.15.12:39920) 2024-02-18T11:40:07.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.497+0000 7f7d1ffff640 1 -- 172.21.15.12:0/2348484008 learned_addr learned my addr 172.21.15.12:0/2348484008 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:40:07.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.497+0000 7f7d1f7fe640 1 --2- 172.21.15.12:0/2348484008 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7d20100a50 0x7f7d20197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:07.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.497+0000 7f7d1ffff640 1 -- 172.21.15.12:0/2348484008 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d20101410 msgr2=0x7f7d2019e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:07.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.497+0000 7f7d24bf2640 1 --2- 172.21.15.12:0/2348484008 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d20101410 0x7f7d2019e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:07.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.497+0000 7f7d1ffff640 1 --2- 172.21.15.12:0/2348484008 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d20101410 0x7f7d2019e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:07.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.497+0000 7f7d1ffff640 1 -- 172.21.15.12:0/2348484008 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7d20100a50 msgr2=0x7f7d20197820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:07.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.497+0000 7f7d1ffff640 1 --2- 172.21.15.12:0/2348484008 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7d20100a50 0x7f7d20197820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:07.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.497+0000 7f7d1ffff640 1 -- 172.21.15.12:0/2348484008 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d10009560 con 0x7f7d200ff860 2024-02-18T11:40:07.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.497+0000 7f7d1f7fe640 1 --2- 172.21.15.12:0/2348484008 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7d20100a50 0x7f7d20197820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:40:07.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.497+0000 7f7d24bf2640 1 --2- 172.21.15.12:0/2348484008 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d20101410 0x7f7d2019e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:40:07.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.497+0000 7f7d1ffff640 1 --2- 172.21.15.12:0/2348484008 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d200ff860 0x7f7d201972e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7d10000c00 tx=0x7f7d10004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:07.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.497+0000 7f7d1d7fa640 1 -- 172.21.15.12:0/2348484008 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d10004df0 con 0x7f7d200ff860 2024-02-18T11:40:07.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.497+0000 7f7d1d7fa640 1 -- 172.21.15.12:0/2348484008 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7d10005430 con 0x7f7d200ff860 2024-02-18T11:40:07.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.497+0000 7f7d26655640 1 -- 172.21.15.12:0/2348484008 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d2019f380 con 0x7f7d200ff860 2024-02-18T11:40:07.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.497+0000 7f7d1d7fa640 1 -- 172.21.15.12:0/2348484008 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1003e450 con 0x7f7d200ff860 2024-02-18T11:40:07.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.497+0000 7f7d26655640 1 -- 172.21.15.12:0/2348484008 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d2019f890 con 0x7f7d200ff860 2024-02-18T11:40:07.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.498+0000 7f7d26655640 1 -- 172.21.15.12:0/2348484008 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d20106c50 con 0x7f7d200ff860 2024-02-18T11:40:07.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.499+0000 7f7d1d7fa640 1 -- 172.21.15.12:0/2348484008 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 137296+0+0 (secure 0 0 0) 0x7f7d10003680 con 0x7f7d200ff860 2024-02-18T11:40:07.495 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:40:07 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/3262075191' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-18T11:40:07.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.500+0000 7f7d1d7fa640 1 --2- 172.21.15.12:0/2348484008 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f7d080a27e0 0x7f7d080a4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:07.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.500+0000 7f7d1f7fe640 1 --2- 172.21.15.12:0/2348484008 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f7d080a27e0 0x7f7d080a4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:07.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.500+0000 7f7d1d7fa640 1 -- 172.21.15.12:0/2348484008 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7d100f0b50 con 0x7f7d200ff860 2024-02-18T11:40:07.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.500+0000 7f7d1f7fe640 1 --2- 172.21.15.12:0/2348484008 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f7d080a27e0 0x7f7d080a4ca0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7d140058c0 tx=0x7f7d14005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:07.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.504+0000 7f7d1d7fa640 1 -- 172.21.15.12:0/2348484008 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7d100b98a0 con 0x7f7d200ff860 2024-02-18T11:40:07.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.685+0000 7f7d26655640 1 -- 172.21.15.12:0/2348484008 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7d2002ce10 con 0x7f7d200ff860 2024-02-18T11:40:07.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.685+0000 7f7d1d7fa640 1 -- 172.21.15.12:0/2348484008 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7d100b9070 con 0x7f7d200ff860 2024-02-18T11:40:07.680 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-02-18T11:40:07.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.687+0000 7f7d26655640 1 -- 172.21.15.12:0/2348484008 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f7d080a27e0 msgr2=0x7f7d080a4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:07.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.687+0000 7f7d26655640 1 --2- 172.21.15.12:0/2348484008 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f7d080a27e0 0x7f7d080a4ca0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7d140058c0 tx=0x7f7d14005850 comp rx=0 tx=0).stop 2024-02-18T11:40:07.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.687+0000 7f7d26655640 1 -- 172.21.15.12:0/2348484008 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d200ff860 msgr2=0x7f7d201972e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:07.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.687+0000 7f7d26655640 1 --2- 172.21.15.12:0/2348484008 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d200ff860 0x7f7d201972e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7d10000c00 tx=0x7f7d10004270 comp rx=0 tx=0).stop 2024-02-18T11:40:07.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.687+0000 7f7d26655640 1 -- 172.21.15.12:0/2348484008 shutdown_connections 2024-02-18T11:40:07.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.687+0000 7f7d26655640 1 --2- 172.21.15.12:0/2348484008 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f7d080a27e0 0x7f7d080a4ca0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:07.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.687+0000 7f7d26655640 1 --2- 172.21.15.12:0/2348484008 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d20101410 0x7f7d2019e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:07.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.687+0000 7f7d26655640 1 --2- 172.21.15.12:0/2348484008 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7d20100a50 0x7f7d20197820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:07.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.687+0000 7f7d26655640 1 --2- 172.21.15.12:0/2348484008 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d200ff860 0x7f7d201972e0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:07.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.687+0000 7f7d26655640 1 -- 172.21.15.12:0/2348484008 >> 172.21.15.12:0/2348484008 conn(0x7f7d200fadf0 msgr2=0x7f7d20103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:07.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.687+0000 7f7d26655640 1 -- 172.21.15.12:0/2348484008 shutdown_connections 2024-02-18T11:40:07.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.687+0000 7f7d26655640 1 -- 172.21.15.12:0/2348484008 wait complete. 2024-02-18T11:40:07.693 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-18T11:40:07.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.804+0000 7fe98b3e2640 1 -- 172.21.15.12:0/3968130956 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe984104580 msgr2=0x7fe984106970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:07.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.804+0000 7fe98b3e2640 1 --2- 172.21.15.12:0/3968130956 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe984104580 0x7fe984106970 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe974009900 tx=0x7fe97402bc30 comp rx=0 tx=0).stop 2024-02-18T11:40:07.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.804+0000 7fe98b3e2640 1 -- 172.21.15.12:0/3968130956 shutdown_connections 2024-02-18T11:40:07.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.804+0000 7fe98b3e2640 1 --2- 172.21.15.12:0/3968130956 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe984106eb0 0x7fe984109300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:07.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.804+0000 7fe98b3e2640 1 --2- 172.21.15.12:0/3968130956 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe984104580 0x7fe984106970 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:07.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.804+0000 7fe98b3e2640 1 --2- 172.21.15.12:0/3968130956 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe984101c50 0x7fe984104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:07.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.804+0000 7fe98b3e2640 1 -- 172.21.15.12:0/3968130956 >> 172.21.15.12:0/3968130956 conn(0x7fe9840fb660 msgr2=0x7fe9840fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:07.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.804+0000 7fe98b3e2640 1 -- 172.21.15.12:0/3968130956 shutdown_connections 2024-02-18T11:40:07.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.804+0000 7fe98b3e2640 1 -- 172.21.15.12:0/3968130956 wait complete. 2024-02-18T11:40:07.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.805+0000 7fe98b3e2640 1 Processor -- start 2024-02-18T11:40:07.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.805+0000 7fe98b3e2640 1 -- start start 2024-02-18T11:40:07.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.805+0000 7fe98b3e2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe984101c50 0x7fe9841972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:07.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.805+0000 7fe98b3e2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe984104580 0x7fe9841977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:07.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.805+0000 7fe98b3e2640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe984106eb0 0x7fe98419e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:07.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.805+0000 7fe98b3e2640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe98419eda0 con 0x7fe984104580 2024-02-18T11:40:07.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.805+0000 7fe98b3e2640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe98419ef10 con 0x7fe984106eb0 2024-02-18T11:40:07.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.805+0000 7fe98b3e2640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe98419f0d0 con 0x7fe984101c50 2024-02-18T11:40:07.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.805+0000 7fe98897d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe984104580 0x7fe9841977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:07.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.805+0000 7fe98897d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe984104580 0x7fe9841977f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39930/0 (socket says 172.21.15.12:39930) 2024-02-18T11:40:07.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.805+0000 7fe98897d640 1 -- 172.21.15.12:0/544707630 learned_addr learned my addr 172.21.15.12:0/544707630 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:40:07.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.805+0000 7fe98917e640 1 --2- 172.21.15.12:0/544707630 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe984101c50 0x7fe9841972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:07.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.805+0000 7fe98997f640 1 --2- 172.21.15.12:0/544707630 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe984106eb0 0x7fe98419e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:07.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.805+0000 7fe98897d640 1 -- 172.21.15.12:0/544707630 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe984101c50 msgr2=0x7fe9841972b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:07.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.805+0000 7fe98897d640 1 --2- 172.21.15.12:0/544707630 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe984101c50 0x7fe9841972b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:07.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.805+0000 7fe98897d640 1 -- 172.21.15.12:0/544707630 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe984106eb0 msgr2=0x7fe98419e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:07.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.805+0000 7fe98897d640 1 --2- 172.21.15.12:0/544707630 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe984106eb0 0x7fe98419e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:07.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.805+0000 7fe98897d640 1 -- 172.21.15.12:0/544707630 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe974009560 con 0x7fe984104580 2024-02-18T11:40:07.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.805+0000 7fe98997f640 1 --2- 172.21.15.12:0/544707630 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe984106eb0 0x7fe98419e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:40:07.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.805+0000 7fe98897d640 1 --2- 172.21.15.12:0/544707630 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe984104580 0x7fe9841977f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe974036040 tx=0x7fe9740042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:07.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.806+0000 7fe97a7fc640 1 -- 172.21.15.12:0/544707630 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9740044e0 con 0x7fe984104580 2024-02-18T11:40:07.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.806+0000 7fe97a7fc640 1 -- 172.21.15.12:0/544707630 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe9740054e0 con 0x7fe984104580 2024-02-18T11:40:07.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.806+0000 7fe98b3e2640 1 -- 172.21.15.12:0/544707630 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe98419f350 con 0x7fe984104580 2024-02-18T11:40:07.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.806+0000 7fe97a7fc640 1 -- 172.21.15.12:0/544707630 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe97403e450 con 0x7fe984104580 2024-02-18T11:40:07.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.806+0000 7fe98b3e2640 1 -- 172.21.15.12:0/544707630 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe98419f8e0 con 0x7fe984104580 2024-02-18T11:40:07.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.807+0000 7fe98b3e2640 1 -- 172.21.15.12:0/544707630 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9840fd150 con 0x7fe984104580 2024-02-18T11:40:07.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.808+0000 7fe97a7fc640 1 -- 172.21.15.12:0/544707630 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 137296+0+0 (secure 0 0 0) 0x7fe974003710 con 0x7fe984104580 2024-02-18T11:40:07.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.808+0000 7fe97a7fc640 1 --2- 172.21.15.12:0/544707630 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fe9700a27e0 0x7fe9700a4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:07.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.809+0000 7fe98917e640 1 --2- 172.21.15.12:0/544707630 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fe9700a27e0 0x7fe9700a4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:07.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.809+0000 7fe97a7fc640 1 -- 172.21.15.12:0/544707630 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe9740f0aa0 con 0x7fe984104580 2024-02-18T11:40:07.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.809+0000 7fe98917e640 1 --2- 172.21.15.12:0/544707630 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fe9700a27e0 0x7fe9700a4ca0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe980005e10 tx=0x7fe980005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:07.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.813+0000 7fe97a7fc640 1 -- 172.21.15.12:0/544707630 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe9740b97f0 con 0x7fe984104580 2024-02-18T11:40:07.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.996+0000 7fe98b3e2640 1 -- 172.21.15.12:0/544707630 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe98410ce70 con 0x7fe984104580 2024-02-18T11:40:07.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.996+0000 7fe97a7fc640 1 -- 172.21.15.12:0/544707630 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe9740bceb0 con 0x7fe984104580 2024-02-18T11:40:07.990 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-02-18T11:40:07.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.998+0000 7fe98b3e2640 1 -- 172.21.15.12:0/544707630 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fe9700a27e0 msgr2=0x7fe9700a4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:07.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.998+0000 7fe98b3e2640 1 --2- 172.21.15.12:0/544707630 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fe9700a27e0 0x7fe9700a4ca0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe980005e10 tx=0x7fe980005da0 comp rx=0 tx=0).stop 2024-02-18T11:40:07.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.998+0000 7fe98b3e2640 1 -- 172.21.15.12:0/544707630 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe984104580 msgr2=0x7fe9841977f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:07.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.998+0000 7fe98b3e2640 1 --2- 172.21.15.12:0/544707630 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe984104580 0x7fe9841977f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe974036040 tx=0x7fe9740042f0 comp rx=0 tx=0).stop 2024-02-18T11:40:07.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.999+0000 7fe98b3e2640 1 -- 172.21.15.12:0/544707630 shutdown_connections 2024-02-18T11:40:07.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.999+0000 7fe98b3e2640 1 --2- 172.21.15.12:0/544707630 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fe9700a27e0 0x7fe9700a4ca0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:07.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.999+0000 7fe98b3e2640 1 --2- 172.21.15.12:0/544707630 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe984106eb0 0x7fe98419e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:07.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.999+0000 7fe98b3e2640 1 --2- 172.21.15.12:0/544707630 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe984104580 0x7fe9841977f0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:07.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.999+0000 7fe98b3e2640 1 --2- 172.21.15.12:0/544707630 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe984101c50 0x7fe9841972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:07.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.999+0000 7fe98b3e2640 1 -- 172.21.15.12:0/544707630 >> 172.21.15.12:0/544707630 conn(0x7fe9840fb660 msgr2=0x7fe984102590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:07.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.999+0000 7fe98b3e2640 1 -- 172.21.15.12:0/544707630 shutdown_connections 2024-02-18T11:40:07.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:07.999+0000 7fe98b3e2640 1 -- 172.21.15.12:0/544707630 wait complete. 2024-02-18T11:40:08.004 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-18T11:40:08.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.115+0000 7f8ce870c640 1 -- 172.21.15.12:0/545480602 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ce0106eb0 msgr2=0x7f8ce0109300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:08.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.115+0000 7f8ce870c640 1 --2- 172.21.15.12:0/545480602 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ce0106eb0 0x7f8ce0109300 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8cd8009ca0 tx=0x7f8cd802c0f0 comp rx=0 tx=0).stop 2024-02-18T11:40:08.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.115+0000 7f8ce870c640 1 -- 172.21.15.12:0/545480602 shutdown_connections 2024-02-18T11:40:08.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.115+0000 7f8ce870c640 1 --2- 172.21.15.12:0/545480602 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ce0104580 0x7f8ce0106970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:08.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.115+0000 7f8ce870c640 1 --2- 172.21.15.12:0/545480602 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ce0106eb0 0x7f8ce0109300 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:08.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.115+0000 7f8ce870c640 1 --2- 172.21.15.12:0/545480602 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8ce0101c50 0x7f8ce0104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:08.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.115+0000 7f8ce870c640 1 -- 172.21.15.12:0/545480602 >> 172.21.15.12:0/545480602 conn(0x7f8ce00fb660 msgr2=0x7f8ce00fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:08.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.115+0000 7f8ce870c640 1 -- 172.21.15.12:0/545480602 shutdown_connections 2024-02-18T11:40:08.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.115+0000 7f8ce870c640 1 -- 172.21.15.12:0/545480602 wait complete. 2024-02-18T11:40:08.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.116+0000 7f8ce870c640 1 Processor -- start 2024-02-18T11:40:08.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.116+0000 7f8ce870c640 1 -- start start 2024-02-18T11:40:08.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.116+0000 7f8ce870c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ce0101c50 0x7f8ce01015c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:08.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.116+0000 7f8ce870c640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8ce0104580 0x7f8ce00ffc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:08.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.116+0000 7f8ce870c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ce0106eb0 0x7f8ce0100140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:08.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.116+0000 7f8ce870c640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce01008a0 con 0x7f8ce0106eb0 2024-02-18T11:40:08.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.116+0000 7f8ce870c640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce0100a10 con 0x7f8ce0104580 2024-02-18T11:40:08.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.116+0000 7f8ce870c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce01a13a0 con 0x7f8ce0101c50 2024-02-18T11:40:08.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.116+0000 7f8ce6ca9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ce0106eb0 0x7f8ce0100140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:08.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.116+0000 7f8ce6ca9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ce0106eb0 0x7f8ce0100140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39948/0 (socket says 172.21.15.12:39948) 2024-02-18T11:40:08.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.116+0000 7f8ce6ca9640 1 -- 172.21.15.12:0/3203441635 learned_addr learned my addr 172.21.15.12:0/3203441635 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:40:08.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.116+0000 7f8ce64a8640 1 --2- 172.21.15.12:0/3203441635 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ce0101c50 0x7f8ce01015c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:08.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.116+0000 7f8ce5ca7640 1 --2- 172.21.15.12:0/3203441635 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8ce0104580 0x7f8ce00ffc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:08.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.116+0000 7f8ce6ca9640 1 -- 172.21.15.12:0/3203441635 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ce0101c50 msgr2=0x7f8ce01015c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:08.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.116+0000 7f8ce6ca9640 1 --2- 172.21.15.12:0/3203441635 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ce0101c50 0x7f8ce01015c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:08.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.116+0000 7f8ce6ca9640 1 -- 172.21.15.12:0/3203441635 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8ce0104580 msgr2=0x7f8ce00ffc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:08.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.116+0000 7f8ce6ca9640 1 --2- 172.21.15.12:0/3203441635 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8ce0104580 0x7f8ce00ffc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:08.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.116+0000 7f8ce6ca9640 1 -- 172.21.15.12:0/3203441635 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cd8009900 con 0x7f8ce0106eb0 2024-02-18T11:40:08.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.116+0000 7f8ce5ca7640 1 --2- 172.21.15.12:0/3203441635 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8ce0104580 0x7f8ce00ffc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:40:08.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.117+0000 7f8ce6ca9640 1 --2- 172.21.15.12:0/3203441635 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ce0106eb0 0x7f8ce0100140 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8cd8034d20 tx=0x7f8cd8034d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:08.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.117+0000 7f8cd77fe640 1 -- 172.21.15.12:0/3203441635 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd8003970 con 0x7f8ce0106eb0 2024-02-18T11:40:08.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.117+0000 7f8ce870c640 1 -- 172.21.15.12:0/3203441635 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ce01a15f0 con 0x7f8ce0106eb0 2024-02-18T11:40:08.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.117+0000 7f8cd77fe640 1 -- 172.21.15.12:0/3203441635 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8cd8004040 con 0x7f8ce0106eb0 2024-02-18T11:40:08.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.117+0000 7f8cd77fe640 1 -- 172.21.15.12:0/3203441635 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd80406b0 con 0x7f8ce0106eb0 2024-02-18T11:40:08.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.117+0000 7f8ce870c640 1 -- 172.21.15.12:0/3203441635 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ce01a1b20 con 0x7f8ce0106eb0 2024-02-18T11:40:08.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.118+0000 7f8ce870c640 1 -- 172.21.15.12:0/3203441635 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ce00fd150 con 0x7f8ce0106eb0 2024-02-18T11:40:08.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.119+0000 7f8cd77fe640 1 -- 172.21.15.12:0/3203441635 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 137296+0+0 (secure 0 0 0) 0x7f8cd800a040 con 0x7f8ce0106eb0 2024-02-18T11:40:08.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.119+0000 7f8cd77fe640 1 --2- 172.21.15.12:0/3203441635 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f8ccc0a27e0 0x7f8ccc0a4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:08.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.119+0000 7f8ce64a8640 1 --2- 172.21.15.12:0/3203441635 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f8ccc0a27e0 0x7f8ccc0a4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:08.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.119+0000 7f8cd77fe640 1 -- 172.21.15.12:0/3203441635 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f8cd80f1bc0 con 0x7f8ce0106eb0 2024-02-18T11:40:08.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.120+0000 7f8ce64a8640 1 --2- 172.21.15.12:0/3203441635 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f8ccc0a27e0 0x7f8ccc0a4ca0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc009990 tx=0x7f8cdc009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:08.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.123+0000 7f8cd77fe640 1 -- 172.21.15.12:0/3203441635 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8cd80ba9a0 con 0x7f8ce0106eb0 2024-02-18T11:40:08.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.303+0000 7f8ce870c640 1 -- 172.21.15.12:0/3203441635 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8ce010ccc0 con 0x7f8ce0106eb0 2024-02-18T11:40:08.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.303+0000 7f8cd77fe640 1 -- 172.21.15.12:0/3203441635 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8cd80be060 con 0x7f8ce0106eb0 2024-02-18T11:40:08.297 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2024-02-18T11:40:08.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.305+0000 7f8ce870c640 1 -- 172.21.15.12:0/3203441635 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f8ccc0a27e0 msgr2=0x7f8ccc0a4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:08.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.305+0000 7f8ce870c640 1 --2- 172.21.15.12:0/3203441635 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f8ccc0a27e0 0x7f8ccc0a4ca0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc009990 tx=0x7f8cdc009110 comp rx=0 tx=0).stop 2024-02-18T11:40:08.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.305+0000 7f8ce870c640 1 -- 172.21.15.12:0/3203441635 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ce0106eb0 msgr2=0x7f8ce0100140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:08.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.305+0000 7f8ce870c640 1 --2- 172.21.15.12:0/3203441635 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ce0106eb0 0x7f8ce0100140 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8cd8034d20 tx=0x7f8cd8034d50 comp rx=0 tx=0).stop 2024-02-18T11:40:08.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.305+0000 7f8ce870c640 1 -- 172.21.15.12:0/3203441635 shutdown_connections 2024-02-18T11:40:08.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.305+0000 7f8ce870c640 1 --2- 172.21.15.12:0/3203441635 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f8ccc0a27e0 0x7f8ccc0a4ca0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:08.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.305+0000 7f8ce870c640 1 --2- 172.21.15.12:0/3203441635 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ce0106eb0 0x7f8ce0100140 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:08.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.305+0000 7f8ce870c640 1 --2- 172.21.15.12:0/3203441635 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8ce0104580 0x7f8ce00ffc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:08.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.305+0000 7f8ce870c640 1 --2- 172.21.15.12:0/3203441635 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ce0101c50 0x7f8ce01015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:08.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.305+0000 7f8ce870c640 1 -- 172.21.15.12:0/3203441635 >> 172.21.15.12:0/3203441635 conn(0x7f8ce00fb660 msgr2=0x7f8ce0102510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:08.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.305+0000 7f8ce870c640 1 -- 172.21.15.12:0/3203441635 shutdown_connections 2024-02-18T11:40:08.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.305+0000 7f8ce870c640 1 -- 172.21.15.12:0/3203441635 wait complete. 2024-02-18T11:40:08.309 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:40:08 smithi012 ceph-mon[32460]: pgmap v327: 97 pgs: 97 active+clean; 639 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-02-18T11:40:08.309 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:40:08 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/3185188805' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-18T11:40:08.309 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:40:08 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/2348484008' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-18T11:40:08.309 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:40:08 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/544707630' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-18T11:40:08.310 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-18T11:40:08.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:40:08 smithi188 ceph-mon[36925]: pgmap v327: 97 pgs: 97 active+clean; 639 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-02-18T11:40:08.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:40:08 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/3185188805' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-18T11:40:08.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:40:08 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/2348484008' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-18T11:40:08.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:40:08 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/544707630' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-18T11:40:08.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.419+0000 7f2237165640 1 -- 172.21.15.12:0/2381055680 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22301000e0 msgr2=0x7f2230100500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:08.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.419+0000 7f2237165640 1 --2- 172.21.15.12:0/2381055680 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22301000e0 0x7f2230100500 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f2224009900 tx=0x7f222402bc30 comp rx=0 tx=0).stop 2024-02-18T11:40:08.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.419+0000 7f2237165640 1 -- 172.21.15.12:0/2381055680 shutdown_connections 2024-02-18T11:40:08.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.419+0000 7f2237165640 1 --2- 172.21.15.12:0/2381055680 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2230101c90 0x7f2230108520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:08.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.419+0000 7f2237165640 1 --2- 172.21.15.12:0/2381055680 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f22301012d0 0x7f2230101750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:08.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.419+0000 7f2237165640 1 --2- 172.21.15.12:0/2381055680 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22301000e0 0x7f2230100500 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:08.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.419+0000 7f2237165640 1 -- 172.21.15.12:0/2381055680 >> 172.21.15.12:0/2381055680 conn(0x7f22300fb670 msgr2=0x7f22300fda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:08.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.419+0000 7f2237165640 1 -- 172.21.15.12:0/2381055680 shutdown_connections 2024-02-18T11:40:08.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.419+0000 7f2237165640 1 -- 172.21.15.12:0/2381055680 wait complete. 2024-02-18T11:40:08.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.419+0000 7f2237165640 1 Processor -- start 2024-02-18T11:40:08.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.419+0000 7f2237165640 1 -- start start 2024-02-18T11:40:08.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.420+0000 7f2237165640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f22301000e0 0x7f2230197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:08.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.420+0000 7f2237165640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22301012d0 0x7f2230197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:08.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.420+0000 7f2237165640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2230101c90 0x7f223019e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:08.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.420+0000 7f2237165640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f223019ecf0 con 0x7f22301012d0 2024-02-18T11:40:08.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.420+0000 7f2237165640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f223019ee60 con 0x7f22301000e0 2024-02-18T11:40:08.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.420+0000 7f2237165640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f223019f020 con 0x7f2230101c90 2024-02-18T11:40:08.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.420+0000 7f222ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22301012d0 0x7f2230197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:08.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.420+0000 7f222ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22301012d0 0x7f2230197740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39968/0 (socket says 172.21.15.12:39968) 2024-02-18T11:40:08.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.420+0000 7f222ffff640 1 -- 172.21.15.12:0/3687405523 learned_addr learned my addr 172.21.15.12:0/3687405523 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:40:08.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.420+0000 7f2234f01640 1 --2- 172.21.15.12:0/3687405523 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f22301000e0 0x7f2230197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:08.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.420+0000 7f222ffff640 1 -- 172.21.15.12:0/3687405523 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2230101c90 msgr2=0x7f223019e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:08.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.420+0000 7f2235702640 1 --2- 172.21.15.12:0/3687405523 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2230101c90 0x7f223019e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:08.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.420+0000 7f222ffff640 1 --2- 172.21.15.12:0/3687405523 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2230101c90 0x7f223019e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:08.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.420+0000 7f222ffff640 1 -- 172.21.15.12:0/3687405523 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f22301000e0 msgr2=0x7f2230197200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:08.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.420+0000 7f222ffff640 1 --2- 172.21.15.12:0/3687405523 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f22301000e0 0x7f2230197200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:08.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.420+0000 7f222ffff640 1 -- 172.21.15.12:0/3687405523 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2224009560 con 0x7f22301012d0 2024-02-18T11:40:08.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.420+0000 7f2234f01640 1 --2- 172.21.15.12:0/3687405523 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f22301000e0 0x7f2230197200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:40:08.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.420+0000 7f2235702640 1 --2- 172.21.15.12:0/3687405523 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2230101c90 0x7f223019e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:40:08.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.420+0000 7f222ffff640 1 --2- 172.21.15.12:0/3687405523 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22301012d0 0x7f2230197740 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f221c00c920 tx=0x7f221c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:08.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.420+0000 7f222dffb640 1 -- 172.21.15.12:0/3687405523 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f221c007c70 con 0x7f22301012d0 2024-02-18T11:40:08.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.420+0000 7f222dffb640 1 -- 172.21.15.12:0/3687405523 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f221c007dd0 con 0x7f22301012d0 2024-02-18T11:40:08.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.420+0000 7f2237165640 1 -- 172.21.15.12:0/3687405523 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f223019f300 con 0x7f22301012d0 2024-02-18T11:40:08.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.420+0000 7f222dffb640 1 -- 172.21.15.12:0/3687405523 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f221c018860 con 0x7f22301012d0 2024-02-18T11:40:08.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.421+0000 7f2237165640 1 -- 172.21.15.12:0/3687405523 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f223019f810 con 0x7f22301012d0 2024-02-18T11:40:08.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.422+0000 7f2237165640 1 -- 172.21.15.12:0/3687405523 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f223010baf0 con 0x7f22301012d0 2024-02-18T11:40:08.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.423+0000 7f222dffb640 1 -- 172.21.15.12:0/3687405523 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 137296+0+0 (secure 0 0 0) 0x7f221c00fac0 con 0x7f22301012d0 2024-02-18T11:40:08.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.423+0000 7f222dffb640 1 --2- 172.21.15.12:0/3687405523 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f22200a27e0 0x7f22200a4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:08.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.423+0000 7f2234f01640 1 --2- 172.21.15.12:0/3687405523 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f22200a27e0 0x7f22200a4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:08.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.423+0000 7f222dffb640 1 -- 172.21.15.12:0/3687405523 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f221c0ce890 con 0x7f22301012d0 2024-02-18T11:40:08.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.424+0000 7f2234f01640 1 --2- 172.21.15.12:0/3687405523 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f22200a27e0 0x7f22200a4ca0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2224006010 tx=0x7f2224005be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:08.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.427+0000 7f222dffb640 1 -- 172.21.15.12:0/3687405523 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f221c0974d0 con 0x7f22301012d0 2024-02-18T11:40:08.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:40:08 smithi158 ceph-mon[36626]: pgmap v327: 97 pgs: 97 active+clean; 639 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-02-18T11:40:08.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:40:08 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/3185188805' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-18T11:40:08.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:40:08 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/2348484008' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-18T11:40:08.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:40:08 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/544707630' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-18T11:40:08.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.646+0000 7f2237165640 1 -- 172.21.15.12:0/3687405523 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f223019fa90 con 0x7f22301012d0 2024-02-18T11:40:08.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.647+0000 7f222dffb640 1 -- 172.21.15.12:0/3687405523 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11349 (secure 0 0 0) 0x7f221c09ab90 con 0x7f22301012d0 2024-02-18T11:40:08.641 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:40:08.641 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1788335302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1788335302},{"type":"v1","addr":"172.21.15.12:6835","nonce":1788335302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/2144692455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":2144692455},{"type":"v1","addr":"172.21.15.12:6837","nonce":2144692455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3569056036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3569056036},{"type":"v1","addr":"172.21.15.12:6839","nonce":3569056036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14622,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/1973234486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":1973234486},{"type":"v1","addr":"172.21.15.12:6841","nonce":1973234486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1207202465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1207202465},{"type":"v1","addr":"172.21.15.188:6833","nonce":1207202465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6833/2636348323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6832","nonce":2636348323},{"type":"v1","addr":"172.21.15.158:6833","nonce":2636348323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-18T11:38:46.888173+0000","modified":"2024-02-18T11:38:51.531552+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24524,"mds_1":24508,"mds_2":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.158:6835/3673253642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":3673253642},{"type":"v1","addr":"172.21.15.158:6835","nonce":3673253642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/806709644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":806709644},{"type":"v1","addr":"172.21.15.188:6835","nonce":806709644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.158:6837/813251825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6836","nonce":813251825},{"type":"v1","addr":"172.21.15.158:6837","nonce":813251825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.188:6837/1036630143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1036630143},{"type":"v1","addr":"172.21.15.188:6837","nonce":1036630143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.188:6839/1342671190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":1342671190},{"type":"v1","addr":"172.21.15.188:6839","nonce":1342671190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.158:6839/1127200849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6838","nonce":1127200849},{"type":"v1","addr":"172.21.15.158:6839","nonce":1127200849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-18T11:40:08.641 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 20 2024-02-18T11:40:08.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.649+0000 7f2237165640 1 -- 172.21.15.12:0/3687405523 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f22200a27e0 msgr2=0x7f22200a4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:08.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.649+0000 7f2237165640 1 --2- 172.21.15.12:0/3687405523 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f22200a27e0 0x7f22200a4ca0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2224006010 tx=0x7f2224005be0 comp rx=0 tx=0).stop 2024-02-18T11:40:08.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.649+0000 7f2237165640 1 -- 172.21.15.12:0/3687405523 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22301012d0 msgr2=0x7f2230197740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:08.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.649+0000 7f2237165640 1 --2- 172.21.15.12:0/3687405523 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22301012d0 0x7f2230197740 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f221c00c920 tx=0x7f221c00cde0 comp rx=0 tx=0).stop 2024-02-18T11:40:08.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.649+0000 7f2237165640 1 -- 172.21.15.12:0/3687405523 shutdown_connections 2024-02-18T11:40:08.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.649+0000 7f2237165640 1 --2- 172.21.15.12:0/3687405523 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f22200a27e0 0x7f22200a4ca0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:08.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.649+0000 7f2237165640 1 --2- 172.21.15.12:0/3687405523 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2230101c90 0x7f223019e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:08.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.649+0000 7f2237165640 1 --2- 172.21.15.12:0/3687405523 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22301012d0 0x7f2230197740 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:08.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.649+0000 7f2237165640 1 --2- 172.21.15.12:0/3687405523 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f22301000e0 0x7f2230197200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:08.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.649+0000 7f2237165640 1 -- 172.21.15.12:0/3687405523 >> 172.21.15.12:0/3687405523 conn(0x7f22300fb670 msgr2=0x7f22300fd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:08.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.649+0000 7f2237165640 1 -- 172.21.15.12:0/3687405523 shutdown_connections 2024-02-18T11:40:08.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.649+0000 7f2237165640 1 -- 172.21.15.12:0/3687405523 wait complete. 2024-02-18T11:40:08.654 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-18T11:40:08.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.763+0000 7f2b94abb640 1 -- 172.21.15.12:0/732562515 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b901023e0 msgr2=0x7f2b90102860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:08.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.763+0000 7f2b94abb640 1 --2- 172.21.15.12:0/732562515 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b901023e0 0x7f2b90102860 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f2b84009900 tx=0x7f2b8402bc30 comp rx=0 tx=0).stop 2024-02-18T11:40:08.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.763+0000 7f2b94abb640 1 -- 172.21.15.12:0/732562515 shutdown_connections 2024-02-18T11:40:08.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.763+0000 7f2b94abb640 1 --2- 172.21.15.12:0/732562515 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2b90102da0 0x7f2b90109630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:08.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.763+0000 7f2b94abb640 1 --2- 172.21.15.12:0/732562515 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b901023e0 0x7f2b90102860 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:08.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.763+0000 7f2b94abb640 1 --2- 172.21.15.12:0/732562515 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b901011f0 0x7f2b90101610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:08.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.763+0000 7f2b94abb640 1 -- 172.21.15.12:0/732562515 >> 172.21.15.12:0/732562515 conn(0x7f2b900fc760 msgr2=0x7f2b900feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:08.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.763+0000 7f2b94abb640 1 -- 172.21.15.12:0/732562515 shutdown_connections 2024-02-18T11:40:08.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.763+0000 7f2b94abb640 1 -- 172.21.15.12:0/732562515 wait complete. 2024-02-18T11:40:08.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.764+0000 7f2b94abb640 1 Processor -- start 2024-02-18T11:40:08.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.764+0000 7f2b94abb640 1 -- start start 2024-02-18T11:40:08.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.764+0000 7f2b94abb640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2b901011f0 0x7f2b90197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:08.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.764+0000 7f2b94abb640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b901023e0 0x7f2b901977c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:08.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.764+0000 7f2b94abb640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b90102da0 0x7f2b9019e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:08.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.764+0000 7f2b94abb640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9019ed70 con 0x7f2b90102da0 2024-02-18T11:40:08.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.764+0000 7f2b94abb640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9019eee0 con 0x7f2b901011f0 2024-02-18T11:40:08.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.764+0000 7f2b94abb640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9019f0a0 con 0x7f2b901023e0 2024-02-18T11:40:08.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.764+0000 7f2b8ed9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b90102da0 0x7f2b9019e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:08.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.764+0000 7f2b8ed9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b90102da0 0x7f2b9019e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39988/0 (socket says 172.21.15.12:39988) 2024-02-18T11:40:08.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.764+0000 7f2b8ed9d640 1 -- 172.21.15.12:0/1376423061 learned_addr learned my addr 172.21.15.12:0/1376423061 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:40:08.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.764+0000 7f2b8dd9b640 1 --2- 172.21.15.12:0/1376423061 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b901023e0 0x7f2b901977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:08.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.764+0000 7f2b8e59c640 1 --2- 172.21.15.12:0/1376423061 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2b901011f0 0x7f2b90197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:08.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.764+0000 7f2b8ed9d640 1 -- 172.21.15.12:0/1376423061 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b901023e0 msgr2=0x7f2b901977c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:08.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.764+0000 7f2b8ed9d640 1 --2- 172.21.15.12:0/1376423061 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b901023e0 0x7f2b901977c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:08.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.764+0000 7f2b8ed9d640 1 -- 172.21.15.12:0/1376423061 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2b901011f0 msgr2=0x7f2b90197280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:08.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.764+0000 7f2b8ed9d640 1 --2- 172.21.15.12:0/1376423061 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2b901011f0 0x7f2b90197280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:08.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.764+0000 7f2b8ed9d640 1 -- 172.21.15.12:0/1376423061 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b84009560 con 0x7f2b90102da0 2024-02-18T11:40:08.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.764+0000 7f2b8ed9d640 1 --2- 172.21.15.12:0/1376423061 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b90102da0 0x7f2b9019e830 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2b88009fd0 tx=0x7f2b8800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:08.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.765+0000 7f2b7f7fe640 1 -- 172.21.15.12:0/1376423061 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b8800bea0 con 0x7f2b90102da0 2024-02-18T11:40:08.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.765+0000 7f2b7f7fe640 1 -- 172.21.15.12:0/1376423061 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2b88009940 con 0x7f2b90102da0 2024-02-18T11:40:08.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.765+0000 7f2b94abb640 1 -- 172.21.15.12:0/1376423061 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b9019f380 con 0x7f2b90102da0 2024-02-18T11:40:08.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.765+0000 7f2b7f7fe640 1 -- 172.21.15.12:0/1376423061 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b880106c0 con 0x7f2b90102da0 2024-02-18T11:40:08.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.765+0000 7f2b94abb640 1 -- 172.21.15.12:0/1376423061 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b9019f860 con 0x7f2b90102da0 2024-02-18T11:40:08.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.766+0000 7f2b94abb640 1 -- 172.21.15.12:0/1376423061 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b90108590 con 0x7f2b90102da0 2024-02-18T11:40:08.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.767+0000 7f2b7f7fe640 1 -- 172.21.15.12:0/1376423061 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 137296+0+0 (secure 0 0 0) 0x7f2b88004240 con 0x7f2b90102da0 2024-02-18T11:40:08.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.767+0000 7f2b7f7fe640 1 --2- 172.21.15.12:0/1376423061 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f2b780a2860 0x7f2b780a4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:08.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.767+0000 7f2b8e59c640 1 --2- 172.21.15.12:0/1376423061 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f2b780a2860 0x7f2b780a4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:08.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.767+0000 7f2b7f7fe640 1 -- 172.21.15.12:0/1376423061 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2b880ce370 con 0x7f2b90102da0 2024-02-18T11:40:08.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.767+0000 7f2b8e59c640 1 --2- 172.21.15.12:0/1376423061 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f2b780a2860 0x7f2b780a4d20 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2b80005fd0 tx=0x7f2b80005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:08.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.771+0000 7f2b7f7fe640 1 -- 172.21.15.12:0/1376423061 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2b88096fb0 con 0x7f2b90102da0 2024-02-18T11:40:08.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.954+0000 7f2b94abb640 1 -- 172.21.15.12:0/1376423061 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2b9010cc00 con 0x7f2b90102da0 2024-02-18T11:40:08.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.955+0000 7f2b7f7fe640 1 -- 172.21.15.12:0/1376423061 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+22662 (secure 0 0 0) 0x7f2b8809a670 con 0x7f2b90102da0 2024-02-18T11:40:08.949 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:40:08.950 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":89,"fsid":"0c8bb0de-ce51-11ee-95ba-87774f69a715","created":"2024-02-18T11:30:49.110059+0000","modified":"2024-02-18T11:39:52.478480+0000","last_up_change":"2024-02-18T11:37:57.512337+0000","last_in_change":"2024-02-18T11:37:36.567969+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-18T11:34:15.281138+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-18T11:38:44.802134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-18T11:38:46.081188+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"35f15ea6-deb8-48a1-a7dd-ab649ec13533","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":536522426},{"type":"v1","addr":"172.21.15.12:6803","nonce":536522426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":536522426},{"type":"v1","addr":"172.21.15.12:6805","nonce":536522426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":536522426},{"type":"v1","addr":"172.21.15.12:6809","nonce":536522426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":536522426},{"type":"v1","addr":"172.21.15.12:6807","nonce":536522426}]},"public_addr":"172.21.15.12:6803/536522426","cluster_addr":"172.21.15.12:6805/536522426","heartbeat_back_addr":"172.21.15.12:6809/536522426","heartbeat_front_addr":"172.21.15.12:6807/536522426","state":["exists","up"]},{"osd":1,"uuid":"2af937ef-d827-44e6-b805-70cffbfe1126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":4249111488},{"type":"v1","addr":"172.21.15.158:6801","nonce":4249111488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":4249111488},{"type":"v1","addr":"172.21.15.158:6803","nonce":4249111488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":4249111488},{"type":"v1","addr":"172.21.15.158:6807","nonce":4249111488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":4249111488},{"type":"v1","addr":"172.21.15.158:6805","nonce":4249111488}]},"public_addr":"172.21.15.158:6801/4249111488","cluster_addr":"172.21.15.158:6803/4249111488","heartbeat_back_addr":"172.21.15.158:6807/4249111488","heartbeat_front_addr":"172.21.15.158:6805/4249111488","state":["exists","up"]},{"osd":2,"uuid":"17b804dd-31df-45ae-a637-4a5f65bbaf26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":627976150},{"type":"v1","addr":"172.21.15.188:6801","nonce":627976150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":627976150},{"type":"v1","addr":"172.21.15.188:6803","nonce":627976150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":627976150},{"type":"v1","addr":"172.21.15.188:6807","nonce":627976150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":627976150},{"type":"v1","addr":"172.21.15.188:6805","nonce":627976150}]},"public_addr":"172.21.15.188:6801/627976150","cluster_addr":"172.21.15.188:6803/627976150","heartbeat_back_addr":"172.21.15.188:6807/627976150","heartbeat_front_addr":"172.21.15.188:6805/627976150","state":["exists","up"]},{"osd":3,"uuid":"3f9cc423-cdfa-4acf-aaa2-b58b32ccb9cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":3758512503},{"type":"v1","addr":"172.21.15.12:6811","nonce":3758512503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":3758512503},{"type":"v1","addr":"172.21.15.12:6813","nonce":3758512503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3758512503},{"type":"v1","addr":"172.21.15.12:6817","nonce":3758512503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":3758512503},{"type":"v1","addr":"172.21.15.12:6815","nonce":3758512503}]},"public_addr":"172.21.15.12:6811/3758512503","cluster_addr":"172.21.15.12:6813/3758512503","heartbeat_back_addr":"172.21.15.12:6817/3758512503","heartbeat_front_addr":"172.21.15.12:6815/3758512503","state":["exists","up"]},{"osd":4,"uuid":"a92e20ec-af1c-4ef4-a0cb-caba5fcc71aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":3892820997},{"type":"v1","addr":"172.21.15.158:6809","nonce":3892820997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":3892820997},{"type":"v1","addr":"172.21.15.158:6811","nonce":3892820997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":3892820997},{"type":"v1","addr":"172.21.15.158:6815","nonce":3892820997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":3892820997},{"type":"v1","addr":"172.21.15.158:6813","nonce":3892820997}]},"public_addr":"172.21.15.158:6809/3892820997","cluster_addr":"172.21.15.158:6811/3892820997","heartbeat_back_addr":"172.21.15.158:6815/3892820997","heartbeat_front_addr":"172.21.15.158:6813/3892820997","state":["exists","up"]},{"osd":5,"uuid":"d9c2e0c8-85f5-4ae2-bf3e-125c1be3a2db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3939495896},{"type":"v1","addr":"172.21.15.188:6809","nonce":3939495896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3939495896},{"type":"v1","addr":"172.21.15.188:6811","nonce":3939495896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3939495896},{"type":"v1","addr":"172.21.15.188:6815","nonce":3939495896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3939495896},{"type":"v1","addr":"172.21.15.188:6813","nonce":3939495896}]},"public_addr":"172.21.15.188:6809/3939495896","cluster_addr":"172.21.15.188:6811/3939495896","heartbeat_back_addr":"172.21.15.188:6815/3939495896","heartbeat_front_addr":"172.21.15.188:6813/3939495896","state":["exists","up"]},{"osd":6,"uuid":"4a523512-de6b-49f0-be81-00f2f00c1c10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":1206337767},{"type":"v1","addr":"172.21.15.12:6819","nonce":1206337767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":1206337767},{"type":"v1","addr":"172.21.15.12:6821","nonce":1206337767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":1206337767},{"type":"v1","addr":"172.21.15.12:6825","nonce":1206337767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":1206337767},{"type":"v1","addr":"172.21.15.12:6823","nonce":1206337767}]},"public_addr":"172.21.15.12:6819/1206337767","cluster_addr":"172.21.15.12:6821/1206337767","heartbeat_back_addr":"172.21.15.12:6825/1206337767","heartbeat_front_addr":"172.21.15.12:6823/1206337767","state":["exists","up"]},{"osd":7,"uuid":"363271a6-df93-44c9-b0e5-3d29768aa96c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":3283400032},{"type":"v1","addr":"172.21.15.158:6817","nonce":3283400032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":3283400032},{"type":"v1","addr":"172.21.15.158:6819","nonce":3283400032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":3283400032},{"type":"v1","addr":"172.21.15.158:6823","nonce":3283400032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":3283400032},{"type":"v1","addr":"172.21.15.158:6821","nonce":3283400032}]},"public_addr":"172.21.15.158:6817/3283400032","cluster_addr":"172.21.15.158:6819/3283400032","heartbeat_back_addr":"172.21.15.158:6823/3283400032","heartbeat_front_addr":"172.21.15.158:6821/3283400032","state":["exists","up"]},{"osd":8,"uuid":"9a089528-3e8e-45d3-9279-ddb54088b96a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2110170682},{"type":"v1","addr":"172.21.15.188:6817","nonce":2110170682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2110170682},{"type":"v1","addr":"172.21.15.188:6819","nonce":2110170682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":2110170682},{"type":"v1","addr":"172.21.15.188:6823","nonce":2110170682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2110170682},{"type":"v1","addr":"172.21.15.188:6821","nonce":2110170682}]},"public_addr":"172.21.15.188:6817/2110170682","cluster_addr":"172.21.15.188:6819/2110170682","heartbeat_back_addr":"172.21.15.188:6823/2110170682","heartbeat_front_addr":"172.21.15.188:6821/2110170682","state":["exists","up"]},{"osd":9,"uuid":"b12e2727-d649-4464-90e1-fafbc1754832","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1825928986},{"type":"v1","addr":"172.21.15.12:6827","nonce":1825928986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1825928986},{"type":"v1","addr":"172.21.15.12:6829","nonce":1825928986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":1825928986},{"type":"v1","addr":"172.21.15.12:6833","nonce":1825928986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1825928986},{"type":"v1","addr":"172.21.15.12:6831","nonce":1825928986}]},"public_addr":"172.21.15.12:6827/1825928986","cluster_addr":"172.21.15.12:6829/1825928986","heartbeat_back_addr":"172.21.15.12:6833/1825928986","heartbeat_front_addr":"172.21.15.12:6831/1825928986","state":["exists","up"]},{"osd":10,"uuid":"518b4cbf-9d90-4e9a-8ac9-b5cb9b3dee4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":3239052167},{"type":"v1","addr":"172.21.15.158:6825","nonce":3239052167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":3239052167},{"type":"v1","addr":"172.21.15.158:6827","nonce":3239052167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":3239052167},{"type":"v1","addr":"172.21.15.158:6831","nonce":3239052167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":3239052167},{"type":"v1","addr":"172.21.15.158:6829","nonce":3239052167}]},"public_addr":"172.21.15.158:6825/3239052167","cluster_addr":"172.21.15.158:6827/3239052167","heartbeat_back_addr":"172.21.15.158:6831/3239052167","heartbeat_front_addr":"172.21.15.158:6829/3239052167","state":["exists","up"]},{"osd":11,"uuid":"98aa3a0d-4b2f-46c8-9341-375d13794c3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1135773350},{"type":"v1","addr":"172.21.15.188:6825","nonce":1135773350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1135773350},{"type":"v1","addr":"172.21.15.188:6827","nonce":1135773350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1135773350},{"type":"v1","addr":"172.21.15.188:6831","nonce":1135773350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1135773350},{"type":"v1","addr":"172.21.15.188:6829","nonce":1135773350}]},"public_addr":"172.21.15.188:6825/1135773350","cluster_addr":"172.21.15.188:6827/1135773350","heartbeat_back_addr":"172.21.15.188:6831/1135773350","heartbeat_front_addr":"172.21.15.188:6829/1135773350","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:33:24.280879+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:33:47.859461+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:34:11.725283+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:34:35.707629+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:35:00.024197+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:35:24.499803+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:35:48.953870+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:36:13.642456+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:36:38.936457+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:37:04.021466+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:37:29.073178+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:37:54.863919+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.37","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:6801/4135274411":"2024-02-19T11:31:50.936928+0000","172.21.15.12:6800/4135274411":"2024-02-19T11:31:50.936928+0000","172.21.15.12:0/118428584":"2024-02-19T11:31:50.936928+0000","172.21.15.12:0/4276810915":"2024-02-19T11:31:50.936928+0000","172.21.15.12:0/3777545262":"2024-02-19T11:31:24.000830+0000","172.21.15.12:0/114707743":"2024-02-19T11:31:24.000830+0000","172.21.15.12:6800/3925204723":"2024-02-19T11:31:24.000830+0000","172.21.15.12:6801/3925204723":"2024-02-19T11:31:24.000830+0000","172.21.15.12:0/74881618":"2024-02-19T11:31:24.000830+0000","172.21.15.12:0/1614805375":"2024-02-19T11:31:50.936928+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-18T11:40:08.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.957+0000 7f2b94abb640 1 -- 172.21.15.12:0/1376423061 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f2b780a2860 msgr2=0x7f2b780a4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:08.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.957+0000 7f2b94abb640 1 --2- 172.21.15.12:0/1376423061 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f2b780a2860 0x7f2b780a4d20 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2b80005fd0 tx=0x7f2b80005850 comp rx=0 tx=0).stop 2024-02-18T11:40:08.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.957+0000 7f2b94abb640 1 -- 172.21.15.12:0/1376423061 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b90102da0 msgr2=0x7f2b9019e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:08.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.957+0000 7f2b94abb640 1 --2- 172.21.15.12:0/1376423061 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b90102da0 0x7f2b9019e830 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2b88009fd0 tx=0x7f2b8800b260 comp rx=0 tx=0).stop 2024-02-18T11:40:08.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.957+0000 7f2b94abb640 1 -- 172.21.15.12:0/1376423061 shutdown_connections 2024-02-18T11:40:08.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.957+0000 7f2b94abb640 1 --2- 172.21.15.12:0/1376423061 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f2b780a2860 0x7f2b780a4d20 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:08.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.957+0000 7f2b94abb640 1 --2- 172.21.15.12:0/1376423061 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b901023e0 0x7f2b901977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:08.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.957+0000 7f2b94abb640 1 --2- 172.21.15.12:0/1376423061 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b90102da0 0x7f2b9019e830 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:08.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.957+0000 7f2b94abb640 1 --2- 172.21.15.12:0/1376423061 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2b901011f0 0x7f2b90197280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:08.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.957+0000 7f2b94abb640 1 -- 172.21.15.12:0/1376423061 >> 172.21.15.12:0/1376423061 conn(0x7f2b900fc760 msgr2=0x7f2b90105760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:08.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.957+0000 7f2b94abb640 1 -- 172.21.15.12:0/1376423061 shutdown_connections 2024-02-18T11:40:08.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:08.957+0000 7f2b94abb640 1 -- 172.21.15.12:0/1376423061 wait complete. 2024-02-18T11:40:08.963 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-18T11:34:15.281138+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-18T11:38:44.802134+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-18T11:38:46.081188+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-18T11:40:08.963 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-18T11:40:09.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.075+0000 7f72913f4640 1 -- 172.21.15.12:0/3100115607 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f728c100d40 msgr2=0x7f728c1085e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:09.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.075+0000 7f72913f4640 1 --2- 172.21.15.12:0/3100115607 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f728c100d40 0x7f728c1085e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7284009ca0 tx=0x7f728402c0f0 comp rx=0 tx=0).stop 2024-02-18T11:40:09.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.075+0000 7f72913f4640 1 -- 172.21.15.12:0/3100115607 shutdown_connections 2024-02-18T11:40:09.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.075+0000 7f72913f4640 1 --2- 172.21.15.12:0/3100115607 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f728c100d40 0x7f728c1085e0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:09.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.075+0000 7f72913f4640 1 --2- 172.21.15.12:0/3100115607 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f728c100380 0x7f728c100800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:09.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.075+0000 7f72913f4640 1 --2- 172.21.15.12:0/3100115607 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f728c1016e0 0x7f728c101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:09.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.075+0000 7f72913f4640 1 -- 172.21.15.12:0/3100115607 >> 172.21.15.12:0/3100115607 conn(0x7f728c0fadf0 msgr2=0x7f728c0fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:09.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.075+0000 7f72913f4640 1 -- 172.21.15.12:0/3100115607 shutdown_connections 2024-02-18T11:40:09.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.075+0000 7f72913f4640 1 -- 172.21.15.12:0/3100115607 wait complete. 2024-02-18T11:40:09.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.075+0000 7f72913f4640 1 Processor -- start 2024-02-18T11:40:09.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.076+0000 7f72913f4640 1 -- start start 2024-02-18T11:40:09.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.076+0000 7f72913f4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f728c100380 0x7f728c197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:09.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.076+0000 7f72913f4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f728c100d40 0x7f728c197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:09.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.076+0000 7f72913f4640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f728c1016e0 0x7f728c19e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:09.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.076+0000 7f72913f4640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f728c19ed40 con 0x7f728c100d40 2024-02-18T11:40:09.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.076+0000 7f72913f4640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f728c19eeb0 con 0x7f728c1016e0 2024-02-18T11:40:09.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.076+0000 7f72913f4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f728c19f070 con 0x7f728c100380 2024-02-18T11:40:09.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.076+0000 7f728a7fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f728c100d40 0x7f728c197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:09.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.076+0000 7f728a7fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f728c100d40 0x7f728c197790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40002/0 (socket says 172.21.15.12:40002) 2024-02-18T11:40:09.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.076+0000 7f728a7fc640 1 -- 172.21.15.12:0/3774046745 learned_addr learned my addr 172.21.15.12:0/3774046745 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:40:09.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.076+0000 7f728affd640 1 --2- 172.21.15.12:0/3774046745 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f728c100380 0x7f728c197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:09.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.076+0000 7f728b7fe640 1 --2- 172.21.15.12:0/3774046745 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f728c1016e0 0x7f728c19e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:09.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.076+0000 7f728a7fc640 1 -- 172.21.15.12:0/3774046745 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f728c100380 msgr2=0x7f728c197250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:09.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.076+0000 7f728a7fc640 1 --2- 172.21.15.12:0/3774046745 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f728c100380 0x7f728c197250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:09.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.076+0000 7f728a7fc640 1 -- 172.21.15.12:0/3774046745 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f728c1016e0 msgr2=0x7f728c19e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:09.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.076+0000 7f728a7fc640 1 --2- 172.21.15.12:0/3774046745 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f728c1016e0 0x7f728c19e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:09.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.076+0000 7f728a7fc640 1 -- 172.21.15.12:0/3774046745 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7284009900 con 0x7f728c100d40 2024-02-18T11:40:09.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.076+0000 7f728b7fe640 1 --2- 172.21.15.12:0/3774046745 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f728c1016e0 0x7f728c19e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:40:09.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.076+0000 7f728a7fc640 1 --2- 172.21.15.12:0/3774046745 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f728c100d40 0x7f728c197790 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f728000b4a0 tx=0x7f728000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:09.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.076+0000 7f7273fff640 1 -- 172.21.15.12:0/3774046745 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7280010610 con 0x7f728c100d40 2024-02-18T11:40:09.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.077+0000 7f7273fff640 1 -- 172.21.15.12:0/3774046745 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7280010c50 con 0x7f728c100d40 2024-02-18T11:40:09.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.077+0000 7f72913f4640 1 -- 172.21.15.12:0/3774046745 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f728c19f350 con 0x7f728c100d40 2024-02-18T11:40:09.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.077+0000 7f7273fff640 1 -- 172.21.15.12:0/3774046745 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7280017e20 con 0x7f728c100d40 2024-02-18T11:40:09.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.077+0000 7f72913f4640 1 -- 172.21.15.12:0/3774046745 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f728c19f8b0 con 0x7f728c100d40 2024-02-18T11:40:09.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.078+0000 7f72913f4640 1 -- 172.21.15.12:0/3774046745 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f728c1073f0 con 0x7f728c100d40 2024-02-18T11:40:09.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.079+0000 7f7273fff640 1 -- 172.21.15.12:0/3774046745 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 137296+0+0 (secure 0 0 0) 0x7f7280017480 con 0x7f728c100d40 2024-02-18T11:40:09.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.079+0000 7f7273fff640 1 --2- 172.21.15.12:0/3774046745 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f72740a2860 0x7f72740a4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:09.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.080+0000 7f728affd640 1 --2- 172.21.15.12:0/3774046745 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f72740a2860 0x7f72740a4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:09.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.080+0000 7f7273fff640 1 -- 172.21.15.12:0/3774046745 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f72800cdfd0 con 0x7f728c100d40 2024-02-18T11:40:09.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.080+0000 7f728affd640 1 --2- 172.21.15.12:0/3774046745 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f72740a2860 0x7f72740a4d20 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f727c005fd0 tx=0x7f727c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:09.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.083+0000 7f7273fff640 1 -- 172.21.15.12:0/3774046745 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7280096dc0 con 0x7f728c100d40 2024-02-18T11:40:09.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.266+0000 7f72913f4640 1 -- 172.21.15.12:0/3774046745 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f728c10cdf0 con 0x7f728c100d40 2024-02-18T11:40:09.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.267+0000 7f7273fff640 1 -- 172.21.15.12:0/3774046745 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7f72800092b0 con 0x7f728c100d40 2024-02-18T11:40:09.261 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-02-18T11:40:09.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.269+0000 7f72913f4640 1 -- 172.21.15.12:0/3774046745 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f72740a2860 msgr2=0x7f72740a4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:09.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.269+0000 7f72913f4640 1 --2- 172.21.15.12:0/3774046745 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f72740a2860 0x7f72740a4d20 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f727c005fd0 tx=0x7f727c00b390 comp rx=0 tx=0).stop 2024-02-18T11:40:09.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.269+0000 7f72913f4640 1 -- 172.21.15.12:0/3774046745 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f728c100d40 msgr2=0x7f728c197790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:09.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.269+0000 7f72913f4640 1 --2- 172.21.15.12:0/3774046745 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f728c100d40 0x7f728c197790 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f728000b4a0 tx=0x7f728000b960 comp rx=0 tx=0).stop 2024-02-18T11:40:09.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.269+0000 7f72913f4640 1 -- 172.21.15.12:0/3774046745 shutdown_connections 2024-02-18T11:40:09.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.269+0000 7f72913f4640 1 --2- 172.21.15.12:0/3774046745 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f72740a2860 0x7f72740a4d20 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:09.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.269+0000 7f72913f4640 1 --2- 172.21.15.12:0/3774046745 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f728c1016e0 0x7f728c19e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:09.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.269+0000 7f72913f4640 1 --2- 172.21.15.12:0/3774046745 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f728c100d40 0x7f728c197790 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:09.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.269+0000 7f72913f4640 1 --2- 172.21.15.12:0/3774046745 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f728c100380 0x7f728c197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:09.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.269+0000 7f72913f4640 1 -- 172.21.15.12:0/3774046745 >> 172.21.15.12:0/3774046745 conn(0x7f728c0fadf0 msgr2=0x7f728c1045c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:09.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.269+0000 7f72913f4640 1 -- 172.21.15.12:0/3774046745 shutdown_connections 2024-02-18T11:40:09.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.269+0000 7f72913f4640 1 -- 172.21.15.12:0/3774046745 wait complete. 2024-02-18T11:40:09.273 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:40:09 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/3203441635' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-18T11:40:09.273 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:40:09 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/3687405523' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-18T11:40:09.273 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:40:09 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/1376423061' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-18T11:40:09.274 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-18T11:40:09.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:40:09 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/3203441635' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-18T11:40:09.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:40:09 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/3687405523' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-18T11:40:09.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:40:09 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/1376423061' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-18T11:40:09.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.387+0000 7fa9523b5640 1 -- 172.21.15.12:0/3196346884 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa94c1012d0 msgr2=0x7fa94c101750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:09.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.387+0000 7fa9523b5640 1 --2- 172.21.15.12:0/3196346884 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa94c1012d0 0x7fa94c101750 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa940009900 tx=0x7fa94002bc30 comp rx=0 tx=0).stop 2024-02-18T11:40:09.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.387+0000 7fa9523b5640 1 -- 172.21.15.12:0/3196346884 shutdown_connections 2024-02-18T11:40:09.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.387+0000 7fa9523b5640 1 --2- 172.21.15.12:0/3196346884 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa94c101c90 0x7fa94c108520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:09.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.387+0000 7fa9523b5640 1 --2- 172.21.15.12:0/3196346884 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa94c1012d0 0x7fa94c101750 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:09.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.387+0000 7fa9523b5640 1 --2- 172.21.15.12:0/3196346884 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa94c1000e0 0x7fa94c100500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:09.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.387+0000 7fa9523b5640 1 -- 172.21.15.12:0/3196346884 >> 172.21.15.12:0/3196346884 conn(0x7fa94c0fb670 msgr2=0x7fa94c0fda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:09.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.387+0000 7fa9523b5640 1 -- 172.21.15.12:0/3196346884 shutdown_connections 2024-02-18T11:40:09.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.387+0000 7fa9523b5640 1 -- 172.21.15.12:0/3196346884 wait complete. 2024-02-18T11:40:09.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.387+0000 7fa9523b5640 1 Processor -- start 2024-02-18T11:40:09.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.388+0000 7fa9523b5640 1 -- start start 2024-02-18T11:40:09.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.388+0000 7fa9523b5640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa94c1000e0 0x7fa94c197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:09.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.388+0000 7fa9523b5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa94c1012d0 0x7fa94c197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:09.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.388+0000 7fa9523b5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa94c101c90 0x7fa94c19e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:09.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.388+0000 7fa9523b5640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa94c19ee10 con 0x7fa94c101c90 2024-02-18T11:40:09.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.388+0000 7fa9523b5640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa94c19ef80 con 0x7fa94c1000e0 2024-02-18T11:40:09.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.388+0000 7fa9523b5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa94c19f140 con 0x7fa94c1012d0 2024-02-18T11:40:09.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.388+0000 7fa950952640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa94c101c90 0x7fa94c19e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:09.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.388+0000 7fa950952640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa94c101c90 0x7fa94c19e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40016/0 (socket says 172.21.15.12:40016) 2024-02-18T11:40:09.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.388+0000 7fa950952640 1 -- 172.21.15.12:0/4242788693 learned_addr learned my addr 172.21.15.12:0/4242788693 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:40:09.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.388+0000 7fa94b7fe640 1 --2- 172.21.15.12:0/4242788693 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa94c1012d0 0x7fa94c197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:09.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.388+0000 7fa94bfff640 1 --2- 172.21.15.12:0/4242788693 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa94c1000e0 0x7fa94c197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:09.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.388+0000 7fa950952640 1 -- 172.21.15.12:0/4242788693 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa94c1012d0 msgr2=0x7fa94c197860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:09.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.388+0000 7fa950952640 1 --2- 172.21.15.12:0/4242788693 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa94c1012d0 0x7fa94c197860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:09.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.388+0000 7fa950952640 1 -- 172.21.15.12:0/4242788693 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa94c1000e0 msgr2=0x7fa94c197320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:09.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.388+0000 7fa950952640 1 --2- 172.21.15.12:0/4242788693 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa94c1000e0 0x7fa94c197320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:09.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.388+0000 7fa950952640 1 -- 172.21.15.12:0/4242788693 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa940009560 con 0x7fa94c101c90 2024-02-18T11:40:09.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.388+0000 7fa950952640 1 --2- 172.21.15.12:0/4242788693 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa94c101c90 0x7fa94c19e8d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa94400b970 tx=0x7fa94400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:09.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.389+0000 7fa9497fa640 1 -- 172.21.15.12:0/4242788693 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa94400cb40 con 0x7fa94c101c90 2024-02-18T11:40:09.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.389+0000 7fa9523b5640 1 -- 172.21.15.12:0/4242788693 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa94c19f420 con 0x7fa94c101c90 2024-02-18T11:40:09.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.389+0000 7fa9497fa640 1 -- 172.21.15.12:0/4242788693 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa94400cca0 con 0x7fa94c101c90 2024-02-18T11:40:09.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.389+0000 7fa9497fa640 1 -- 172.21.15.12:0/4242788693 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9440077d0 con 0x7fa94c101c90 2024-02-18T11:40:09.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.389+0000 7fa9523b5640 1 -- 172.21.15.12:0/4242788693 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa94c19f930 con 0x7fa94c101c90 2024-02-18T11:40:09.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.390+0000 7fa9523b5640 1 -- 172.21.15.12:0/4242788693 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa94c10bb50 con 0x7fa94c101c90 2024-02-18T11:40:09.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.391+0000 7fa9497fa640 1 -- 172.21.15.12:0/4242788693 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 137296+0+0 (secure 0 0 0) 0x7fa944014440 con 0x7fa94c101c90 2024-02-18T11:40:09.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.392+0000 7fa9497fa640 1 --2- 172.21.15.12:0/4242788693 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fa9340a27e0 0x7fa9340a4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:09.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.392+0000 7fa94bfff640 1 --2- 172.21.15.12:0/4242788693 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fa9340a27e0 0x7fa9340a4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:09.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.392+0000 7fa9497fa640 1 -- 172.21.15.12:0/4242788693 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa9440ce320 con 0x7fa94c101c90 2024-02-18T11:40:09.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.392+0000 7fa94bfff640 1 --2- 172.21.15.12:0/4242788693 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fa9340a27e0 0x7fa9340a4ca0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa93c005fd0 tx=0x7fa93c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:09.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.395+0000 7fa9497fa640 1 -- 172.21.15.12:0/4242788693 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa944097120 con 0x7fa94c101c90 2024-02-18T11:40:09.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:40:09 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/3203441635' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-18T11:40:09.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:40:09 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/3687405523' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-18T11:40:09.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:40:09 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/1376423061' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-18T11:40:09.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.576+0000 7fa9523b5640 1 -- 172.21.15.12:0/4242788693 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa94c19fd70 con 0x7fa94c101c90 2024-02-18T11:40:09.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.576+0000 7fa9497fa640 1 -- 172.21.15.12:0/4242788693 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa944023020 con 0x7fa94c101c90 2024-02-18T11:40:09.570 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-02-18T11:40:09.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.578+0000 7fa9523b5640 1 -- 172.21.15.12:0/4242788693 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fa9340a27e0 msgr2=0x7fa9340a4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:09.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.578+0000 7fa9523b5640 1 --2- 172.21.15.12:0/4242788693 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fa9340a27e0 0x7fa9340a4ca0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa93c005fd0 tx=0x7fa93c005850 comp rx=0 tx=0).stop 2024-02-18T11:40:09.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.578+0000 7fa9523b5640 1 -- 172.21.15.12:0/4242788693 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa94c101c90 msgr2=0x7fa94c19e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:09.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.578+0000 7fa9523b5640 1 --2- 172.21.15.12:0/4242788693 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa94c101c90 0x7fa94c19e8d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa94400b970 tx=0x7fa94400be30 comp rx=0 tx=0).stop 2024-02-18T11:40:09.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.578+0000 7fa9523b5640 1 -- 172.21.15.12:0/4242788693 shutdown_connections 2024-02-18T11:40:09.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.578+0000 7fa9523b5640 1 --2- 172.21.15.12:0/4242788693 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fa9340a27e0 0x7fa9340a4ca0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:09.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.578+0000 7fa9523b5640 1 --2- 172.21.15.12:0/4242788693 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa94c1012d0 0x7fa94c197860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:09.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.578+0000 7fa9523b5640 1 --2- 172.21.15.12:0/4242788693 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa94c101c90 0x7fa94c19e8d0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:09.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.578+0000 7fa9523b5640 1 --2- 172.21.15.12:0/4242788693 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa94c1000e0 0x7fa94c197320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:09.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.578+0000 7fa9523b5640 1 -- 172.21.15.12:0/4242788693 >> 172.21.15.12:0/4242788693 conn(0x7fa94c0fb670 msgr2=0x7fa94c0697b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:09.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.578+0000 7fa9523b5640 1 -- 172.21.15.12:0/4242788693 shutdown_connections 2024-02-18T11:40:09.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.578+0000 7fa9523b5640 1 -- 172.21.15.12:0/4242788693 wait complete. 2024-02-18T11:40:09.583 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-18T11:40:09.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.695+0000 7f68ba94b640 1 -- 172.21.15.12:0/956821594 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68b4102470 msgr2=0x7f68b41028f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:09.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.695+0000 7f68ba94b640 1 --2- 172.21.15.12:0/956821594 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68b4102470 0x7f68b41028f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f68a8009900 tx=0x7f68a802bc30 comp rx=0 tx=0).stop 2024-02-18T11:40:09.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.695+0000 7f68ba94b640 1 -- 172.21.15.12:0/956821594 shutdown_connections 2024-02-18T11:40:09.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.695+0000 7f68ba94b640 1 --2- 172.21.15.12:0/956821594 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f68b4102e30 0x7f68b41096c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:09.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.695+0000 7f68ba94b640 1 --2- 172.21.15.12:0/956821594 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68b4102470 0x7f68b41028f0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:09.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.695+0000 7f68ba94b640 1 --2- 172.21.15.12:0/956821594 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f68b40ff4a0 0x7f68b40ff8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:09.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.695+0000 7f68ba94b640 1 -- 172.21.15.12:0/956821594 >> 172.21.15.12:0/956821594 conn(0x7f68b40fadd0 msgr2=0x7f68b40fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:09.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.695+0000 7f68ba94b640 1 -- 172.21.15.12:0/956821594 shutdown_connections 2024-02-18T11:40:09.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.695+0000 7f68ba94b640 1 -- 172.21.15.12:0/956821594 wait complete. 2024-02-18T11:40:09.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.695+0000 7f68ba94b640 1 Processor -- start 2024-02-18T11:40:09.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.695+0000 7f68ba94b640 1 -- start start 2024-02-18T11:40:09.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.696+0000 7f68ba94b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f68b40ff4a0 0x7f68b4101de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:09.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.696+0000 7f68ba94b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68b4102470 0x7f68b4100420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:09.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.696+0000 7f68ba94b640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f68b4102e30 0x7f68b4100960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:09.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.696+0000 7f68ba94b640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b41010c0 con 0x7f68b4102470 2024-02-18T11:40:09.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.696+0000 7f68ba94b640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b4101230 con 0x7f68b4102e30 2024-02-18T11:40:09.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.696+0000 7f68ba94b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b41a1230 con 0x7f68b40ff4a0 2024-02-18T11:40:09.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.696+0000 7f68b37fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68b4102470 0x7f68b4100420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:09.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.696+0000 7f68b37fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68b4102470 0x7f68b4100420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40038/0 (socket says 172.21.15.12:40038) 2024-02-18T11:40:09.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.696+0000 7f68b37fe640 1 -- 172.21.15.12:0/3330144362 learned_addr learned my addr 172.21.15.12:0/3330144362 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:40:09.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.696+0000 7f68b37fe640 1 -- 172.21.15.12:0/3330144362 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f68b40ff4a0 msgr2=0x7f68b4101de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:09.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.696+0000 7f68b37fe640 1 --2- 172.21.15.12:0/3330144362 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f68b40ff4a0 0x7f68b4101de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:09.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.696+0000 7f68b37fe640 1 -- 172.21.15.12:0/3330144362 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f68b4102e30 msgr2=0x7f68b4100960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:09.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.696+0000 7f68b37fe640 1 --2- 172.21.15.12:0/3330144362 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f68b4102e30 0x7f68b4100960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:09.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.696+0000 7f68b37fe640 1 -- 172.21.15.12:0/3330144362 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68a8009560 con 0x7f68b4102470 2024-02-18T11:40:09.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.696+0000 7f68b37fe640 1 --2- 172.21.15.12:0/3330144362 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68b4102470 0x7f68b4100420 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f68a8036040 tx=0x7f68a80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:09.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.696+0000 7f68b17fa640 1 -- 172.21.15.12:0/3330144362 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a80044e0 con 0x7f68b4102470 2024-02-18T11:40:09.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.696+0000 7f68ba94b640 1 -- 172.21.15.12:0/3330144362 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68b41a1480 con 0x7f68b4102470 2024-02-18T11:40:09.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.696+0000 7f68b17fa640 1 -- 172.21.15.12:0/3330144362 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f68a80054e0 con 0x7f68b4102470 2024-02-18T11:40:09.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.696+0000 7f68b17fa640 1 -- 172.21.15.12:0/3330144362 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a803e450 con 0x7f68b4102470 2024-02-18T11:40:09.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.696+0000 7f68ba94b640 1 -- 172.21.15.12:0/3330144362 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68b41a19e0 con 0x7f68b4102470 2024-02-18T11:40:09.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.697+0000 7f68ba94b640 1 -- 172.21.15.12:0/3330144362 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68b4108540 con 0x7f68b4102470 2024-02-18T11:40:09.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.698+0000 7f68b17fa640 1 -- 172.21.15.12:0/3330144362 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 137296+0+0 (secure 0 0 0) 0x7f68a8003680 con 0x7f68b4102470 2024-02-18T11:40:09.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.699+0000 7f68b17fa640 1 --2- 172.21.15.12:0/3330144362 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f689c0a27e0 0x7f689c0a4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:09.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.699+0000 7f68b3fff640 1 --2- 172.21.15.12:0/3330144362 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f689c0a27e0 0x7f689c0a4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:09.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.699+0000 7f68b17fa640 1 -- 172.21.15.12:0/3330144362 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f68a80f0a60 con 0x7f68b4102470 2024-02-18T11:40:09.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.699+0000 7f68b3fff640 1 --2- 172.21.15.12:0/3330144362 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f689c0a27e0 0x7f689c0a4ca0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f68b40695b0 tx=0x7f68a4005c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:09.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.703+0000 7f68b17fa640 1 -- 172.21.15.12:0/3330144362 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f68a80b97b0 con 0x7f68b4102470 2024-02-18T11:40:09.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.885+0000 7f68ba94b640 1 -- 172.21.15.12:0/3330144362 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f68b410cc00 con 0x7f68b4102470 2024-02-18T11:40:09.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.885+0000 7f68b17fa640 1 -- 172.21.15.12:0/3330144362 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+11 (secure 0 0 0) 0x7f68a80bce70 con 0x7f68b4102470 2024-02-18T11:40:09.879 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2024-02-18T11:40:09.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.887+0000 7f68ba94b640 1 -- 172.21.15.12:0/3330144362 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f689c0a27e0 msgr2=0x7f689c0a4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:09.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.887+0000 7f68ba94b640 1 --2- 172.21.15.12:0/3330144362 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f689c0a27e0 0x7f689c0a4ca0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f68b40695b0 tx=0x7f68a4005c90 comp rx=0 tx=0).stop 2024-02-18T11:40:09.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.887+0000 7f68ba94b640 1 -- 172.21.15.12:0/3330144362 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68b4102470 msgr2=0x7f68b4100420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:09.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.887+0000 7f68ba94b640 1 --2- 172.21.15.12:0/3330144362 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68b4102470 0x7f68b4100420 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f68a8036040 tx=0x7f68a80042f0 comp rx=0 tx=0).stop 2024-02-18T11:40:09.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.887+0000 7f68ba94b640 1 -- 172.21.15.12:0/3330144362 shutdown_connections 2024-02-18T11:40:09.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.887+0000 7f68ba94b640 1 --2- 172.21.15.12:0/3330144362 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f689c0a27e0 0x7f689c0a4ca0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:09.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.887+0000 7f68ba94b640 1 --2- 172.21.15.12:0/3330144362 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f68b4102e30 0x7f68b4100960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:09.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.887+0000 7f68ba94b640 1 --2- 172.21.15.12:0/3330144362 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68b4102470 0x7f68b4100420 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:09.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.887+0000 7f68ba94b640 1 --2- 172.21.15.12:0/3330144362 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f68b40ff4a0 0x7f68b4101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:09.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.887+0000 7f68ba94b640 1 -- 172.21.15.12:0/3330144362 >> 172.21.15.12:0/3330144362 conn(0x7f68b40fadd0 msgr2=0x7f68b4105710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:09.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.888+0000 7f68ba94b640 1 -- 172.21.15.12:0/3330144362 shutdown_connections 2024-02-18T11:40:09.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:09.888+0000 7f68ba94b640 1 -- 172.21.15.12:0/3330144362 wait complete. 2024-02-18T11:40:09.893 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-18T11:40:09.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.000+0000 7ff7a64c2640 1 -- 172.21.15.12:0/2742656344 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7a0104610 msgr2=0x7ff7a0106a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:09.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.000+0000 7ff7a64c2640 1 --2- 172.21.15.12:0/2742656344 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7a0104610 0x7ff7a0106a00 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff790009900 tx=0x7ff79002bc30 comp rx=0 tx=0).stop 2024-02-18T11:40:09.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.001+0000 7ff7a64c2640 1 -- 172.21.15.12:0/2742656344 shutdown_connections 2024-02-18T11:40:09.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.001+0000 7ff7a64c2640 1 --2- 172.21.15.12:0/2742656344 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff7a0106f40 0x7ff7a0109390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:09.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.001+0000 7ff7a64c2640 1 --2- 172.21.15.12:0/2742656344 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7a0104610 0x7ff7a0106a00 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:09.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.001+0000 7ff7a64c2640 1 --2- 172.21.15.12:0/2742656344 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff7a0101ce0 0x7ff7a01040d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:09.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.001+0000 7ff7a64c2640 1 -- 172.21.15.12:0/2742656344 >> 172.21.15.12:0/2742656344 conn(0x7ff7a00fb6f0 msgr2=0x7ff7a00fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:09.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.001+0000 7ff7a64c2640 1 -- 172.21.15.12:0/2742656344 shutdown_connections 2024-02-18T11:40:09.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.001+0000 7ff7a64c2640 1 -- 172.21.15.12:0/2742656344 wait complete. 2024-02-18T11:40:09.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.001+0000 7ff7a64c2640 1 Processor -- start 2024-02-18T11:40:09.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.001+0000 7ff7a64c2640 1 -- start start 2024-02-18T11:40:09.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.001+0000 7ff7a64c2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff7a0101ce0 0x7ff7a0197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:09.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.001+0000 7ff7a64c2640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff7a0104610 0x7ff7a01977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:09.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.002+0000 7ff7a64c2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7a0106f40 0x7ff7a019e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:09.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.002+0000 7ff7a64c2640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a019ed80 con 0x7ff7a0106f40 2024-02-18T11:40:09.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.002+0000 7ff7a64c2640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a019eef0 con 0x7ff7a0104610 2024-02-18T11:40:09.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.002+0000 7ff7a64c2640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a019f0b0 con 0x7ff7a0101ce0 2024-02-18T11:40:09.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.002+0000 7ff7a4a5f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7a0106f40 0x7ff7a019e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:09.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.002+0000 7ff7a4a5f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7a0106f40 0x7ff7a019e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40048/0 (socket says 172.21.15.12:40048) 2024-02-18T11:40:09.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.002+0000 7ff7a4a5f640 1 -- 172.21.15.12:0/309864830 learned_addr learned my addr 172.21.15.12:0/309864830 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:40:09.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.002+0000 7ff79ffff640 1 --2- 172.21.15.12:0/309864830 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff7a0101ce0 0x7ff7a0197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:09.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.002+0000 7ff79f7fe640 1 --2- 172.21.15.12:0/309864830 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff7a0104610 0x7ff7a01977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:09.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.002+0000 7ff7a4a5f640 1 -- 172.21.15.12:0/309864830 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff7a0101ce0 msgr2=0x7ff7a0197290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:09.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.002+0000 7ff7a4a5f640 1 --2- 172.21.15.12:0/309864830 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff7a0101ce0 0x7ff7a0197290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:09.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.002+0000 7ff7a4a5f640 1 -- 172.21.15.12:0/309864830 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff7a0104610 msgr2=0x7ff7a01977d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:09.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.002+0000 7ff7a4a5f640 1 --2- 172.21.15.12:0/309864830 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff7a0104610 0x7ff7a01977d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:09.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.002+0000 7ff7a4a5f640 1 -- 172.21.15.12:0/309864830 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff790009560 con 0x7ff7a0106f40 2024-02-18T11:40:10.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.002+0000 7ff79f7fe640 1 --2- 172.21.15.12:0/309864830 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff7a0104610 0x7ff7a01977d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:40:10.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.002+0000 7ff7a4a5f640 1 --2- 172.21.15.12:0/309864830 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7a0106f40 0x7ff7a019e840 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff79800eaa0 tx=0x7ff79800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:10.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.002+0000 7ff79d7fa640 1 -- 172.21.15.12:0/309864830 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff79800cdb0 con 0x7ff7a0106f40 2024-02-18T11:40:10.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.002+0000 7ff79d7fa640 1 -- 172.21.15.12:0/309864830 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff798005020 con 0x7ff7a0106f40 2024-02-18T11:40:10.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.002+0000 7ff7a64c2640 1 -- 172.21.15.12:0/309864830 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7a019f390 con 0x7ff7a0106f40 2024-02-18T11:40:10.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.002+0000 7ff79d7fa640 1 -- 172.21.15.12:0/309864830 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7980116b0 con 0x7ff7a0106f40 2024-02-18T11:40:10.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.002+0000 7ff7a64c2640 1 -- 172.21.15.12:0/309864830 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7a019f870 con 0x7ff7a0106f40 2024-02-18T11:40:10.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.003+0000 7ff7a64c2640 1 -- 172.21.15.12:0/309864830 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7a00fd1e0 con 0x7ff7a0106f40 2024-02-18T11:40:10.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.005+0000 7ff79d7fa640 1 -- 172.21.15.12:0/309864830 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 137296+0+0 (secure 0 0 0) 0x7ff798005520 con 0x7ff7a0106f40 2024-02-18T11:40:10.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.005+0000 7ff79d7fa640 1 --2- 172.21.15.12:0/309864830 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7ff7880a27e0 0x7ff7880a4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:10.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.005+0000 7ff79ffff640 1 --2- 172.21.15.12:0/309864830 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7ff7880a27e0 0x7ff7880a4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:10.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.005+0000 7ff79d7fa640 1 -- 172.21.15.12:0/309864830 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff7980ce8e0 con 0x7ff7a0106f40 2024-02-18T11:40:10.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.006+0000 7ff79ffff640 1 --2- 172.21.15.12:0/309864830 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7ff7880a27e0 0x7ff7880a4ca0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff79400bbe0 tx=0x7ff79400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:10.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.010+0000 7ff79d7fa640 1 -- 172.21.15.12:0/309864830 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff798097520 con 0x7ff7a0106f40 2024-02-18T11:40:10.172 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:40:10 smithi012 ceph-mon[32460]: pgmap v328: 97 pgs: 97 active+clean; 639 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-18T11:40:10.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:40:10 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/3774046745' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-18T11:40:10.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:40:10 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/4242788693' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-18T11:40:10.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:40:10 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/3330144362' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-18T11:40:10.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.230+0000 7ff7a64c2640 1 -- 172.21.15.12:0/309864830 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff7a010cc00 con 0x7ff7a0106f40 2024-02-18T11:40:10.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.231+0000 7ff79d7fa640 1 -- 172.21.15.12:0/309864830 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11349 (secure 0 0 0) 0x7ff79801f020 con 0x7ff7a0106f40 2024-02-18T11:40:10.225 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:40:10.225 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1788335302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1788335302},{"type":"v1","addr":"172.21.15.12:6835","nonce":1788335302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/2144692455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":2144692455},{"type":"v1","addr":"172.21.15.12:6837","nonce":2144692455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3569056036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3569056036},{"type":"v1","addr":"172.21.15.12:6839","nonce":3569056036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14622,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/1973234486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":1973234486},{"type":"v1","addr":"172.21.15.12:6841","nonce":1973234486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1207202465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1207202465},{"type":"v1","addr":"172.21.15.188:6833","nonce":1207202465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6833/2636348323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6832","nonce":2636348323},{"type":"v1","addr":"172.21.15.158:6833","nonce":2636348323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-18T11:38:46.888173+0000","modified":"2024-02-18T11:38:51.531552+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24524,"mds_1":24508,"mds_2":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.158:6835/3673253642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":3673253642},{"type":"v1","addr":"172.21.15.158:6835","nonce":3673253642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/806709644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":806709644},{"type":"v1","addr":"172.21.15.188:6835","nonce":806709644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.158:6837/813251825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6836","nonce":813251825},{"type":"v1","addr":"172.21.15.158:6837","nonce":813251825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.188:6837/1036630143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1036630143},{"type":"v1","addr":"172.21.15.188:6837","nonce":1036630143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.188:6839/1342671190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":1342671190},{"type":"v1","addr":"172.21.15.188:6839","nonce":1342671190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.158:6839/1127200849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6838","nonce":1127200849},{"type":"v1","addr":"172.21.15.158:6839","nonce":1127200849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-18T11:40:10.225 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 20 2024-02-18T11:40:10.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.233+0000 7ff7a64c2640 1 -- 172.21.15.12:0/309864830 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7ff7880a27e0 msgr2=0x7ff7880a4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:10.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.233+0000 7ff7a64c2640 1 --2- 172.21.15.12:0/309864830 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7ff7880a27e0 0x7ff7880a4ca0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff79400bbe0 tx=0x7ff79400b390 comp rx=0 tx=0).stop 2024-02-18T11:40:10.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.233+0000 7ff7a64c2640 1 -- 172.21.15.12:0/309864830 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7a0106f40 msgr2=0x7ff7a019e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:10.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.233+0000 7ff7a64c2640 1 --2- 172.21.15.12:0/309864830 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7a0106f40 0x7ff7a019e840 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff79800eaa0 tx=0x7ff79800ef60 comp rx=0 tx=0).stop 2024-02-18T11:40:10.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.233+0000 7ff7a64c2640 1 -- 172.21.15.12:0/309864830 shutdown_connections 2024-02-18T11:40:10.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.233+0000 7ff7a64c2640 1 --2- 172.21.15.12:0/309864830 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7ff7880a27e0 0x7ff7880a4ca0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:10.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.233+0000 7ff7a64c2640 1 --2- 172.21.15.12:0/309864830 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7a0106f40 0x7ff7a019e840 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:10.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.233+0000 7ff7a64c2640 1 --2- 172.21.15.12:0/309864830 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff7a0104610 0x7ff7a01977d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:10.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.233+0000 7ff7a64c2640 1 --2- 172.21.15.12:0/309864830 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff7a0101ce0 0x7ff7a0197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:10.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.233+0000 7ff7a64c2640 1 -- 172.21.15.12:0/309864830 >> 172.21.15.12:0/309864830 conn(0x7ff7a00fb6f0 msgr2=0x7ff7a0102580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:10.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.233+0000 7ff7a64c2640 1 -- 172.21.15.12:0/309864830 shutdown_connections 2024-02-18T11:40:10.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.233+0000 7ff7a64c2640 1 -- 172.21.15.12:0/309864830 wait complete. 2024-02-18T11:40:10.238 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-18T11:40:10.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.347+0000 7f31a6b3a640 1 -- 172.21.15.12:0/3483004135 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31a0102d90 msgr2=0x7f31a0109620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:10.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.347+0000 7f31a6b3a640 1 --2- 172.21.15.12:0/3483004135 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31a0102d90 0x7f31a0109620 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f3198009ca0 tx=0x7f319802c0f0 comp rx=0 tx=0).stop 2024-02-18T11:40:10.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.347+0000 7f31a6b3a640 1 -- 172.21.15.12:0/3483004135 shutdown_connections 2024-02-18T11:40:10.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.347+0000 7f31a6b3a640 1 --2- 172.21.15.12:0/3483004135 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31a0102d90 0x7f31a0109620 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:10.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.347+0000 7f31a6b3a640 1 --2- 172.21.15.12:0/3483004135 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f31a01023d0 0x7f31a0102850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:10.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.347+0000 7f31a6b3a640 1 --2- 172.21.15.12:0/3483004135 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31a01011e0 0x7f31a0101600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:10.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.347+0000 7f31a6b3a640 1 -- 172.21.15.12:0/3483004135 >> 172.21.15.12:0/3483004135 conn(0x7f31a00fc770 msgr2=0x7f31a00feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:10.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.347+0000 7f31a6b3a640 1 -- 172.21.15.12:0/3483004135 shutdown_connections 2024-02-18T11:40:10.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.347+0000 7f31a6b3a640 1 -- 172.21.15.12:0/3483004135 wait complete. 2024-02-18T11:40:10.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.348+0000 7f31a6b3a640 1 Processor -- start 2024-02-18T11:40:10.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.348+0000 7f31a6b3a640 1 -- start start 2024-02-18T11:40:10.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.348+0000 7f31a6b3a640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f31a01011e0 0x7f31a0197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:10.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.348+0000 7f31a6b3a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31a01023d0 0x7f31a01977c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:10.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.348+0000 7f31a6b3a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31a0102d90 0x7f31a019e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:10.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.348+0000 7f31a6b3a640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a019ed70 con 0x7f31a0102d90 2024-02-18T11:40:10.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.348+0000 7f31a6b3a640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a019eee0 con 0x7f31a01011e0 2024-02-18T11:40:10.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.348+0000 7f31a6b3a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a019f0a0 con 0x7f31a01023d0 2024-02-18T11:40:10.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.348+0000 7f31a50d7640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31a0102d90 0x7f31a019e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:10.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.348+0000 7f31a50d7640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31a0102d90 0x7f31a019e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40064/0 (socket says 172.21.15.12:40064) 2024-02-18T11:40:10.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.348+0000 7f31a50d7640 1 -- 172.21.15.12:0/3363054188 learned_addr learned my addr 172.21.15.12:0/3363054188 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:40:10.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.348+0000 7f31a48d6640 1 --2- 172.21.15.12:0/3363054188 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f31a01011e0 0x7f31a0197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:10.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.348+0000 7f319ffff640 1 --2- 172.21.15.12:0/3363054188 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31a01023d0 0x7f31a01977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:10.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.348+0000 7f31a50d7640 1 -- 172.21.15.12:0/3363054188 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31a01023d0 msgr2=0x7f31a01977c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:10.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.348+0000 7f31a50d7640 1 --2- 172.21.15.12:0/3363054188 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31a01023d0 0x7f31a01977c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:10.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.348+0000 7f31a50d7640 1 -- 172.21.15.12:0/3363054188 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f31a01011e0 msgr2=0x7f31a0197280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:10.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.348+0000 7f31a50d7640 1 --2- 172.21.15.12:0/3363054188 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f31a01011e0 0x7f31a0197280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:10.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.348+0000 7f31a50d7640 1 -- 172.21.15.12:0/3363054188 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3198009900 con 0x7f31a0102d90 2024-02-18T11:40:10.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.348+0000 7f319ffff640 1 --2- 172.21.15.12:0/3363054188 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31a01023d0 0x7f31a01977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:40:10.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.348+0000 7f31a50d7640 1 --2- 172.21.15.12:0/3363054188 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31a0102d90 0x7f31a019e830 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f3198032650 tx=0x7f3198034b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:10.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.348+0000 7f319dffb640 1 -- 172.21.15.12:0/3363054188 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31980329b0 con 0x7f31a0102d90 2024-02-18T11:40:10.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.349+0000 7f319dffb640 1 -- 172.21.15.12:0/3363054188 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3198003e50 con 0x7f31a0102d90 2024-02-18T11:40:10.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.349+0000 7f31a6b3a640 1 -- 172.21.15.12:0/3363054188 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31a019f320 con 0x7f31a0102d90 2024-02-18T11:40:10.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.349+0000 7f319dffb640 1 -- 172.21.15.12:0/3363054188 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3198040450 con 0x7f31a0102d90 2024-02-18T11:40:10.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.349+0000 7f31a6b3a640 1 -- 172.21.15.12:0/3363054188 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31a019f7f0 con 0x7f31a0102d90 2024-02-18T11:40:10.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.350+0000 7f31a6b3a640 1 -- 172.21.15.12:0/3363054188 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31a01084a0 con 0x7f31a0102d90 2024-02-18T11:40:10.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.351+0000 7f319dffb640 1 -- 172.21.15.12:0/3363054188 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 137296+0+0 (secure 0 0 0) 0x7f319800a040 con 0x7f31a0102d90 2024-02-18T11:40:10.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.351+0000 7f319dffb640 1 --2- 172.21.15.12:0/3363054188 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f31900a2860 0x7f31900a4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:10.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.351+0000 7f31a48d6640 1 --2- 172.21.15.12:0/3363054188 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f31900a2860 0x7f31900a4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:10.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.352+0000 7f319dffb640 1 -- 172.21.15.12:0/3363054188 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f31980f1450 con 0x7f31a0102d90 2024-02-18T11:40:10.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.352+0000 7f31a48d6640 1 --2- 172.21.15.12:0/3363054188 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f31900a2860 0x7f31900a4d20 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3194005fd0 tx=0x7f3194005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:10.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.356+0000 7f319dffb640 1 -- 172.21.15.12:0/3363054188 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f31980ba140 con 0x7f31a0102d90 2024-02-18T11:40:10.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:40:10 smithi188 ceph-mon[36925]: pgmap v328: 97 pgs: 97 active+clean; 639 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-18T11:40:10.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:40:10 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/3774046745' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-18T11:40:10.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:40:10 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/4242788693' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-18T11:40:10.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:40:10 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/3330144362' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-18T11:40:10.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:40:10 smithi158 ceph-mon[36626]: pgmap v328: 97 pgs: 97 active+clean; 639 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-18T11:40:10.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:40:10 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/3774046745' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-18T11:40:10.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:40:10 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/4242788693' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-18T11:40:10.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:40:10 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/3330144362' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-18T11:40:10.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.538+0000 7f31a6b3a640 1 -- 172.21.15.12:0/3363054188 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f31a0107f10 con 0x7f31a0102d90 2024-02-18T11:40:10.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.539+0000 7f319dffb640 1 -- 172.21.15.12:0/3363054188 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+22662 (secure 0 0 0) 0x7f31980bd800 con 0x7f31a0102d90 2024-02-18T11:40:10.534 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:40:10.534 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":89,"fsid":"0c8bb0de-ce51-11ee-95ba-87774f69a715","created":"2024-02-18T11:30:49.110059+0000","modified":"2024-02-18T11:39:52.478480+0000","last_up_change":"2024-02-18T11:37:57.512337+0000","last_in_change":"2024-02-18T11:37:36.567969+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-18T11:34:15.281138+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-18T11:38:44.802134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-18T11:38:46.081188+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"35f15ea6-deb8-48a1-a7dd-ab649ec13533","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":536522426},{"type":"v1","addr":"172.21.15.12:6803","nonce":536522426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":536522426},{"type":"v1","addr":"172.21.15.12:6805","nonce":536522426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":536522426},{"type":"v1","addr":"172.21.15.12:6809","nonce":536522426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":536522426},{"type":"v1","addr":"172.21.15.12:6807","nonce":536522426}]},"public_addr":"172.21.15.12:6803/536522426","cluster_addr":"172.21.15.12:6805/536522426","heartbeat_back_addr":"172.21.15.12:6809/536522426","heartbeat_front_addr":"172.21.15.12:6807/536522426","state":["exists","up"]},{"osd":1,"uuid":"2af937ef-d827-44e6-b805-70cffbfe1126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":4249111488},{"type":"v1","addr":"172.21.15.158:6801","nonce":4249111488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":4249111488},{"type":"v1","addr":"172.21.15.158:6803","nonce":4249111488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":4249111488},{"type":"v1","addr":"172.21.15.158:6807","nonce":4249111488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":4249111488},{"type":"v1","addr":"172.21.15.158:6805","nonce":4249111488}]},"public_addr":"172.21.15.158:6801/4249111488","cluster_addr":"172.21.15.158:6803/4249111488","heartbeat_back_addr":"172.21.15.158:6807/4249111488","heartbeat_front_addr":"172.21.15.158:6805/4249111488","state":["exists","up"]},{"osd":2,"uuid":"17b804dd-31df-45ae-a637-4a5f65bbaf26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":627976150},{"type":"v1","addr":"172.21.15.188:6801","nonce":627976150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":627976150},{"type":"v1","addr":"172.21.15.188:6803","nonce":627976150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":627976150},{"type":"v1","addr":"172.21.15.188:6807","nonce":627976150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":627976150},{"type":"v1","addr":"172.21.15.188:6805","nonce":627976150}]},"public_addr":"172.21.15.188:6801/627976150","cluster_addr":"172.21.15.188:6803/627976150","heartbeat_back_addr":"172.21.15.188:6807/627976150","heartbeat_front_addr":"172.21.15.188:6805/627976150","state":["exists","up"]},{"osd":3,"uuid":"3f9cc423-cdfa-4acf-aaa2-b58b32ccb9cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":3758512503},{"type":"v1","addr":"172.21.15.12:6811","nonce":3758512503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":3758512503},{"type":"v1","addr":"172.21.15.12:6813","nonce":3758512503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3758512503},{"type":"v1","addr":"172.21.15.12:6817","nonce":3758512503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":3758512503},{"type":"v1","addr":"172.21.15.12:6815","nonce":3758512503}]},"public_addr":"172.21.15.12:6811/3758512503","cluster_addr":"172.21.15.12:6813/3758512503","heartbeat_back_addr":"172.21.15.12:6817/3758512503","heartbeat_front_addr":"172.21.15.12:6815/3758512503","state":["exists","up"]},{"osd":4,"uuid":"a92e20ec-af1c-4ef4-a0cb-caba5fcc71aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":3892820997},{"type":"v1","addr":"172.21.15.158:6809","nonce":3892820997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":3892820997},{"type":"v1","addr":"172.21.15.158:6811","nonce":3892820997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":3892820997},{"type":"v1","addr":"172.21.15.158:6815","nonce":3892820997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":3892820997},{"type":"v1","addr":"172.21.15.158:6813","nonce":3892820997}]},"public_addr":"172.21.15.158:6809/3892820997","cluster_addr":"172.21.15.158:6811/3892820997","heartbeat_back_addr":"172.21.15.158:6815/3892820997","heartbeat_front_addr":"172.21.15.158:6813/3892820997","state":["exists","up"]},{"osd":5,"uuid":"d9c2e0c8-85f5-4ae2-bf3e-125c1be3a2db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3939495896},{"type":"v1","addr":"172.21.15.188:6809","nonce":3939495896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3939495896},{"type":"v1","addr":"172.21.15.188:6811","nonce":3939495896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3939495896},{"type":"v1","addr":"172.21.15.188:6815","nonce":3939495896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3939495896},{"type":"v1","addr":"172.21.15.188:6813","nonce":3939495896}]},"public_addr":"172.21.15.188:6809/3939495896","cluster_addr":"172.21.15.188:6811/3939495896","heartbeat_back_addr":"172.21.15.188:6815/3939495896","heartbeat_front_addr":"172.21.15.188:6813/3939495896","state":["exists","up"]},{"osd":6,"uuid":"4a523512-de6b-49f0-be81-00f2f00c1c10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":1206337767},{"type":"v1","addr":"172.21.15.12:6819","nonce":1206337767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":1206337767},{"type":"v1","addr":"172.21.15.12:6821","nonce":1206337767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":1206337767},{"type":"v1","addr":"172.21.15.12:6825","nonce":1206337767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":1206337767},{"type":"v1","addr":"172.21.15.12:6823","nonce":1206337767}]},"public_addr":"172.21.15.12:6819/1206337767","cluster_addr":"172.21.15.12:6821/1206337767","heartbeat_back_addr":"172.21.15.12:6825/1206337767","heartbeat_front_addr":"172.21.15.12:6823/1206337767","state":["exists","up"]},{"osd":7,"uuid":"363271a6-df93-44c9-b0e5-3d29768aa96c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":3283400032},{"type":"v1","addr":"172.21.15.158:6817","nonce":3283400032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":3283400032},{"type":"v1","addr":"172.21.15.158:6819","nonce":3283400032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":3283400032},{"type":"v1","addr":"172.21.15.158:6823","nonce":3283400032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":3283400032},{"type":"v1","addr":"172.21.15.158:6821","nonce":3283400032}]},"public_addr":"172.21.15.158:6817/3283400032","cluster_addr":"172.21.15.158:6819/3283400032","heartbeat_back_addr":"172.21.15.158:6823/3283400032","heartbeat_front_addr":"172.21.15.158:6821/3283400032","state":["exists","up"]},{"osd":8,"uuid":"9a089528-3e8e-45d3-9279-ddb54088b96a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2110170682},{"type":"v1","addr":"172.21.15.188:6817","nonce":2110170682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2110170682},{"type":"v1","addr":"172.21.15.188:6819","nonce":2110170682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":2110170682},{"type":"v1","addr":"172.21.15.188:6823","nonce":2110170682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2110170682},{"type":"v1","addr":"172.21.15.188:6821","nonce":2110170682}]},"public_addr":"172.21.15.188:6817/2110170682","cluster_addr":"172.21.15.188:6819/2110170682","heartbeat_back_addr":"172.21.15.188:6823/2110170682","heartbeat_front_addr":"172.21.15.188:6821/2110170682","state":["exists","up"]},{"osd":9,"uuid":"b12e2727-d649-4464-90e1-fafbc1754832","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1825928986},{"type":"v1","addr":"172.21.15.12:6827","nonce":1825928986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1825928986},{"type":"v1","addr":"172.21.15.12:6829","nonce":1825928986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":1825928986},{"type":"v1","addr":"172.21.15.12:6833","nonce":1825928986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1825928986},{"type":"v1","addr":"172.21.15.12:6831","nonce":1825928986}]},"public_addr":"172.21.15.12:6827/1825928986","cluster_addr":"172.21.15.12:6829/1825928986","heartbeat_back_addr":"172.21.15.12:6833/1825928986","heartbeat_front_addr":"172.21.15.12:6831/1825928986","state":["exists","up"]},{"osd":10,"uuid":"518b4cbf-9d90-4e9a-8ac9-b5cb9b3dee4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":3239052167},{"type":"v1","addr":"172.21.15.158:6825","nonce":3239052167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":3239052167},{"type":"v1","addr":"172.21.15.158:6827","nonce":3239052167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":3239052167},{"type":"v1","addr":"172.21.15.158:6831","nonce":3239052167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":3239052167},{"type":"v1","addr":"172.21.15.158:6829","nonce":3239052167}]},"public_addr":"172.21.15.158:6825/3239052167","cluster_addr":"172.21.15.158:6827/3239052167","heartbeat_back_addr":"172.21.15.158:6831/3239052167","heartbeat_front_addr":"172.21.15.158:6829/3239052167","state":["exists","up"]},{"osd":11,"uuid":"98aa3a0d-4b2f-46c8-9341-375d13794c3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1135773350},{"type":"v1","addr":"172.21.15.188:6825","nonce":1135773350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1135773350},{"type":"v1","addr":"172.21.15.188:6827","nonce":1135773350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1135773350},{"type":"v1","addr":"172.21.15.188:6831","nonce":1135773350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1135773350},{"type":"v1","addr":"172.21.15.188:6829","nonce":1135773350}]},"public_addr":"172.21.15.188:6825/1135773350","cluster_addr":"172.21.15.188:6827/1135773350","heartbeat_back_addr":"172.21.15.188:6831/1135773350","heartbeat_front_addr":"172.21.15.188:6829/1135773350","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:33:24.280879+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:33:47.859461+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:34:11.725283+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:34:35.707629+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:35:00.024197+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:35:24.499803+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:35:48.953870+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:36:13.642456+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:36:38.936457+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:37:04.021466+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:37:29.073178+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:37:54.863919+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.37","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:6801/4135274411":"2024-02-19T11:31:50.936928+0000","172.21.15.12:6800/4135274411":"2024-02-19T11:31:50.936928+0000","172.21.15.12:0/118428584":"2024-02-19T11:31:50.936928+0000","172.21.15.12:0/4276810915":"2024-02-19T11:31:50.936928+0000","172.21.15.12:0/3777545262":"2024-02-19T11:31:24.000830+0000","172.21.15.12:0/114707743":"2024-02-19T11:31:24.000830+0000","172.21.15.12:6800/3925204723":"2024-02-19T11:31:24.000830+0000","172.21.15.12:6801/3925204723":"2024-02-19T11:31:24.000830+0000","172.21.15.12:0/74881618":"2024-02-19T11:31:24.000830+0000","172.21.15.12:0/1614805375":"2024-02-19T11:31:50.936928+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-18T11:40:10.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.541+0000 7f31a6b3a640 1 -- 172.21.15.12:0/3363054188 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f31900a2860 msgr2=0x7f31900a4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:10.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.541+0000 7f31a6b3a640 1 --2- 172.21.15.12:0/3363054188 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f31900a2860 0x7f31900a4d20 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3194005fd0 tx=0x7f3194005850 comp rx=0 tx=0).stop 2024-02-18T11:40:10.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.541+0000 7f31a6b3a640 1 -- 172.21.15.12:0/3363054188 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31a0102d90 msgr2=0x7f31a019e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:10.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.541+0000 7f31a6b3a640 1 --2- 172.21.15.12:0/3363054188 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31a0102d90 0x7f31a019e830 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f3198032650 tx=0x7f3198034b10 comp rx=0 tx=0).stop 2024-02-18T11:40:10.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.541+0000 7f31a6b3a640 1 -- 172.21.15.12:0/3363054188 shutdown_connections 2024-02-18T11:40:10.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.541+0000 7f31a6b3a640 1 --2- 172.21.15.12:0/3363054188 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f31900a2860 0x7f31900a4d20 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:10.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.541+0000 7f31a6b3a640 1 --2- 172.21.15.12:0/3363054188 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31a01023d0 0x7f31a01977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:10.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.541+0000 7f31a6b3a640 1 --2- 172.21.15.12:0/3363054188 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31a0102d90 0x7f31a019e830 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:10.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.541+0000 7f31a6b3a640 1 --2- 172.21.15.12:0/3363054188 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f31a01011e0 0x7f31a0197280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:10.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.541+0000 7f31a6b3a640 1 -- 172.21.15.12:0/3363054188 >> 172.21.15.12:0/3363054188 conn(0x7f31a00fc770 msgr2=0x7f31a0105670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:10.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.541+0000 7f31a6b3a640 1 -- 172.21.15.12:0/3363054188 shutdown_connections 2024-02-18T11:40:10.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.541+0000 7f31a6b3a640 1 -- 172.21.15.12:0/3363054188 wait complete. 2024-02-18T11:40:10.547 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-18T11:40:10.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.655+0000 7f0767476640 1 -- 172.21.15.12:0/2123792675 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0760101200 msgr2=0x7f0760101620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:10.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.655+0000 7f0767476640 1 --2- 172.21.15.12:0/2123792675 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0760101200 0x7f0760101620 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f075c009900 tx=0x7f075c02bc30 comp rx=0 tx=0).stop 2024-02-18T11:40:10.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.655+0000 7f0767476640 1 -- 172.21.15.12:0/2123792675 shutdown_connections 2024-02-18T11:40:10.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.655+0000 7f0767476640 1 --2- 172.21.15.12:0/2123792675 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0760102db0 0x7f0760109640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:10.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.655+0000 7f0767476640 1 --2- 172.21.15.12:0/2123792675 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f07601023f0 0x7f0760102870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:10.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.655+0000 7f0767476640 1 --2- 172.21.15.12:0/2123792675 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0760101200 0x7f0760101620 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:10.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.655+0000 7f0767476640 1 -- 172.21.15.12:0/2123792675 >> 172.21.15.12:0/2123792675 conn(0x7f07600fc770 msgr2=0x7f07600febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:10.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.655+0000 7f0767476640 1 -- 172.21.15.12:0/2123792675 shutdown_connections 2024-02-18T11:40:10.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.655+0000 7f0767476640 1 -- 172.21.15.12:0/2123792675 wait complete. 2024-02-18T11:40:10.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.656+0000 7f0767476640 1 Processor -- start 2024-02-18T11:40:10.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.656+0000 7f0767476640 1 -- start start 2024-02-18T11:40:10.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.656+0000 7f0767476640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0760101200 0x7f0760074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:10.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.656+0000 7f0767476640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f07601023f0 0x7f07600735b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:10.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.656+0000 7f0767476640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0760102db0 0x7f0760073af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:10.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.656+0000 7f0767476640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0760074250 con 0x7f0760102db0 2024-02-18T11:40:10.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.656+0000 7f0767476640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07600743c0 con 0x7f07601023f0 2024-02-18T11:40:10.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.656+0000 7f0767476640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07601a1330 con 0x7f0760101200 2024-02-18T11:40:10.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.656+0000 7f0765a13640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0760102db0 0x7f0760073af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:10.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.656+0000 7f0765a13640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0760102db0 0x7f0760073af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40090/0 (socket says 172.21.15.12:40090) 2024-02-18T11:40:10.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.656+0000 7f0765a13640 1 -- 172.21.15.12:0/4203948026 learned_addr learned my addr 172.21.15.12:0/4203948026 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:40:10.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.656+0000 7f0765212640 1 --2- 172.21.15.12:0/4203948026 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0760101200 0x7f0760074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:10.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.656+0000 7f0764a11640 1 --2- 172.21.15.12:0/4203948026 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f07601023f0 0x7f07600735b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:10.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.656+0000 7f0765a13640 1 -- 172.21.15.12:0/4203948026 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0760101200 msgr2=0x7f0760074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:10.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.656+0000 7f0765a13640 1 --2- 172.21.15.12:0/4203948026 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0760101200 0x7f0760074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:10.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.656+0000 7f0765a13640 1 -- 172.21.15.12:0/4203948026 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f07601023f0 msgr2=0x7f07600735b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:10.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.656+0000 7f0765a13640 1 --2- 172.21.15.12:0/4203948026 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f07601023f0 0x7f07600735b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:10.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.656+0000 7f0765a13640 1 -- 172.21.15.12:0/4203948026 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f075c009560 con 0x7f0760102db0 2024-02-18T11:40:10.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.656+0000 7f0764a11640 1 --2- 172.21.15.12:0/4203948026 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f07601023f0 0x7f07600735b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:40:10.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.657+0000 7f0765a13640 1 --2- 172.21.15.12:0/4203948026 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0760102db0 0x7f0760073af0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f075800eb40 tx=0x7f075800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:10.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.657+0000 7f07567fc640 1 -- 172.21.15.12:0/4203948026 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0758004b20 con 0x7f0760102db0 2024-02-18T11:40:10.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.657+0000 7f0767476640 1 -- 172.21.15.12:0/4203948026 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07601a15e0 con 0x7f0760102db0 2024-02-18T11:40:10.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.657+0000 7f07567fc640 1 -- 172.21.15.12:0/4203948026 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0758004c80 con 0x7f0760102db0 2024-02-18T11:40:10.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.657+0000 7f07567fc640 1 -- 172.21.15.12:0/4203948026 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0758010690 con 0x7f0760102db0 2024-02-18T11:40:10.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.657+0000 7f0767476640 1 -- 172.21.15.12:0/4203948026 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07601a1b40 con 0x7f0760102db0 2024-02-18T11:40:10.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.658+0000 7f0767476640 1 -- 172.21.15.12:0/4203948026 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0760108540 con 0x7f0760102db0 2024-02-18T11:40:10.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.659+0000 7f07567fc640 1 -- 172.21.15.12:0/4203948026 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 137296+0+0 (secure 0 0 0) 0x7f0758004df0 con 0x7f0760102db0 2024-02-18T11:40:10.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.659+0000 7f07567fc640 1 --2- 172.21.15.12:0/4203948026 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f074c0a27e0 0x7f074c0a4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:10.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.660+0000 7f0765212640 1 --2- 172.21.15.12:0/4203948026 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f074c0a27e0 0x7f074c0a4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:10.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.660+0000 7f07567fc640 1 -- 172.21.15.12:0/4203948026 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f0758014070 con 0x7f0760102db0 2024-02-18T11:40:10.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.660+0000 7f0765212640 1 --2- 172.21.15.12:0/4203948026 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f074c0a27e0 0x7f074c0a4ca0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f075c005ab0 tx=0x7f075c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:10.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.663+0000 7f07567fc640 1 -- 172.21.15.12:0/4203948026 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0758097d40 con 0x7f0760102db0 2024-02-18T11:40:10.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.883+0000 7f0767476640 1 -- 172.21.15.12:0/4203948026 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f076002ce80 con 0x7f0760102db0 2024-02-18T11:40:10.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.884+0000 7f07567fc640 1 -- 172.21.15.12:0/4203948026 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11349 (secure 0 0 0) 0x7f075809b400 con 0x7f0760102db0 2024-02-18T11:40:10.878 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:40:10.878 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1788335302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1788335302},{"type":"v1","addr":"172.21.15.12:6835","nonce":1788335302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/2144692455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":2144692455},{"type":"v1","addr":"172.21.15.12:6837","nonce":2144692455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3569056036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3569056036},{"type":"v1","addr":"172.21.15.12:6839","nonce":3569056036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14622,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/1973234486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":1973234486},{"type":"v1","addr":"172.21.15.12:6841","nonce":1973234486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1207202465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1207202465},{"type":"v1","addr":"172.21.15.188:6833","nonce":1207202465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6833/2636348323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6832","nonce":2636348323},{"type":"v1","addr":"172.21.15.158:6833","nonce":2636348323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-18T11:38:46.888173+0000","modified":"2024-02-18T11:38:51.531552+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24524,"mds_1":24508,"mds_2":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.158:6835/3673253642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":3673253642},{"type":"v1","addr":"172.21.15.158:6835","nonce":3673253642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/806709644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":806709644},{"type":"v1","addr":"172.21.15.188:6835","nonce":806709644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.158:6837/813251825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6836","nonce":813251825},{"type":"v1","addr":"172.21.15.158:6837","nonce":813251825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.188:6837/1036630143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1036630143},{"type":"v1","addr":"172.21.15.188:6837","nonce":1036630143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.188:6839/1342671190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":1342671190},{"type":"v1","addr":"172.21.15.188:6839","nonce":1342671190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.158:6839/1127200849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6838","nonce":1127200849},{"type":"v1","addr":"172.21.15.158:6839","nonce":1127200849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-18T11:40:10.878 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 20 2024-02-18T11:40:10.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.886+0000 7f0767476640 1 -- 172.21.15.12:0/4203948026 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f074c0a27e0 msgr2=0x7f074c0a4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:10.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.886+0000 7f0767476640 1 --2- 172.21.15.12:0/4203948026 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f074c0a27e0 0x7f074c0a4ca0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f075c005ab0 tx=0x7f075c005fd0 comp rx=0 tx=0).stop 2024-02-18T11:40:10.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.886+0000 7f0767476640 1 -- 172.21.15.12:0/4203948026 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0760102db0 msgr2=0x7f0760073af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:10.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.886+0000 7f0767476640 1 --2- 172.21.15.12:0/4203948026 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0760102db0 0x7f0760073af0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f075800eb40 tx=0x7f075800c2d0 comp rx=0 tx=0).stop 2024-02-18T11:40:10.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.886+0000 7f0767476640 1 -- 172.21.15.12:0/4203948026 shutdown_connections 2024-02-18T11:40:10.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.886+0000 7f0767476640 1 --2- 172.21.15.12:0/4203948026 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f074c0a27e0 0x7f074c0a4ca0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:10.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.886+0000 7f0767476640 1 --2- 172.21.15.12:0/4203948026 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0760102db0 0x7f0760073af0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:10.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.886+0000 7f0767476640 1 --2- 172.21.15.12:0/4203948026 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f07601023f0 0x7f07600735b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:10.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.886+0000 7f0767476640 1 --2- 172.21.15.12:0/4203948026 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0760101200 0x7f0760074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:10.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.886+0000 7f0767476640 1 -- 172.21.15.12:0/4203948026 >> 172.21.15.12:0/4203948026 conn(0x7f07600fc770 msgr2=0x7f0760105710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:10.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.886+0000 7f0767476640 1 -- 172.21.15.12:0/4203948026 shutdown_connections 2024-02-18T11:40:10.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:10.886+0000 7f0767476640 1 -- 172.21.15.12:0/4203948026 wait complete. 2024-02-18T11:40:10.892 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 20, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-02-18T11:38:46.888173+0000', 'modified': '2024-02-18T11:38:51.531552+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24524, 'mds_1': 24508, 'mds_2': 24478}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24446': {'gid': 24446, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.158:6835/3673253642', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.158:6834', 'nonce': 3673253642}, {'type': 'v1', 'addr': '172.21.15.158:6835', 'nonce': 3673253642}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24466': {'gid': 24466, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.188:6835/806709644', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6834', 'nonce': 806709644}, {'type': 'v1', 'addr': '172.21.15.188:6835', 'nonce': 806709644}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24476': {'gid': 24476, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.158:6837/813251825', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.158:6836', 'nonce': 813251825}, {'type': 'v1', 'addr': '172.21.15.158:6837', 'nonce': 813251825}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24478': {'gid': 24478, 'name': 'c', 'rank': 2, 'incarnation': 19, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.188:6837/1036630143', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6836', 'nonce': 1036630143}, {'type': 'v1', 'addr': '172.21.15.188:6837', 'nonce': 1036630143}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24508': {'gid': 24508, 'name': 'i', 'rank': 1, 'incarnation': 17, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.188:6839/1342671190', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6838', 'nonce': 1342671190}, {'type': 'v1', 'addr': '172.21.15.188:6839', 'nonce': 1342671190}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24524': {'gid': 24524, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.158:6839/1127200849', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.158:6838', 'nonce': 1127200849}, {'type': 'v1', 'addr': '172.21.15.158:6839', 'nonce': 1127200849}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-02-18T11:40:10.892 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-02-18T11:40:10.892 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-02-18T11:40:10.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.002+0000 7f514b039640 1 -- 172.21.15.12:0/113499705 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f51441011d0 msgr2=0x7f51441015f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:10.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.002+0000 7f514b039640 1 --2- 172.21.15.12:0/113499705 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f51441011d0 0x7f51441015f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5138009900 tx=0x7f513802bc30 comp rx=0 tx=0).stop 2024-02-18T11:40:10.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.002+0000 7f514b039640 1 -- 172.21.15.12:0/113499705 shutdown_connections 2024-02-18T11:40:10.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.002+0000 7f514b039640 1 --2- 172.21.15.12:0/113499705 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51441023c0 0x7f5144102840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:10.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.002+0000 7f514b039640 1 --2- 172.21.15.12:0/113499705 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5144102d80 0x7f5144109610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:10.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.002+0000 7f514b039640 1 --2- 172.21.15.12:0/113499705 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f51441011d0 0x7f51441015f0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:10.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.002+0000 7f514b039640 1 -- 172.21.15.12:0/113499705 >> 172.21.15.12:0/113499705 conn(0x7f51440fc760 msgr2=0x7f51440feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:10.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.002+0000 7f514b039640 1 -- 172.21.15.12:0/113499705 shutdown_connections 2024-02-18T11:40:10.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.002+0000 7f514b039640 1 -- 172.21.15.12:0/113499705 wait complete. 2024-02-18T11:40:10.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.002+0000 7f514b039640 1 Processor -- start 2024-02-18T11:40:10.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.002+0000 7f514b039640 1 -- start start 2024-02-18T11:40:10.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.002+0000 7f514b039640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f51441011d0 0x7f51441972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:10.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.002+0000 7f514b039640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51441023c0 0x7f51441977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:10.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.002+0000 7f514b039640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5144102d80 0x7f514419e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:10.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.003+0000 7f514b039640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f514419ed90 con 0x7f5144102d80 2024-02-18T11:40:10.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.003+0000 7f514b039640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f514419ef00 con 0x7f51441011d0 2024-02-18T11:40:10.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.003+0000 7f514b039640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f514419f0c0 con 0x7f51441023c0 2024-02-18T11:40:10.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.003+0000 7f51495d6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5144102d80 0x7f514419e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:11.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.003+0000 7f51495d6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5144102d80 0x7f514419e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40112/0 (socket says 172.21.15.12:40112) 2024-02-18T11:40:11.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.003+0000 7f51495d6640 1 -- 172.21.15.12:0/784085644 learned_addr learned my addr 172.21.15.12:0/784085644 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:40:11.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.003+0000 7f5148dd5640 1 --2- 172.21.15.12:0/784085644 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f51441011d0 0x7f51441972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:11.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.003+0000 7f5143fff640 1 --2- 172.21.15.12:0/784085644 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51441023c0 0x7f51441977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:11.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.003+0000 7f51495d6640 1 -- 172.21.15.12:0/784085644 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51441023c0 msgr2=0x7f51441977e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:11.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.003+0000 7f51495d6640 1 --2- 172.21.15.12:0/784085644 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51441023c0 0x7f51441977e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.003+0000 7f51495d6640 1 -- 172.21.15.12:0/784085644 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f51441011d0 msgr2=0x7f51441972a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:11.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.003+0000 7f51495d6640 1 --2- 172.21.15.12:0/784085644 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f51441011d0 0x7f51441972a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.003+0000 7f51495d6640 1 -- 172.21.15.12:0/784085644 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5138009560 con 0x7f5144102d80 2024-02-18T11:40:11.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.003+0000 7f51495d6640 1 --2- 172.21.15.12:0/784085644 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5144102d80 0x7f514419e850 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f513c00eaa0 tx=0x7f513c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:11.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.003+0000 7f5141ffb640 1 -- 172.21.15.12:0/784085644 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f513c00cdb0 con 0x7f5144102d80 2024-02-18T11:40:11.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.003+0000 7f5141ffb640 1 -- 172.21.15.12:0/784085644 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f513c005020 con 0x7f5144102d80 2024-02-18T11:40:11.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.003+0000 7f514b039640 1 -- 172.21.15.12:0/784085644 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f514419f3a0 con 0x7f5144102d80 2024-02-18T11:40:11.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.003+0000 7f5141ffb640 1 -- 172.21.15.12:0/784085644 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f513c0116b0 con 0x7f5144102d80 2024-02-18T11:40:11.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.003+0000 7f514b039640 1 -- 172.21.15.12:0/784085644 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f514419f860 con 0x7f5144102d80 2024-02-18T11:40:11.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.005+0000 7f5141ffb640 1 -- 172.21.15.12:0/784085644 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 137296+0+0 (secure 0 0 0) 0x7f513c005520 con 0x7f5144102d80 2024-02-18T11:40:11.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.006+0000 7f514dc07740 1 Processor -- start 2024-02-18T11:40:11.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.006+0000 7f514dc07740 1 -- start start 2024-02-18T11:40:11.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.006+0000 7f5141ffb640 1 --2- 172.21.15.12:0/784085644 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f51340a27e0 0x7f51340a4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:11.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.006+0000 7f514dc07740 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55df60ed0070 0x55df60ed0490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:11.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.006+0000 7f514dc07740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55df60ed0a60 0x55df60ed5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:11.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.006+0000 7f514dc07740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df60ed5bd0 0x55df60ed8060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:11.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.006+0000 7f514dc07740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df60e8bdc0 con 0x55df60ed5bd0 2024-02-18T11:40:11.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.006+0000 7f514dc07740 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df60e3ef20 con 0x55df60ed0070 2024-02-18T11:40:11.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.006+0000 7f514dc07740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df60e3c450 con 0x55df60ed0a60 2024-02-18T11:40:11.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.006+0000 7f5143fff640 1 --2- 172.21.15.12:0/784085644 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f51340a27e0 0x7f51340a4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:11.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.006+0000 7f51495d6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df60ed5bd0 0x55df60ed8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:11.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.006+0000 7f51495d6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df60ed5bd0 0x55df60ed8060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40114/0 (socket says 172.21.15.12:40114) 2024-02-18T11:40:11.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.006+0000 7f51495d6640 1 -- 172.21.15.12:0/480953460 learned_addr learned my addr 172.21.15.12:0/480953460 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:40:11.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.006+0000 7f5141ffb640 1 -- 172.21.15.12:0/784085644 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f513c0cea20 con 0x7f5144102d80 2024-02-18T11:40:11.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.006+0000 7f5148dd5640 1 --2- 172.21.15.12:0/480953460 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55df60ed0070 0x55df60ed0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:11.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.006+0000 7f5143fff640 1 --2- 172.21.15.12:0/480953460 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55df60ed0a60 0x55df60ed5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:11.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.006+0000 7f51495d6640 1 -- 172.21.15.12:0/480953460 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55df60ed0a60 msgr2=0x55df60ed5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:40:11.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.006+0000 7f51495d6640 1 --2- 172.21.15.12:0/480953460 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55df60ed0a60 0x55df60ed5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.006+0000 7f51495d6640 1 -- 172.21.15.12:0/480953460 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55df60ed0070 msgr2=0x55df60ed0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:40:11.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.006+0000 7f51495d6640 1 --2- 172.21.15.12:0/480953460 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55df60ed0070 0x55df60ed0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.006+0000 7f51495d6640 1 -- 172.21.15.12:0/480953460 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df60decf20 con 0x55df60ed5bd0 2024-02-18T11:40:11.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.006+0000 7f5143fff640 1 --2- 172.21.15.12:0/784085644 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f51340a27e0 0x7f51340a4ca0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5130005d90 tx=0x7f513000aa70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:11.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.007+0000 7f51495d6640 1 --2- 172.21.15.12:0/480953460 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df60ed5bd0 0x55df60ed8060 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f513c09b2f0 tx=0x7f513c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f536deadf5dbac6 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:11.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.007+0000 7f512effd640 1 -- 172.21.15.12:0/480953460 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f513c09ba60 con 0x55df60ed5bd0 2024-02-18T11:40:11.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.007+0000 7f512effd640 1 -- 172.21.15.12:0/480953460 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f513c09b360 con 0x55df60ed5bd0 2024-02-18T11:40:11.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.007+0000 7f512effd640 1 -- 172.21.15.12:0/480953460 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f513c0cc5d0 con 0x55df60ed5bd0 2024-02-18T11:40:11.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.007+0000 7f514dc07740 1 -- 172.21.15.12:0/480953460 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df60ed5bd0 msgr2=0x55df60ed8060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:11.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.007+0000 7f514dc07740 1 --2- 172.21.15.12:0/480953460 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df60ed5bd0 0x55df60ed8060 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f513c09b2f0 tx=0x7f513c00cf90 comp rx=0 tx=0).stop 2024-02-18T11:40:11.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.007+0000 7f514dc07740 1 -- 172.21.15.12:0/480953460 shutdown_connections 2024-02-18T11:40:11.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.007+0000 7f514dc07740 1 --2- 172.21.15.12:0/480953460 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55df60ed0a60 0x55df60ed5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.007+0000 7f514dc07740 1 --2- 172.21.15.12:0/480953460 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df60ed5bd0 0x55df60ed8060 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.007+0000 7f514dc07740 1 --2- 172.21.15.12:0/480953460 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55df60ed0070 0x55df60ed0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.007+0000 7f514dc07740 1 -- 172.21.15.12:0/480953460 >> 172.21.15.12:0/480953460 conn(0x55df60ecb5e0 msgr2=0x55df60ecb9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:11.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.007+0000 7f514dc07740 1 -- 172.21.15.12:0/480953460 shutdown_connections 2024-02-18T11:40:11.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.007+0000 7f514dc07740 1 -- 172.21.15.12:0/480953460 wait complete. 2024-02-18T11:40:11.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.008+0000 7f514dc07740 1 -- start start 2024-02-18T11:40:11.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.008+0000 7f514dc07740 1 Processor -- start 2024-02-18T11:40:11.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.008+0000 7f514dc07740 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55df60ed0070 0x55df60eda160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:11.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.008+0000 7f514dc07740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df60ed0a60 0x55df60eda6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:11.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.008+0000 7f514dc07740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55df60ed5bd0 0x55df60ee16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:11.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.008+0000 7f514dc07740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df60e37c30 con 0x55df60ed0a60 2024-02-18T11:40:11.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.008+0000 7f514dc07740 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df60ddcd30 con 0x55df60ed0070 2024-02-18T11:40:11.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.008+0000 7f514dc07740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df60b0b310 con 0x55df60ed5bd0 2024-02-18T11:40:11.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.008+0000 7f514dc07740 10 client.0 _ll_register_callbacks cb 0x55df60de4d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-18T11:40:11.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.008+0000 7f5143fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df60ed0a60 0x55df60eda6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:11.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.008+0000 7f5143fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df60ed0a60 0x55df60eda6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40118/0 (socket says 172.21.15.12:40118) 2024-02-18T11:40:11.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.008+0000 7f5143fff640 1 -- 172.21.15.12:0/2799605983 learned_addr learned my addr 172.21.15.12:0/2799605983 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:40:11.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.008+0000 7f51495d6640 1 --2- 172.21.15.12:0/2799605983 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55df60ed5bd0 0x55df60ee16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:11.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.008+0000 7f5148dd5640 1 --2- 172.21.15.12:0/2799605983 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55df60ed0070 0x55df60eda160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:11.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.008+0000 7f5143fff640 1 -- 172.21.15.12:0/2799605983 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55df60ed5bd0 msgr2=0x55df60ee16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:40:11.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.008+0000 7f5143fff640 1 --2- 172.21.15.12:0/2799605983 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55df60ed5bd0 0x55df60ee16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.008+0000 7f5143fff640 1 -- 172.21.15.12:0/2799605983 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55df60ed0070 msgr2=0x55df60eda160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:40:11.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.008+0000 7f5143fff640 1 --2- 172.21.15.12:0/2799605983 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55df60ed0070 0x55df60eda160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.009+0000 7f5143fff640 1 -- 172.21.15.12:0/2799605983 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f514419f860 con 0x55df60ed0a60 2024-02-18T11:40:11.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.009+0000 7f5148dd5640 1 --2- 172.21.15.12:0/2799605983 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55df60ed0070 0x55df60eda160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:40:11.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.009+0000 7f5143fff640 1 --2- 172.21.15.12:0/2799605983 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df60ed0a60 0x55df60eda6a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f5130010ba0 tx=0x7f5130004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec236d08e30ce3f5 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:11.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.009+0000 7f5123fff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-02-18T11:40:11.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.009+0000 7f5123fff640 1 -- 172.21.15.12:0/2799605983 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5130005020 con 0x55df60ed0a60 2024-02-18T11:40:11.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.009+0000 7f5123fff640 1 -- 172.21.15.12:0/2799605983 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5130005180 con 0x55df60ed0a60 2024-02-18T11:40:11.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.009+0000 7f514dc07740 1 -- 172.21.15.12:0/2799605983 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55df60b376c0 con 0x55df60ed0a60 2024-02-18T11:40:11.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.009+0000 7f5123fff640 1 -- 172.21.15.12:0/2799605983 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5130005340 con 0x55df60ed0a60 2024-02-18T11:40:11.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.009+0000 7f514dc07740 10 client.15036 fetch_fsmap learned FSMap version 20 2024-02-18T11:40:11.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.009+0000 7f514dc07740 1 -- 172.21.15.12:0/2799605983 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55df60e362b0 con 0x55df60ed0a60 2024-02-18T11:40:11.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.009+0000 7f5123fff640 1 -- 172.21.15.12:0/2799605983 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55df60b376c0 con 0x55df60ed0a60 2024-02-18T11:40:11.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.009+0000 7f5123fff640 1 -- 172.21.15.12:0/2799605983 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f5130033070 con 0x55df60ed0a60 2024-02-18T11:40:11.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.009+0000 7f514dc07740 10 client.15036 fetch_fsmap finished waiting for FSMap version 20 2024-02-18T11:40:11.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.009+0000 7f514dc07740 10 client.15036 resolve_mds: resolved name 'c' to daemon mds.c 2024-02-18T11:40:11.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.009+0000 7f514dc07740 20 client.15036 populate_metadata read hostname 'smithi012' 2024-02-18T11:40:11.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.009+0000 7f514dc07740 1 --2- 172.21.15.12:0/2799605983 >> [v2:172.21.15.188:6836/1036630143,v1:172.21.15.188:6837/1036630143] conn(0x55df60ee6b30 0x55df60ee5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:11.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.009+0000 7f514dc07740 4 client.15036 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-18T11:40:11.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.009+0000 7f514dc07740 1 -- 172.21.15.12:0/2799605983 --> [v2:172.21.15.188:6836/1036630143,v1:172.21.15.188:6837/1036630143] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55df60b88270 con 0x55df60ee6b30 2024-02-18T11:40:11.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.010+0000 7f5148dd5640 1 --2- 172.21.15.12:0/2799605983 >> [v2:172.21.15.188:6836/1036630143,v1:172.21.15.188:6837/1036630143] conn(0x55df60ee6b30 0x55df60ee5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:11.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.010+0000 7f5123fff640 10 client.15036 ms_handle_connect on v2:172.21.15.188:6836/1036630143 2024-02-18T11:40:11.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.010+0000 7f5148dd5640 1 --2- 172.21.15.12:0/2799605983 >> [v2:172.21.15.188:6836/1036630143,v1:172.21.15.188:6837/1036630143] conn(0x55df60ee6b30 0x55df60ee5db0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=540da2e6464e66f2 server_cookie=29acbdeff121656c in_seq=0 out_seq=0 2024-02-18T11:40:11.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.011+0000 7f5123fff640 1 -- 172.21.15.12:0/2799605983 <== mds.2 v2:172.21.15.188:6836/1036630143 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55df60b88270 con 0x55df60ee6b30 2024-02-18T11:40:11.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.011+0000 7f5123fff640 10 client.15036 handle_command_reply: tid=0 2024-02-18T11:40:11.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.011+0000 7f514dc07740 1 client.15036 shutdown 2024-02-18T11:40:11.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.011+0000 7f514dc07740 1 -- 172.21.15.12:0/2799605983 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df60ed0a60 msgr2=0x55df60eda6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:11.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.011+0000 7f514dc07740 1 --2- 172.21.15.12:0/2799605983 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df60ed0a60 0x55df60eda6a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f5130010ba0 tx=0x7f5130004730 comp rx=0 tx=0).stop 2024-02-18T11:40:11.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.011+0000 7f514dc07740 1 -- 172.21.15.12:0/2799605983 shutdown_connections 2024-02-18T11:40:11.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.011+0000 7f514dc07740 1 --2- 172.21.15.12:0/2799605983 >> [v2:172.21.15.188:6836/1036630143,v1:172.21.15.188:6837/1036630143] conn(0x55df60ee6b30 0x55df60ee5db0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.011+0000 7f514dc07740 1 --2- 172.21.15.12:0/2799605983 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55df60ed5bd0 0x55df60ee16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.011+0000 7f514dc07740 1 --2- 172.21.15.12:0/2799605983 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df60ed0a60 0x55df60eda6a0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.011+0000 7f5123fff640 0 client.15036 ms_handle_reset on v2:172.21.15.188:6836/1036630143 2024-02-18T11:40:11.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.011+0000 7f514dc07740 1 --2- 172.21.15.12:0/2799605983 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55df60ed0070 0x55df60eda160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.011+0000 7f514dc07740 1 -- 172.21.15.12:0/2799605983 >> 172.21.15.12:0/2799605983 conn(0x55df60ecb5e0 msgr2=0x55df60eca8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:11.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.011+0000 7f514dc07740 1 -- 172.21.15.12:0/2799605983 shutdown_connections 2024-02-18T11:40:11.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.011+0000 7f514dc07740 1 -- 172.21.15.12:0/2799605983 wait complete. 2024-02-18T11:40:11.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.011+0000 7f514dc07740 20 client.15036 trim_cache size 0 max 16384 2024-02-18T11:40:11.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.027+0000 7f514dc07740 1 Processor -- start 2024-02-18T11:40:11.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.027+0000 7f514dc07740 1 -- start start 2024-02-18T11:40:11.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.027+0000 7f514dc07740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55df60ed0070 0x55df60ed8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:11.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.027+0000 7f514dc07740 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55df60ed0a60 0x55df60ee16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:11.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.027+0000 7f514dc07740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df60ed5bd0 0x55df60ed1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:11.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.027+0000 7f514dc07740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df60dbd490 con 0x55df60ed5bd0 2024-02-18T11:40:11.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.027+0000 7f514dc07740 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df60ecc5e0 con 0x55df60ed0a60 2024-02-18T11:40:11.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.027+0000 7f514dc07740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df60b882a0 con 0x55df60ed0070 2024-02-18T11:40:11.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.027+0000 7f51495d6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df60ed5bd0 0x55df60ed1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:11.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.027+0000 7f51495d6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df60ed5bd0 0x55df60ed1f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40132/0 (socket says 172.21.15.12:40132) 2024-02-18T11:40:11.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.027+0000 7f51495d6640 1 -- 172.21.15.12:0/2775938450 learned_addr learned my addr 172.21.15.12:0/2775938450 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:40:11.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.027+0000 7f5148dd5640 1 --2- 172.21.15.12:0/2775938450 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55df60ed0070 0x55df60ed8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:11.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.028+0000 7f5143fff640 1 --2- 172.21.15.12:0/2775938450 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55df60ed0a60 0x55df60ee16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:11.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.028+0000 7f51495d6640 1 -- 172.21.15.12:0/2775938450 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55df60ed0070 msgr2=0x55df60ed8ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:40:11.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.028+0000 7f51495d6640 1 --2- 172.21.15.12:0/2775938450 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55df60ed0070 0x55df60ed8ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.028+0000 7f51495d6640 1 -- 172.21.15.12:0/2775938450 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55df60ed0a60 msgr2=0x55df60ee16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:40:11.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.028+0000 7f51495d6640 1 --2- 172.21.15.12:0/2775938450 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55df60ed0a60 0x55df60ee16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.028+0000 7f51495d6640 1 -- 172.21.15.12:0/2775938450 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df60dd2a20 con 0x55df60ed5bd0 2024-02-18T11:40:11.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.028+0000 7f5143fff640 1 --2- 172.21.15.12:0/2775938450 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55df60ed0a60 0x55df60ee16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:40:11.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.028+0000 7f51495d6640 1 --2- 172.21.15.12:0/2775938450 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df60ed5bd0 0x55df60ed1f20 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f513c050010 tx=0x7f513c005820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fe118f266d7bd23 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:11.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.028+0000 7f512e7fc640 1 -- 172.21.15.12:0/2775938450 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f513c097600 con 0x55df60ed5bd0 2024-02-18T11:40:11.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.028+0000 7f512e7fc640 1 -- 172.21.15.12:0/2775938450 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f513c0a3b00 con 0x55df60ed5bd0 2024-02-18T11:40:11.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.028+0000 7f512e7fc640 1 -- 172.21.15.12:0/2775938450 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f513c0cc790 con 0x55df60ed5bd0 2024-02-18T11:40:11.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.028+0000 7f514dc07740 1 -- 172.21.15.12:0/2775938450 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df60ed5bd0 msgr2=0x55df60ed1f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:11.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.028+0000 7f514dc07740 1 --2- 172.21.15.12:0/2775938450 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df60ed5bd0 0x55df60ed1f20 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f513c050010 tx=0x7f513c005820 comp rx=0 tx=0).stop 2024-02-18T11:40:11.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.028+0000 7f514dc07740 1 -- 172.21.15.12:0/2775938450 shutdown_connections 2024-02-18T11:40:11.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.028+0000 7f514dc07740 1 --2- 172.21.15.12:0/2775938450 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df60ed5bd0 0x55df60ed1f20 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.028+0000 7f514dc07740 1 --2- 172.21.15.12:0/2775938450 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55df60ed0a60 0x55df60ee16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.028+0000 7f514dc07740 1 --2- 172.21.15.12:0/2775938450 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55df60ed0070 0x55df60ed8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.028+0000 7f514dc07740 1 -- 172.21.15.12:0/2775938450 >> 172.21.15.12:0/2775938450 conn(0x55df60ecb5e0 msgr2=0x55df60ed1460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:11.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.029+0000 7f514dc07740 1 -- 172.21.15.12:0/2775938450 shutdown_connections 2024-02-18T11:40:11.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.029+0000 7f514dc07740 1 -- 172.21.15.12:0/2775938450 wait complete. 2024-02-18T11:40:11.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.029+0000 7f514dc07740 1 -- start start 2024-02-18T11:40:11.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.029+0000 7f514dc07740 1 Processor -- start 2024-02-18T11:40:11.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.029+0000 7f514dc07740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55df60ed0070 0x55df60ee1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:11.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.029+0000 7f514dc07740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df60ed0a60 0x55df60edddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:11.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.029+0000 7f514dc07740 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55df60ed5bd0 0x55df60ede330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:11.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.029+0000 7f514dc07740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df60ede870 con 0x55df60ed0a60 2024-02-18T11:40:11.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.029+0000 7f514dc07740 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df60ede9e0 con 0x55df60ed5bd0 2024-02-18T11:40:11.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.029+0000 7f514dc07740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df60f3c730 con 0x55df60ed0070 2024-02-18T11:40:11.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.029+0000 7f514dc07740 10 client.0 _ll_register_callbacks cb 0x7f513c096f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-18T11:40:11.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.029+0000 7f5143fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df60ed0a60 0x55df60edddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:11.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.029+0000 7f5143fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df60ed0a60 0x55df60edddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40144/0 (socket says 172.21.15.12:40144) 2024-02-18T11:40:11.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.029+0000 7f5143fff640 1 -- 172.21.15.12:0/3509884492 learned_addr learned my addr 172.21.15.12:0/3509884492 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:40:11.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.029+0000 7f5148dd5640 1 --2- 172.21.15.12:0/3509884492 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55df60ed0070 0x55df60ee1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:11.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.029+0000 7f5143fff640 1 -- 172.21.15.12:0/3509884492 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55df60ed0070 msgr2=0x55df60ee1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:40:11.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.030+0000 7f5143fff640 1 --2- 172.21.15.12:0/3509884492 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55df60ed0070 0x55df60ee1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.030+0000 7f5143fff640 1 -- 172.21.15.12:0/3509884492 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55df60ed5bd0 msgr2=0x55df60ede330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:40:11.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.030+0000 7f51495d6640 1 --2- 172.21.15.12:0/3509884492 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55df60ed5bd0 0x55df60ede330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:11.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.030+0000 7f5143fff640 1 --2- 172.21.15.12:0/3509884492 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55df60ed5bd0 0x55df60ede330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.030+0000 7f5143fff640 1 -- 172.21.15.12:0/3509884492 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55df60decf20 con 0x55df60ed0a60 2024-02-18T11:40:11.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.030+0000 7f5143fff640 1 --2- 172.21.15.12:0/3509884492 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df60ed0a60 0x55df60edddf0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f5130010b70 tx=0x7f5130010870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7428df347e4102e server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:11.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.030+0000 7f5123fff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-02-18T11:40:11.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.030+0000 7f5123fff640 1 -- 172.21.15.12:0/3509884492 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5130011ef0 con 0x55df60ed0a60 2024-02-18T11:40:11.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.030+0000 7f5123fff640 1 -- 172.21.15.12:0/3509884492 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5130013040 con 0x55df60ed0a60 2024-02-18T11:40:11.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.030+0000 7f514dc07740 1 -- 172.21.15.12:0/3509884492 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55df60ee6840 con 0x55df60ed0a60 2024-02-18T11:40:11.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.030+0000 7f5123fff640 1 -- 172.21.15.12:0/3509884492 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51300117b0 con 0x55df60ed0a60 2024-02-18T11:40:11.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.030+0000 7f514dc07740 10 client.15042 fetch_fsmap learned FSMap version 20 2024-02-18T11:40:11.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.030+0000 7f514dc07740 1 -- 172.21.15.12:0/3509884492 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55df60df8780 con 0x55df60ed0a60 2024-02-18T11:40:11.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.030+0000 7f5123fff640 1 -- 172.21.15.12:0/3509884492 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55df60ee6840 con 0x55df60ed0a60 2024-02-18T11:40:11.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.030+0000 7f5123fff640 1 -- 172.21.15.12:0/3509884492 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f5130046070 con 0x55df60ed0a60 2024-02-18T11:40:11.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.030+0000 7f514dc07740 10 client.15042 fetch_fsmap finished waiting for FSMap version 20 2024-02-18T11:40:11.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.030+0000 7f514dc07740 10 client.15042 resolve_mds: resolved name 'c' to daemon mds.c 2024-02-18T11:40:11.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.030+0000 7f514dc07740 20 client.15042 populate_metadata read hostname 'smithi012' 2024-02-18T11:40:11.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.030+0000 7f514dc07740 1 --2- 172.21.15.12:0/3509884492 >> [v2:172.21.15.188:6836/1036630143,v1:172.21.15.188:6837/1036630143] conn(0x55df60f419a0 0x55df60f3de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:11.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.030+0000 7f514dc07740 4 client.15042 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "status"}] 2024-02-18T11:40:11.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.030+0000 7f514dc07740 1 -- 172.21.15.12:0/3509884492 --> [v2:172.21.15.188:6836/1036630143,v1:172.21.15.188:6837/1036630143] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f513c09b360 con 0x55df60f419a0 2024-02-18T11:40:11.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.031+0000 7f5148dd5640 1 --2- 172.21.15.12:0/3509884492 >> [v2:172.21.15.188:6836/1036630143,v1:172.21.15.188:6837/1036630143] conn(0x55df60f419a0 0x55df60f3de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:11.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.031+0000 7f5148dd5640 1 --2- 172.21.15.12:0/3509884492 >> [v2:172.21.15.188:6836/1036630143,v1:172.21.15.188:6837/1036630143] conn(0x55df60f419a0 0x55df60f3de00 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ae4bdf93b559db1c server_cookie=dad4b02317a085b8 in_seq=0 out_seq=0 2024-02-18T11:40:11.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.031+0000 7f5123fff640 10 client.15042 ms_handle_connect on v2:172.21.15.188:6836/1036630143 2024-02-18T11:40:11.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.031+0000 7f5123fff640 1 -- 172.21.15.12:0/3509884492 <== mds.2 v2:172.21.15.188:6836/1036630143 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x7f513c09b360 con 0x55df60f419a0 2024-02-18T11:40:11.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.031+0000 7f5123fff640 10 client.15042 handle_command_reply: tid=0 2024-02-18T11:40:11.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.031+0000 7f514dc07740 1 client.15042 shutdown 2024-02-18T11:40:11.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.031+0000 7f514dc07740 1 -- 172.21.15.12:0/3509884492 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df60ed0a60 msgr2=0x55df60edddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:11.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.031+0000 7f514dc07740 1 --2- 172.21.15.12:0/3509884492 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df60ed0a60 0x55df60edddf0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f5130010b70 tx=0x7f5130010870 comp rx=0 tx=0).stop 2024-02-18T11:40:11.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.031+0000 7f514dc07740 1 -- 172.21.15.12:0/3509884492 shutdown_connections 2024-02-18T11:40:11.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.031+0000 7f514dc07740 1 --2- 172.21.15.12:0/3509884492 >> [v2:172.21.15.188:6836/1036630143,v1:172.21.15.188:6837/1036630143] conn(0x55df60f419a0 0x55df60f3de00 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.031+0000 7f514dc07740 1 --2- 172.21.15.12:0/3509884492 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55df60ed5bd0 0x55df60ede330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.031+0000 7f5123fff640 0 client.15042 ms_handle_reset on v2:172.21.15.188:6836/1036630143 2024-02-18T11:40:11.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.031+0000 7f514dc07740 1 --2- 172.21.15.12:0/3509884492 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df60ed0a60 0x55df60edddf0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.031+0000 7f514dc07740 1 --2- 172.21.15.12:0/3509884492 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55df60ed0070 0x55df60ee1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.031+0000 7f514dc07740 1 -- 172.21.15.12:0/3509884492 >> 172.21.15.12:0/3509884492 conn(0x55df60ecb5e0 msgr2=0x55df60ed8ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:11.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.032+0000 7f514dc07740 1 -- 172.21.15.12:0/3509884492 shutdown_connections 2024-02-18T11:40:11.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.032+0000 7f514dc07740 1 -- 172.21.15.12:0/3509884492 wait complete. 2024-02-18T11:40:11.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.032+0000 7f514dc07740 20 client.15042 trim_cache size 0 max 16384 2024-02-18T11:40:11.033 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-02-18T11:40:11.033 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_fsid": "0c8bb0de-ce51-11ee-95ba-87774f69a715", 2024-02-18T11:40:11.033 INFO:teuthology.orchestra.run.smithi012.stdout: "whoami": 2, 2024-02-18T11:40:11.034 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 24478, 2024-02-18T11:40:11.034 INFO:teuthology.orchestra.run.smithi012.stdout: "want_state": "up:active", 2024-02-18T11:40:11.034 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "up:active", 2024-02-18T11:40:11.034 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs", 2024-02-18T11:40:11.034 INFO:teuthology.orchestra.run.smithi012.stdout: "rank_uptime": 80.179666948000005, 2024-02-18T11:40:11.034 INFO:teuthology.orchestra.run.smithi012.stdout: "mdsmap_epoch": 20, 2024-02-18T11:40:11.034 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch": 89, 2024-02-18T11:40:11.034 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch_barrier": 87, 2024-02-18T11:40:11.034 INFO:teuthology.orchestra.run.smithi012.stdout: "uptime": 103.091880574 2024-02-18T11:40:11.034 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-02-18T11:40:11.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.032+0000 7f514b039640 1 -- 172.21.15.12:0/784085644 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f51340a27e0 msgr2=0x7f51340a4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:11.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.032+0000 7f514b039640 1 --2- 172.21.15.12:0/784085644 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f51340a27e0 0x7f51340a4ca0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5130005d90 tx=0x7f513000aa70 comp rx=0 tx=0).stop 2024-02-18T11:40:11.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.032+0000 7f514b039640 1 -- 172.21.15.12:0/784085644 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5144102d80 msgr2=0x7f514419e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:11.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.032+0000 7f514b039640 1 --2- 172.21.15.12:0/784085644 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5144102d80 0x7f514419e850 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f513c00eaa0 tx=0x7f513c00ef60 comp rx=0 tx=0).stop 2024-02-18T11:40:11.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.032+0000 7f514b039640 1 -- 172.21.15.12:0/784085644 shutdown_connections 2024-02-18T11:40:11.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.032+0000 7f514b039640 1 --2- 172.21.15.12:0/784085644 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f51340a27e0 0x7f51340a4ca0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.032+0000 7f514b039640 1 --2- 172.21.15.12:0/784085644 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51441023c0 0x7f51441977e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.032+0000 7f514b039640 1 --2- 172.21.15.12:0/784085644 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5144102d80 0x7f514419e850 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.032+0000 7f514b039640 1 --2- 172.21.15.12:0/784085644 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f51441011d0 0x7f51441972a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.032+0000 7f514b039640 1 -- 172.21.15.12:0/784085644 >> 172.21.15.12:0/784085644 conn(0x7f51440fc760 msgr2=0x7f5144105730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:11.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.032+0000 7f514b039640 1 -- 172.21.15.12:0/784085644 shutdown_connections 2024-02-18T11:40:11.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.032+0000 7f514b039640 1 -- 172.21.15.12:0/784085644 wait complete. 2024-02-18T11:40:11.037 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-02-18T11:40:11.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.147+0000 7ff4d9402640 1 -- 172.21.15.12:0/3355361937 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff4d4102ca0 msgr2=0x7ff4d4109530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:11.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.147+0000 7ff4d9402640 1 --2- 172.21.15.12:0/3355361937 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff4d4102ca0 0x7ff4d4109530 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4009ca0 tx=0x7ff4c402c0f0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.147+0000 7ff4d9402640 1 -- 172.21.15.12:0/3355361937 shutdown_connections 2024-02-18T11:40:11.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.147+0000 7ff4d9402640 1 --2- 172.21.15.12:0/3355361937 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff4d41022e0 0x7ff4d4102760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.147+0000 7ff4d9402640 1 --2- 172.21.15.12:0/3355361937 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff4d4102ca0 0x7ff4d4109530 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.147+0000 7ff4d9402640 1 --2- 172.21.15.12:0/3355361937 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff4d41010f0 0x7ff4d4101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.147+0000 7ff4d9402640 1 -- 172.21.15.12:0/3355361937 >> 172.21.15.12:0/3355361937 conn(0x7ff4d40fc660 msgr2=0x7ff4d40feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:11.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.147+0000 7ff4d9402640 1 -- 172.21.15.12:0/3355361937 shutdown_connections 2024-02-18T11:40:11.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.147+0000 7ff4d9402640 1 -- 172.21.15.12:0/3355361937 wait complete. 2024-02-18T11:40:11.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.148+0000 7ff4d9402640 1 Processor -- start 2024-02-18T11:40:11.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.148+0000 7ff4d9402640 1 -- start start 2024-02-18T11:40:11.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.148+0000 7ff4d9402640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff4d41010f0 0x7ff4d41970f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:11.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.148+0000 7ff4d9402640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff4d41022e0 0x7ff4d4197630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:11.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.148+0000 7ff4d9402640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff4d4102ca0 0x7ff4d419e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:11.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.148+0000 7ff4d9402640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d419ebe0 con 0x7ff4d41010f0 2024-02-18T11:40:11.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.148+0000 7ff4d9402640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d419ed50 con 0x7ff4d4102ca0 2024-02-18T11:40:11.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.148+0000 7ff4d9402640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d419ef10 con 0x7ff4d41022e0 2024-02-18T11:40:11.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.148+0000 7ff4d2ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff4d41010f0 0x7ff4d41970f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:11.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.148+0000 7ff4d2ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff4d41010f0 0x7ff4d41970f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40156/0 (socket says 172.21.15.12:40156) 2024-02-18T11:40:11.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.148+0000 7ff4d2ffd640 1 -- 172.21.15.12:0/903171310 learned_addr learned my addr 172.21.15.12:0/903171310 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:40:11.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.148+0000 7ff4d37fe640 1 --2- 172.21.15.12:0/903171310 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff4d4102ca0 0x7ff4d419e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:11.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.149+0000 7ff4d27fc640 1 --2- 172.21.15.12:0/903171310 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff4d41022e0 0x7ff4d4197630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:11.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.149+0000 7ff4d2ffd640 1 -- 172.21.15.12:0/903171310 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff4d41022e0 msgr2=0x7ff4d4197630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:11.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.149+0000 7ff4d2ffd640 1 --2- 172.21.15.12:0/903171310 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff4d41022e0 0x7ff4d4197630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.149+0000 7ff4d2ffd640 1 -- 172.21.15.12:0/903171310 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff4d4102ca0 msgr2=0x7ff4d419e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:11.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.149+0000 7ff4d2ffd640 1 --2- 172.21.15.12:0/903171310 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff4d4102ca0 0x7ff4d419e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.149+0000 7ff4d2ffd640 1 -- 172.21.15.12:0/903171310 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4c4009900 con 0x7ff4d41010f0 2024-02-18T11:40:11.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.149+0000 7ff4d2ffd640 1 --2- 172.21.15.12:0/903171310 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff4d41010f0 0x7ff4d41970f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc00d770 tx=0x7ff4cc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:11.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.149+0000 7ff4cbfff640 1 -- 172.21.15.12:0/903171310 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc009c30 con 0x7ff4d41010f0 2024-02-18T11:40:11.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.149+0000 7ff4cbfff640 1 -- 172.21.15.12:0/903171310 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff4cc00de80 con 0x7ff4d41010f0 2024-02-18T11:40:11.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.149+0000 7ff4d9402640 1 -- 172.21.15.12:0/903171310 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4d419f1f0 con 0x7ff4d41010f0 2024-02-18T11:40:11.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.149+0000 7ff4cbfff640 1 -- 172.21.15.12:0/903171310 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc010460 con 0x7ff4d41010f0 2024-02-18T11:40:11.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.149+0000 7ff4d9402640 1 -- 172.21.15.12:0/903171310 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4d419f700 con 0x7ff4d41010f0 2024-02-18T11:40:11.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.151+0000 7ff4cbfff640 1 -- 172.21.15.12:0/903171310 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 137296+0+0 (secure 0 0 0) 0x7ff4cc003f60 con 0x7ff4d41010f0 2024-02-18T11:40:11.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.151+0000 7ff4dbfd2740 1 Processor -- start 2024-02-18T11:40:11.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.151+0000 7ff4dbfd2740 1 -- start start 2024-02-18T11:40:11.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.152+0000 7ff4dbfd2740 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5601db889270 0x5601db889690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:11.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.152+0000 7ff4dbfd2740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5601db889c60 0x5601db88e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:11.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.152+0000 7ff4dbfd2740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601db88edd0 0x5601db891260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:11.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.152+0000 7ff4dbfd2740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601db844dc0 con 0x5601db88edd0 2024-02-18T11:40:11.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.152+0000 7ff4dbfd2740 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601db7f7f20 con 0x5601db889270 2024-02-18T11:40:11.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.152+0000 7ff4dbfd2740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601db7f5450 con 0x5601db889c60 2024-02-18T11:40:11.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.152+0000 7ff4cbfff640 1 --2- 172.21.15.12:0/903171310 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7ff4c00a27e0 0x7ff4c00a4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:11.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.152+0000 7ff4d27fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601db88edd0 0x5601db891260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:11.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.152+0000 7ff4d27fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601db88edd0 0x5601db891260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40164/0 (socket says 172.21.15.12:40164) 2024-02-18T11:40:11.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.152+0000 7ff4d27fc640 1 -- 172.21.15.12:0/490603217 learned_addr learned my addr 172.21.15.12:0/490603217 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:40:11.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.152+0000 7ff4d37fe640 1 --2- 172.21.15.12:0/903171310 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7ff4c00a27e0 0x7ff4c00a4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:11.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.152+0000 7ff4d37fe640 1 --2- 172.21.15.12:0/490603217 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5601db889270 0x5601db889690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:11.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.152+0000 7ff4cbfff640 1 -- 172.21.15.12:0/903171310 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff4cc0ce450 con 0x7ff4d41010f0 2024-02-18T11:40:11.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.152+0000 7ff4d2ffd640 1 --2- 172.21.15.12:0/490603217 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5601db889c60 0x5601db88e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:11.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.152+0000 7ff4d27fc640 1 -- 172.21.15.12:0/490603217 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5601db889c60 msgr2=0x5601db88e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:40:11.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.152+0000 7ff4d27fc640 1 --2- 172.21.15.12:0/490603217 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5601db889c60 0x5601db88e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.152+0000 7ff4d27fc640 1 -- 172.21.15.12:0/490603217 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5601db889270 msgr2=0x5601db889690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:40:11.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.152+0000 7ff4d27fc640 1 --2- 172.21.15.12:0/490603217 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5601db889270 0x5601db889690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.152+0000 7ff4d27fc640 1 -- 172.21.15.12:0/490603217 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5601db7a5f20 con 0x5601db88edd0 2024-02-18T11:40:11.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.152+0000 7ff4d27fc640 1 --2- 172.21.15.12:0/490603217 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601db88edd0 0x5601db891260 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc00bc90 tx=0x7ff4bc00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1551546dc7b70951 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:11.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.152+0000 7ff4d37fe640 1 --2- 172.21.15.12:0/903171310 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7ff4c00a27e0 0x7ff4c00a4ca0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4005e30 tx=0x7ff4c40323d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:11.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.153+0000 7ff4c97fa640 1 -- 172.21.15.12:0/490603217 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc01a460 con 0x5601db88edd0 2024-02-18T11:40:11.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.153+0000 7ff4c97fa640 1 -- 172.21.15.12:0/490603217 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff4bc01aaa0 con 0x5601db88edd0 2024-02-18T11:40:11.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.153+0000 7ff4c97fa640 1 -- 172.21.15.12:0/490603217 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc011a70 con 0x5601db88edd0 2024-02-18T11:40:11.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.153+0000 7ff4dbfd2740 1 -- 172.21.15.12:0/490603217 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601db88edd0 msgr2=0x5601db891260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:11.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.153+0000 7ff4dbfd2740 1 --2- 172.21.15.12:0/490603217 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601db88edd0 0x5601db891260 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc00bc90 tx=0x7ff4bc00f730 comp rx=0 tx=0).stop 2024-02-18T11:40:11.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.153+0000 7ff4dbfd2740 1 -- 172.21.15.12:0/490603217 shutdown_connections 2024-02-18T11:40:11.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.153+0000 7ff4dbfd2740 1 --2- 172.21.15.12:0/490603217 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5601db889c60 0x5601db88e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.153+0000 7ff4dbfd2740 1 --2- 172.21.15.12:0/490603217 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601db88edd0 0x5601db891260 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.153+0000 7ff4dbfd2740 1 --2- 172.21.15.12:0/490603217 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5601db889270 0x5601db889690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.153+0000 7ff4dbfd2740 1 -- 172.21.15.12:0/490603217 >> 172.21.15.12:0/490603217 conn(0x5601db8847e0 msgr2=0x5601db884be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:11.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.153+0000 7ff4dbfd2740 1 -- 172.21.15.12:0/490603217 shutdown_connections 2024-02-18T11:40:11.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.153+0000 7ff4dbfd2740 1 -- 172.21.15.12:0/490603217 wait complete. 2024-02-18T11:40:11.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.154+0000 7ff4dbfd2740 1 -- start start 2024-02-18T11:40:11.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.154+0000 7ff4dbfd2740 1 Processor -- start 2024-02-18T11:40:11.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.154+0000 7ff4dbfd2740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601db889270 0x5601db893450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:11.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.154+0000 7ff4dbfd2740 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5601db889c60 0x5601db893990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:11.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.154+0000 7ff4dbfd2740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5601db88edd0 0x5601db89a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:11.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.154+0000 7ff4dbfd2740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601db7f0c30 con 0x5601db889270 2024-02-18T11:40:11.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.154+0000 7ff4dbfd2740 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601db795d30 con 0x5601db889c60 2024-02-18T11:40:11.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.154+0000 7ff4dbfd2740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601db4c4310 con 0x5601db88edd0 2024-02-18T11:40:11.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.154+0000 7ff4dbfd2740 10 client.0 _ll_register_callbacks cb 0x5601db79dd40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-18T11:40:11.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.154+0000 7ff4d2ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601db889270 0x5601db893450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:11.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.154+0000 7ff4d2ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601db889270 0x5601db893450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40176/0 (socket says 172.21.15.12:40176) 2024-02-18T11:40:11.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.154+0000 7ff4d2ffd640 1 -- 172.21.15.12:0/2208333930 learned_addr learned my addr 172.21.15.12:0/2208333930 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:40:11.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.154+0000 7ff4d37fe640 1 --2- 172.21.15.12:0/2208333930 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5601db88edd0 0x5601db89a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:11.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.154+0000 7ff4d2ffd640 1 -- 172.21.15.12:0/2208333930 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5601db88edd0 msgr2=0x5601db89a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:40:11.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.154+0000 7ff4d2ffd640 1 --2- 172.21.15.12:0/2208333930 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5601db88edd0 0x5601db89a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.154+0000 7ff4d2ffd640 1 -- 172.21.15.12:0/2208333930 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5601db889c60 msgr2=0x5601db893990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:40:11.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.154+0000 7ff4d2ffd640 1 --2- 172.21.15.12:0/2208333930 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5601db889c60 0x5601db893990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.154+0000 7ff4d2ffd640 1 -- 172.21.15.12:0/2208333930 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4bc00b8c0 con 0x5601db889270 2024-02-18T11:40:11.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.154+0000 7ff4d2ffd640 1 --2- 172.21.15.12:0/2208333930 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601db889270 0x5601db893450 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc000c00 tx=0x7ff4cc09ae50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2212e7fa062ca284 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:11.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.154+0000 7ff4ba7fc640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-02-18T11:40:11.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.155+0000 7ff4ba7fc640 1 -- 172.21.15.12:0/2208333930 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc014070 con 0x5601db889270 2024-02-18T11:40:11.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.155+0000 7ff4ba7fc640 1 -- 172.21.15.12:0/2208333930 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff4cc0a3530 con 0x5601db889270 2024-02-18T11:40:11.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.155+0000 7ff4dbfd2740 1 -- 172.21.15.12:0/2208333930 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5601db4f06c0 con 0x5601db889270 2024-02-18T11:40:11.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.155+0000 7ff4ba7fc640 1 -- 172.21.15.12:0/2208333930 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff4cc0b9170 con 0x5601db889270 2024-02-18T11:40:11.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.155+0000 7ff4dbfd2740 10 client.15054 fetch_fsmap learned FSMap version 20 2024-02-18T11:40:11.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.155+0000 7ff4dbfd2740 1 -- 172.21.15.12:0/2208333930 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x5601db7ef2b0 con 0x5601db889270 2024-02-18T11:40:11.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.155+0000 7ff4ba7fc640 1 -- 172.21.15.12:0/2208333930 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff4cc07d080 con 0x5601db889270 2024-02-18T11:40:11.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.155+0000 7ff4ba7fc640 1 -- 172.21.15.12:0/2208333930 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff4cc023070 con 0x5601db889270 2024-02-18T11:40:11.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.155+0000 7ff4dbfd2740 10 client.15054 fetch_fsmap finished waiting for FSMap version 20 2024-02-18T11:40:11.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.155+0000 7ff4dbfd2740 10 client.15054 resolve_mds: resolved name 'i' to daemon mds.i 2024-02-18T11:40:11.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.155+0000 7ff4dbfd2740 20 client.15054 populate_metadata read hostname 'smithi012' 2024-02-18T11:40:11.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.155+0000 7ff4dbfd2740 1 --2- 172.21.15.12:0/2208333930 >> [v2:172.21.15.188:6838/1342671190,v1:172.21.15.188:6839/1342671190] conn(0x5601db89fe40 0x5601db89f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:11.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.155+0000 7ff4dbfd2740 4 client.15054 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-18T11:40:11.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.155+0000 7ff4dbfd2740 1 -- 172.21.15.12:0/2208333930 --> [v2:172.21.15.188:6838/1342671190,v1:172.21.15.188:6839/1342671190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5601db541270 con 0x5601db89fe40 2024-02-18T11:40:11.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.156+0000 7ff4d27fc640 1 --2- 172.21.15.12:0/2208333930 >> [v2:172.21.15.188:6838/1342671190,v1:172.21.15.188:6839/1342671190] conn(0x5601db89fe40 0x5601db89f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:11.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.156+0000 7ff4d27fc640 1 --2- 172.21.15.12:0/2208333930 >> [v2:172.21.15.188:6838/1342671190,v1:172.21.15.188:6839/1342671190] conn(0x5601db89fe40 0x5601db89f0b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4529da27e8bc9375 server_cookie=c72b31c22cf604dd in_seq=0 out_seq=0 2024-02-18T11:40:11.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.156+0000 7ff4ba7fc640 10 client.15054 ms_handle_connect on v2:172.21.15.188:6838/1342671190 2024-02-18T11:40:11.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.157+0000 7ff4ba7fc640 1 -- 172.21.15.12:0/2208333930 <== mds.1 v2:172.21.15.188:6838/1342671190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5601db541270 con 0x5601db89fe40 2024-02-18T11:40:11.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.157+0000 7ff4ba7fc640 10 client.15054 handle_command_reply: tid=0 2024-02-18T11:40:11.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.157+0000 7ff4dbfd2740 1 client.15054 shutdown 2024-02-18T11:40:11.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.157+0000 7ff4dbfd2740 1 -- 172.21.15.12:0/2208333930 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601db889270 msgr2=0x5601db893450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:11.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.157+0000 7ff4dbfd2740 1 --2- 172.21.15.12:0/2208333930 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601db889270 0x5601db893450 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc000c00 tx=0x7ff4cc09ae50 comp rx=0 tx=0).stop 2024-02-18T11:40:11.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.157+0000 7ff4dbfd2740 1 -- 172.21.15.12:0/2208333930 shutdown_connections 2024-02-18T11:40:11.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.157+0000 7ff4dbfd2740 1 --2- 172.21.15.12:0/2208333930 >> [v2:172.21.15.188:6838/1342671190,v1:172.21.15.188:6839/1342671190] conn(0x5601db89fe40 0x5601db89f0b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.157+0000 7ff4dbfd2740 1 --2- 172.21.15.12:0/2208333930 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5601db88edd0 0x5601db89a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.157+0000 7ff4ba7fc640 0 client.15054 ms_handle_reset on v2:172.21.15.188:6838/1342671190 2024-02-18T11:40:11.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.157+0000 7ff4dbfd2740 1 --2- 172.21.15.12:0/2208333930 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5601db889c60 0x5601db893990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.157+0000 7ff4dbfd2740 1 --2- 172.21.15.12:0/2208333930 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601db889270 0x5601db893450 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.157+0000 7ff4dbfd2740 1 -- 172.21.15.12:0/2208333930 >> 172.21.15.12:0/2208333930 conn(0x5601db8847e0 msgr2=0x5601db883a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:11.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.157+0000 7ff4dbfd2740 1 -- 172.21.15.12:0/2208333930 shutdown_connections 2024-02-18T11:40:11.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.157+0000 7ff4dbfd2740 1 -- 172.21.15.12:0/2208333930 wait complete. 2024-02-18T11:40:11.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.157+0000 7ff4dbfd2740 20 client.15054 trim_cache size 0 max 16384 2024-02-18T11:40:11.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.173+0000 7ff4dbfd2740 1 Processor -- start 2024-02-18T11:40:11.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.173+0000 7ff4dbfd2740 1 -- start start 2024-02-18T11:40:11.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.173+0000 7ff4dbfd2740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5601db889270 0x5601db88a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:11.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.173+0000 7ff4dbfd2740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601db889c60 0x5601db88a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:11.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.173+0000 7ff4dbfd2740 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5601db88edd0 0x5601db893060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:11.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.173+0000 7ff4dbfd2740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601db776490 con 0x5601db889c60 2024-02-18T11:40:11.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.173+0000 7ff4dbfd2740 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601db7a9f70 con 0x5601db88edd0 2024-02-18T11:40:11.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.173+0000 7ff4dbfd2740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601db888e30 con 0x5601db889270 2024-02-18T11:40:11.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.173+0000 7ff4d27fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601db889c60 0x5601db88a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:11.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.173+0000 7ff4d27fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601db889c60 0x5601db88a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40192/0 (socket says 172.21.15.12:40192) 2024-02-18T11:40:11.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.173+0000 7ff4d27fc640 1 -- 172.21.15.12:0/382574487 learned_addr learned my addr 172.21.15.12:0/382574487 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:40:11.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.173+0000 7ff4d2ffd640 1 --2- 172.21.15.12:0/382574487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5601db889270 0x5601db88a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:11.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.173+0000 7ff4d37fe640 1 --2- 172.21.15.12:0/382574487 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5601db88edd0 0x5601db893060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:11.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.173+0000 7ff4d27fc640 1 -- 172.21.15.12:0/382574487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5601db889270 msgr2=0x5601db88a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:40:11.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.173+0000 7ff4d27fc640 1 --2- 172.21.15.12:0/382574487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5601db889270 0x5601db88a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.173+0000 7ff4d27fc640 1 -- 172.21.15.12:0/382574487 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5601db88edd0 msgr2=0x5601db893060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:40:11.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.173+0000 7ff4d27fc640 1 --2- 172.21.15.12:0/382574487 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5601db88edd0 0x5601db893060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.173+0000 7ff4d27fc640 1 -- 172.21.15.12:0/382574487 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5601db78ba20 con 0x5601db889c60 2024-02-18T11:40:11.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.173+0000 7ff4d27fc640 1 --2- 172.21.15.12:0/382574487 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601db889c60 0x5601db88a890 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc006010 tx=0x7ff4bc00b850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f454b84ffae4bc4 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:11.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.174+0000 7ff4c8ff9640 1 -- 172.21.15.12:0/382574487 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc02a030 con 0x5601db889c60 2024-02-18T11:40:11.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.174+0000 7ff4c8ff9640 1 -- 172.21.15.12:0/382574487 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff4bc01f040 con 0x5601db889c60 2024-02-18T11:40:11.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.174+0000 7ff4c8ff9640 1 -- 172.21.15.12:0/382574487 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc01e050 con 0x5601db889c60 2024-02-18T11:40:11.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.174+0000 7ff4dbfd2740 1 -- 172.21.15.12:0/382574487 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601db889c60 msgr2=0x5601db88a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:11.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.174+0000 7ff4dbfd2740 1 --2- 172.21.15.12:0/382574487 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601db889c60 0x5601db88a890 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc006010 tx=0x7ff4bc00b850 comp rx=0 tx=0).stop 2024-02-18T11:40:11.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.174+0000 7ff4dbfd2740 1 -- 172.21.15.12:0/382574487 shutdown_connections 2024-02-18T11:40:11.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.174+0000 7ff4dbfd2740 1 --2- 172.21.15.12:0/382574487 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5601db88edd0 0x5601db893060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.174+0000 7ff4dbfd2740 1 --2- 172.21.15.12:0/382574487 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601db889c60 0x5601db88a890 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.174+0000 7ff4dbfd2740 1 --2- 172.21.15.12:0/382574487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5601db889270 0x5601db88a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.174+0000 7ff4dbfd2740 1 -- 172.21.15.12:0/382574487 >> 172.21.15.12:0/382574487 conn(0x5601db8847e0 msgr2=0x5601db88b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:11.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.174+0000 7ff4dbfd2740 1 -- 172.21.15.12:0/382574487 shutdown_connections 2024-02-18T11:40:11.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.174+0000 7ff4dbfd2740 1 -- 172.21.15.12:0/382574487 wait complete. 2024-02-18T11:40:11.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.175+0000 7ff4dbfd2740 1 -- start start 2024-02-18T11:40:11.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.175+0000 7ff4dbfd2740 1 Processor -- start 2024-02-18T11:40:11.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.175+0000 7ff4dbfd2740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5601db889270 0x5601db8f2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:11.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.175+0000 7ff4dbfd2740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601db889c60 0x5601db8f3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:11.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.175+0000 7ff4dbfd2740 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5601db88edd0 0x5601db8fa260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:11.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.175+0000 7ff4dbfd2740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601db8fa7a0 con 0x5601db889c60 2024-02-18T11:40:11.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.175+0000 7ff4dbfd2740 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601db8fa8e0 con 0x5601db88edd0 2024-02-18T11:40:11.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.175+0000 7ff4dbfd2740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601db8faa50 con 0x5601db889270 2024-02-18T11:40:11.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.175+0000 7ff4dbfd2740 10 client.0 _ll_register_callbacks cb 0x7ff4bc00b420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-18T11:40:11.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.175+0000 7ff4d27fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601db889c60 0x5601db8f3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:11.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.175+0000 7ff4d27fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601db889c60 0x5601db8f3220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40202/0 (socket says 172.21.15.12:40202) 2024-02-18T11:40:11.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.175+0000 7ff4d27fc640 1 -- 172.21.15.12:0/2055531257 learned_addr learned my addr 172.21.15.12:0/2055531257 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:40:11.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.175+0000 7ff4d37fe640 1 --2- 172.21.15.12:0/2055531257 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5601db88edd0 0x5601db8fa260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:11.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.175+0000 7ff4d27fc640 1 -- 172.21.15.12:0/2055531257 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5601db889270 msgr2=0x5601db8f2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:40:11.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.175+0000 7ff4d27fc640 1 --2- 172.21.15.12:0/2055531257 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5601db889270 0x5601db8f2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.175+0000 7ff4d27fc640 1 -- 172.21.15.12:0/2055531257 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5601db88edd0 msgr2=0x5601db8fa260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:40:11.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.175+0000 7ff4d27fc640 1 --2- 172.21.15.12:0/2055531257 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5601db88edd0 0x5601db8fa260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.175+0000 7ff4d27fc640 1 -- 172.21.15.12:0/2055531257 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5601db7a5f20 con 0x5601db889c60 2024-02-18T11:40:11.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.175+0000 7ff4d37fe640 1 --2- 172.21.15.12:0/2055531257 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5601db88edd0 0x5601db8fa260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:40:11.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.175+0000 7ff4d27fc640 1 --2- 172.21.15.12:0/2055531257 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601db889c60 0x5601db8f3220 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc02b040 tx=0x7ff4bc005a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce7a3a0dbb0d14af server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:11.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.175+0000 7ff4ba7fc640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-02-18T11:40:11.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.176+0000 7ff4ba7fc640 1 -- 172.21.15.12:0/2055531257 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc01e040 con 0x5601db889c60 2024-02-18T11:40:11.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.176+0000 7ff4ba7fc640 1 -- 172.21.15.12:0/2055531257 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff4bc02b070 con 0x5601db889c60 2024-02-18T11:40:11.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.176+0000 7ff4dbfd2740 1 -- 172.21.15.12:0/2055531257 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5601db8a0e70 con 0x5601db889c60 2024-02-18T11:40:11.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.176+0000 7ff4ba7fc640 1 -- 172.21.15.12:0/2055531257 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff4bc01a6f0 con 0x5601db889c60 2024-02-18T11:40:11.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.176+0000 7ff4dbfd2740 10 client.15060 fetch_fsmap learned FSMap version 20 2024-02-18T11:40:11.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.176+0000 7ff4dbfd2740 1 -- 172.21.15.12:0/2055531257 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x5601db7b1780 con 0x5601db889c60 2024-02-18T11:40:11.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.176+0000 7ff4ba7fc640 1 -- 172.21.15.12:0/2055531257 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5601db8a0e70 con 0x5601db889c60 2024-02-18T11:40:11.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.176+0000 7ff4ba7fc640 1 -- 172.21.15.12:0/2055531257 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff4bc004640 con 0x5601db889c60 2024-02-18T11:40:11.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.176+0000 7ff4dbfd2740 10 client.15060 fetch_fsmap finished waiting for FSMap version 20 2024-02-18T11:40:11.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.176+0000 7ff4dbfd2740 10 client.15060 resolve_mds: resolved name 'i' to daemon mds.i 2024-02-18T11:40:11.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.176+0000 7ff4dbfd2740 20 client.15060 populate_metadata read hostname 'smithi012' 2024-02-18T11:40:11.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.176+0000 7ff4dbfd2740 1 --2- 172.21.15.12:0/2055531257 >> [v2:172.21.15.188:6838/1342671190,v1:172.21.15.188:6839/1342671190] conn(0x5601db8fe4f0 0x5601db8fc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:11.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.176+0000 7ff4dbfd2740 4 client.15060 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "status"}] 2024-02-18T11:40:11.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.176+0000 7ff4dbfd2740 1 -- 172.21.15.12:0/2055531257 --> [v2:172.21.15.188:6838/1342671190,v1:172.21.15.188:6839/1342671190] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7ff4bc01aaa0 con 0x5601db8fe4f0 2024-02-18T11:40:11.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.176+0000 7ff4d2ffd640 1 --2- 172.21.15.12:0/2055531257 >> [v2:172.21.15.188:6838/1342671190,v1:172.21.15.188:6839/1342671190] conn(0x5601db8fe4f0 0x5601db8fc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:11.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.177+0000 7ff4ba7fc640 10 client.15060 ms_handle_connect on v2:172.21.15.188:6838/1342671190 2024-02-18T11:40:11.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.177+0000 7ff4d2ffd640 1 --2- 172.21.15.12:0/2055531257 >> [v2:172.21.15.188:6838/1342671190,v1:172.21.15.188:6839/1342671190] conn(0x5601db8fe4f0 0x5601db8fc130 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f6a081c26ccfd8d6 server_cookie=e4ce17b319b72f93 in_seq=0 out_seq=0 2024-02-18T11:40:11.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.177+0000 7ff4ba7fc640 1 -- 172.21.15.12:0/2055531257 <== mds.1 v2:172.21.15.188:6838/1342671190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x7ff4bc01aaa0 con 0x5601db8fe4f0 2024-02-18T11:40:11.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.177+0000 7ff4ba7fc640 10 client.15060 handle_command_reply: tid=0 2024-02-18T11:40:11.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.177+0000 7ff4dbfd2740 1 client.15060 shutdown 2024-02-18T11:40:11.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.177+0000 7ff4dbfd2740 1 -- 172.21.15.12:0/2055531257 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601db889c60 msgr2=0x5601db8f3220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:11.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.177+0000 7ff4dbfd2740 1 --2- 172.21.15.12:0/2055531257 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601db889c60 0x5601db8f3220 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc02b040 tx=0x7ff4bc005a70 comp rx=0 tx=0).stop 2024-02-18T11:40:11.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.177+0000 7ff4dbfd2740 1 -- 172.21.15.12:0/2055531257 shutdown_connections 2024-02-18T11:40:11.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.177+0000 7ff4dbfd2740 1 --2- 172.21.15.12:0/2055531257 >> [v2:172.21.15.188:6838/1342671190,v1:172.21.15.188:6839/1342671190] conn(0x5601db8fe4f0 0x5601db8fc130 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.177+0000 7ff4dbfd2740 1 --2- 172.21.15.12:0/2055531257 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5601db88edd0 0x5601db8fa260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.177+0000 7ff4dbfd2740 1 --2- 172.21.15.12:0/2055531257 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601db889c60 0x5601db8f3220 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.177+0000 7ff4dbfd2740 1 --2- 172.21.15.12:0/2055531257 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5601db889270 0x5601db8f2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.177+0000 7ff4ba7fc640 0 client.15060 ms_handle_reset on v2:172.21.15.188:6838/1342671190 2024-02-18T11:40:11.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.177+0000 7ff4dbfd2740 1 -- 172.21.15.12:0/2055531257 >> 172.21.15.12:0/2055531257 conn(0x5601db8847e0 msgr2=0x5601db88b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:11.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.177+0000 7ff4dbfd2740 1 -- 172.21.15.12:0/2055531257 shutdown_connections 2024-02-18T11:40:11.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.177+0000 7ff4dbfd2740 1 -- 172.21.15.12:0/2055531257 wait complete. 2024-02-18T11:40:11.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.177+0000 7ff4dbfd2740 20 client.15060 trim_cache size 0 max 16384 2024-02-18T11:40:11.177 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-02-18T11:40:11.178 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_fsid": "0c8bb0de-ce51-11ee-95ba-87774f69a715", 2024-02-18T11:40:11.178 INFO:teuthology.orchestra.run.smithi012.stdout: "whoami": 1, 2024-02-18T11:40:11.178 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 24508, 2024-02-18T11:40:11.178 INFO:teuthology.orchestra.run.smithi012.stdout: "want_state": "up:active", 2024-02-18T11:40:11.178 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "up:active", 2024-02-18T11:40:11.178 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs", 2024-02-18T11:40:11.178 INFO:teuthology.orchestra.run.smithi012.stdout: "rank_uptime": 81.331232772999996, 2024-02-18T11:40:11.178 INFO:teuthology.orchestra.run.smithi012.stdout: "mdsmap_epoch": 20, 2024-02-18T11:40:11.178 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch": 89, 2024-02-18T11:40:11.178 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch_barrier": 87, 2024-02-18T11:40:11.178 INFO:teuthology.orchestra.run.smithi012.stdout: "uptime": 97.582409736000002 2024-02-18T11:40:11.178 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-02-18T11:40:11.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.178+0000 7ff4d9402640 1 -- 172.21.15.12:0/903171310 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7ff4c00a27e0 msgr2=0x7ff4c00a4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:11.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.178+0000 7ff4d9402640 1 --2- 172.21.15.12:0/903171310 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7ff4c00a27e0 0x7ff4c00a4ca0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4005e30 tx=0x7ff4c40323d0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.178+0000 7ff4d9402640 1 -- 172.21.15.12:0/903171310 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff4d41010f0 msgr2=0x7ff4d41970f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:11.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.178+0000 7ff4d9402640 1 --2- 172.21.15.12:0/903171310 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff4d41010f0 0x7ff4d41970f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc00d770 tx=0x7ff4cc00dc30 comp rx=0 tx=0).stop 2024-02-18T11:40:11.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.178+0000 7ff4d9402640 1 -- 172.21.15.12:0/903171310 shutdown_connections 2024-02-18T11:40:11.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.178+0000 7ff4d9402640 1 --2- 172.21.15.12:0/903171310 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7ff4c00a27e0 0x7ff4c00a4ca0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.178+0000 7ff4d9402640 1 --2- 172.21.15.12:0/903171310 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff4d41022e0 0x7ff4d4197630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.178+0000 7ff4d9402640 1 --2- 172.21.15.12:0/903171310 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff4d4102ca0 0x7ff4d419e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.178+0000 7ff4d9402640 1 --2- 172.21.15.12:0/903171310 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff4d41010f0 0x7ff4d41970f0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.178+0000 7ff4d9402640 1 -- 172.21.15.12:0/903171310 >> 172.21.15.12:0/903171310 conn(0x7ff4d40fc660 msgr2=0x7ff4d4105580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:11.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.178+0000 7ff4d9402640 1 -- 172.21.15.12:0/903171310 shutdown_connections 2024-02-18T11:40:11.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.178+0000 7ff4d9402640 1 -- 172.21.15.12:0/903171310 wait complete. 2024-02-18T11:40:11.180 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:40:11 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/309864830' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-18T11:40:11.180 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:40:11 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/3363054188' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-18T11:40:11.180 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:40:11 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/4203948026' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-18T11:40:11.183 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-02-18T11:40:11.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.292+0000 7f8b4ad4f640 1 -- 172.21.15.12:0/1619316619 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b441023c0 msgr2=0x7f8b44102840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:11.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.292+0000 7f8b4ad4f640 1 --2- 172.21.15.12:0/1619316619 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b441023c0 0x7f8b44102840 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8b34009900 tx=0x7f8b3402bc30 comp rx=0 tx=0).stop 2024-02-18T11:40:11.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.292+0000 7f8b4ad4f640 1 -- 172.21.15.12:0/1619316619 shutdown_connections 2024-02-18T11:40:11.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.292+0000 7f8b4ad4f640 1 --2- 172.21.15.12:0/1619316619 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b44102d80 0x7f8b44109610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.292+0000 7f8b4ad4f640 1 --2- 172.21.15.12:0/1619316619 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b441023c0 0x7f8b44102840 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.292+0000 7f8b4ad4f640 1 --2- 172.21.15.12:0/1619316619 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8b441011d0 0x7f8b441015f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.292+0000 7f8b4ad4f640 1 -- 172.21.15.12:0/1619316619 >> 172.21.15.12:0/1619316619 conn(0x7f8b440fc760 msgr2=0x7f8b440feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:11.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.292+0000 7f8b4ad4f640 1 -- 172.21.15.12:0/1619316619 shutdown_connections 2024-02-18T11:40:11.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.292+0000 7f8b4ad4f640 1 -- 172.21.15.12:0/1619316619 wait complete. 2024-02-18T11:40:11.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.293+0000 7f8b4ad4f640 1 Processor -- start 2024-02-18T11:40:11.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.293+0000 7f8b4ad4f640 1 -- start start 2024-02-18T11:40:11.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.293+0000 7f8b4ad4f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b441011d0 0x7f8b44074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:11.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.293+0000 7f8b4ad4f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b441023c0 0x7f8b440735b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:11.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.293+0000 7f8b4ad4f640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8b44102d80 0x7f8b44073af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:11.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.293+0000 7f8b4ad4f640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b44074250 con 0x7f8b441023c0 2024-02-18T11:40:11.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.293+0000 7f8b4ad4f640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b440743c0 con 0x7f8b44102d80 2024-02-18T11:40:11.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.293+0000 7f8b4ad4f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b441a12d0 con 0x7f8b441011d0 2024-02-18T11:40:11.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.293+0000 7f8b3bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b441023c0 0x7f8b440735b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:11.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.293+0000 7f8b3bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b441023c0 0x7f8b440735b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40216/0 (socket says 172.21.15.12:40216) 2024-02-18T11:40:11.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.293+0000 7f8b3bfff640 1 -- 172.21.15.12:0/2317593650 learned_addr learned my addr 172.21.15.12:0/2317593650 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:40:11.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.293+0000 7f8b492ec640 1 --2- 172.21.15.12:0/2317593650 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8b44102d80 0x7f8b44073af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:11.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.293+0000 7f8b48aeb640 1 --2- 172.21.15.12:0/2317593650 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b441011d0 0x7f8b44074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:11.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.293+0000 7f8b3bfff640 1 -- 172.21.15.12:0/2317593650 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b441011d0 msgr2=0x7f8b44074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:11.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.293+0000 7f8b3bfff640 1 --2- 172.21.15.12:0/2317593650 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b441011d0 0x7f8b44074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.293+0000 7f8b3bfff640 1 -- 172.21.15.12:0/2317593650 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8b44102d80 msgr2=0x7f8b44073af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:11.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.293+0000 7f8b3bfff640 1 --2- 172.21.15.12:0/2317593650 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8b44102d80 0x7f8b44073af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.293+0000 7f8b3bfff640 1 -- 172.21.15.12:0/2317593650 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b34009560 con 0x7f8b441023c0 2024-02-18T11:40:11.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.294+0000 7f8b3bfff640 1 --2- 172.21.15.12:0/2317593650 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b441023c0 0x7f8b440735b0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8b34036040 tx=0x7f8b340040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:11.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.294+0000 7f8b39ffb640 1 -- 172.21.15.12:0/2317593650 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b340044c0 con 0x7f8b441023c0 2024-02-18T11:40:11.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.294+0000 7f8b4ad4f640 1 -- 172.21.15.12:0/2317593650 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b441a1520 con 0x7f8b441023c0 2024-02-18T11:40:11.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.294+0000 7f8b39ffb640 1 -- 172.21.15.12:0/2317593650 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8b340052d0 con 0x7f8b441023c0 2024-02-18T11:40:11.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.294+0000 7f8b39ffb640 1 -- 172.21.15.12:0/2317593650 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b34003e10 con 0x7f8b441023c0 2024-02-18T11:40:11.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.294+0000 7f8b4ad4f640 1 -- 172.21.15.12:0/2317593650 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b441a1a80 con 0x7f8b441023c0 2024-02-18T11:40:11.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.296+0000 7f8b39ffb640 1 -- 172.21.15.12:0/2317593650 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 137296+0+0 (secure 0 0 0) 0x7f8b34005440 con 0x7f8b441023c0 2024-02-18T11:40:11.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.296+0000 7f8b4da04740 1 Processor -- start 2024-02-18T11:40:11.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.296+0000 7f8b39ffb640 1 --2- 172.21.15.12:0/2317593650 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f8b2c0a27e0 0x7f8b2c0a4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:11.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.296+0000 7f8b4da04740 1 -- start start 2024-02-18T11:40:11.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.297+0000 7f8b492ec640 1 --2- 172.21.15.12:0/2317593650 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f8b2c0a27e0 0x7f8b2c0a4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:11.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.297+0000 7f8b39ffb640 1 -- 172.21.15.12:0/2317593650 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f8b340f09f0 con 0x7f8b441023c0 2024-02-18T11:40:11.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.297+0000 7f8b4da04740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5595e2bb2070 0x5595e2bb2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:11.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.297+0000 7f8b4da04740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595e2bb2a60 0x5595e2bb7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:11.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.297+0000 7f8b4da04740 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5595e2bb7bd0 0x5595e2bba060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:11.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.297+0000 7f8b4da04740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595e2b6ddc0 con 0x5595e2bb2070 2024-02-18T11:40:11.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.297+0000 7f8b4da04740 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595e2b20f20 con 0x5595e2bb7bd0 2024-02-18T11:40:11.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.297+0000 7f8b4da04740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595e2b1e450 con 0x5595e2bb2a60 2024-02-18T11:40:11.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.297+0000 7f8b48aeb640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5595e2bb2070 0x5595e2bb2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:11.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.297+0000 7f8b48aeb640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5595e2bb2070 0x5595e2bb2490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40230/0 (socket says 172.21.15.12:40230) 2024-02-18T11:40:11.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.297+0000 7f8b48aeb640 1 -- 172.21.15.12:0/3460602501 learned_addr learned my addr 172.21.15.12:0/3460602501 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:40:11.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.297+0000 7f8b492ec640 1 --2- 172.21.15.12:0/3460602501 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5595e2bb7bd0 0x5595e2bba060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:11.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.297+0000 7f8b48aeb640 1 -- 172.21.15.12:0/3460602501 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595e2bb2a60 msgr2=0x5595e2bb7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:40:11.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.297+0000 7f8b3bfff640 1 --2- 172.21.15.12:0/3460602501 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595e2bb2a60 0x5595e2bb7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:11.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.297+0000 7f8b48aeb640 1 --2- 172.21.15.12:0/3460602501 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595e2bb2a60 0x5595e2bb7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.297+0000 7f8b48aeb640 1 -- 172.21.15.12:0/3460602501 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5595e2bb7bd0 msgr2=0x5595e2bba060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:40:11.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.297+0000 7f8b48aeb640 1 --2- 172.21.15.12:0/3460602501 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5595e2bb7bd0 0x5595e2bba060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.297+0000 7f8b48aeb640 1 -- 172.21.15.12:0/3460602501 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5595e2acef20 con 0x5595e2bb2070 2024-02-18T11:40:11.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.297+0000 7f8b492ec640 1 --2- 172.21.15.12:0/3460602501 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5595e2bb7bd0 0x5595e2bba060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:40:11.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.297+0000 7f8b492ec640 1 --2- 172.21.15.12:0/2317593650 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f8b2c0a27e0 0x7f8b2c0a4ca0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c005fd0 tx=0x7f8b3c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:11.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.297+0000 7f8b48aeb640 1 --2- 172.21.15.12:0/3460602501 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5595e2bb2070 0x5595e2bb2490 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f8b4000bd10 tx=0x7f8b4000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3f6f4828484ceb9 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:11.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.297+0000 7f8b2affd640 1 -- 172.21.15.12:0/3460602501 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b4001a460 con 0x5595e2bb2070 2024-02-18T11:40:11.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.297+0000 7f8b2affd640 1 -- 172.21.15.12:0/3460602501 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8b4001aaa0 con 0x5595e2bb2070 2024-02-18T11:40:11.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.297+0000 7f8b2affd640 1 -- 172.21.15.12:0/3460602501 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b40011ae0 con 0x5595e2bb2070 2024-02-18T11:40:11.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.298+0000 7f8b4da04740 1 -- 172.21.15.12:0/3460602501 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5595e2bb2070 msgr2=0x5595e2bb2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:11.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.298+0000 7f8b4da04740 1 --2- 172.21.15.12:0/3460602501 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5595e2bb2070 0x5595e2bb2490 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f8b4000bd10 tx=0x7f8b4000f730 comp rx=0 tx=0).stop 2024-02-18T11:40:11.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.298+0000 7f8b4da04740 1 -- 172.21.15.12:0/3460602501 shutdown_connections 2024-02-18T11:40:11.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.298+0000 7f8b4da04740 1 --2- 172.21.15.12:0/3460602501 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595e2bb2a60 0x5595e2bb7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.298+0000 7f8b4da04740 1 --2- 172.21.15.12:0/3460602501 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5595e2bb7bd0 0x5595e2bba060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.298+0000 7f8b4da04740 1 --2- 172.21.15.12:0/3460602501 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5595e2bb2070 0x5595e2bb2490 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.298+0000 7f8b4da04740 1 -- 172.21.15.12:0/3460602501 >> 172.21.15.12:0/3460602501 conn(0x5595e2bad5e0 msgr2=0x5595e2bad9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:11.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.298+0000 7f8b4da04740 1 -- 172.21.15.12:0/3460602501 shutdown_connections 2024-02-18T11:40:11.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.298+0000 7f8b4da04740 1 -- 172.21.15.12:0/3460602501 wait complete. 2024-02-18T11:40:11.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.298+0000 7f8b4da04740 1 -- start start 2024-02-18T11:40:11.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.298+0000 7f8b4da04740 1 Processor -- start 2024-02-18T11:40:11.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.299+0000 7f8b4da04740 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5595e2bb2070 0x5595e2bbc220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:11.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.299+0000 7f8b4da04740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5595e2bb2a60 0x5595e2bbc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:11.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.299+0000 7f8b4da04740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595e2bb7bd0 0x5595e2bc37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:11.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.299+0000 7f8b4da04740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595e2b19c30 con 0x5595e2bb2a60 2024-02-18T11:40:11.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.299+0000 7f8b4da04740 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595e2abed30 con 0x5595e2bb2070 2024-02-18T11:40:11.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.299+0000 7f8b4da04740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595e27ed310 con 0x5595e2bb7bd0 2024-02-18T11:40:11.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.299+0000 7f8b4da04740 10 client.0 _ll_register_callbacks cb 0x5595e2ac6d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-18T11:40:11.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.299+0000 7f8b3bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5595e2bb2a60 0x5595e2bbc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:11.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.299+0000 7f8b3bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5595e2bb2a60 0x5595e2bbc760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40232/0 (socket says 172.21.15.12:40232) 2024-02-18T11:40:11.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.299+0000 7f8b3bfff640 1 -- 172.21.15.12:0/1052754108 learned_addr learned my addr 172.21.15.12:0/1052754108 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:40:11.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.299+0000 7f8b48aeb640 1 --2- 172.21.15.12:0/1052754108 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5595e2bb2070 0x5595e2bbc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:11.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.299+0000 7f8b3bfff640 1 -- 172.21.15.12:0/1052754108 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595e2bb7bd0 msgr2=0x5595e2bc37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:40:11.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.299+0000 7f8b3bfff640 1 --2- 172.21.15.12:0/1052754108 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595e2bb7bd0 0x5595e2bc37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.299+0000 7f8b3bfff640 1 -- 172.21.15.12:0/1052754108 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5595e2bb2070 msgr2=0x5595e2bbc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:40:11.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.299+0000 7f8b3bfff640 1 --2- 172.21.15.12:0/1052754108 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5595e2bb2070 0x5595e2bbc220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.299+0000 7f8b3bfff640 1 -- 172.21.15.12:0/1052754108 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b4000b940 con 0x5595e2bb2a60 2024-02-18T11:40:11.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.299+0000 7f8b48aeb640 1 --2- 172.21.15.12:0/1052754108 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5595e2bb2070 0x5595e2bbc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:40:11.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.299+0000 7f8b3bfff640 1 --2- 172.21.15.12:0/1052754108 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5595e2bb2a60 0x5595e2bbc760 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f8b340b9e70 tx=0x7f8b340bce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=285ba5ab10451120 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:11.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.299+0000 7f8b23fff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-02-18T11:40:11.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.299+0000 7f8b23fff640 1 -- 172.21.15.12:0/1052754108 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b340bd4a0 con 0x5595e2bb2a60 2024-02-18T11:40:11.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.299+0000 7f8b23fff640 1 -- 172.21.15.12:0/1052754108 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f8b340c15f0 con 0x5595e2bb2a60 2024-02-18T11:40:11.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.299+0000 7f8b4da04740 1 -- 172.21.15.12:0/1052754108 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5595e28196c0 con 0x5595e2bb2a60 2024-02-18T11:40:11.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.300+0000 7f8b23fff640 1 -- 172.21.15.12:0/1052754108 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b340c5c40 con 0x5595e2bb2a60 2024-02-18T11:40:11.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.300+0000 7f8b4da04740 10 client.15072 fetch_fsmap learned FSMap version 20 2024-02-18T11:40:11.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.300+0000 7f8b4da04740 1 -- 172.21.15.12:0/1052754108 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x5595e2b182b0 con 0x5595e2bb2a60 2024-02-18T11:40:11.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.300+0000 7f8b23fff640 1 -- 172.21.15.12:0/1052754108 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5595e28196c0 con 0x5595e2bb2a60 2024-02-18T11:40:11.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.300+0000 7f8b23fff640 1 -- 172.21.15.12:0/1052754108 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8b3403c070 con 0x5595e2bb2a60 2024-02-18T11:40:11.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.300+0000 7f8b4da04740 10 client.15072 fetch_fsmap finished waiting for FSMap version 20 2024-02-18T11:40:11.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.300+0000 7f8b4da04740 10 client.15072 resolve_mds: resolved name 'b' to daemon mds.b 2024-02-18T11:40:11.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.300+0000 7f8b4da04740 20 client.15072 populate_metadata read hostname 'smithi012' 2024-02-18T11:40:11.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.300+0000 7f8b4da04740 1 --2- 172.21.15.12:0/1052754108 >> [v2:172.21.15.158:6838/1127200849,v1:172.21.15.158:6839/1127200849] conn(0x5595e2bc8900 0x5595e2bccd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:11.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.300+0000 7f8b4da04740 4 client.15072 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-18T11:40:11.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.300+0000 7f8b4da04740 1 -- 172.21.15.12:0/1052754108 --> [v2:172.21.15.158:6838/1127200849,v1:172.21.15.158:6839/1127200849] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5595e286a270 con 0x5595e2bc8900 2024-02-18T11:40:11.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.301+0000 7f8b48aeb640 1 --2- 172.21.15.12:0/1052754108 >> [v2:172.21.15.158:6838/1127200849,v1:172.21.15.158:6839/1127200849] conn(0x5595e2bc8900 0x5595e2bccd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:11.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.301+0000 7f8b23fff640 10 client.15072 ms_handle_connect on v2:172.21.15.158:6838/1127200849 2024-02-18T11:40:11.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.301+0000 7f8b48aeb640 1 --2- 172.21.15.12:0/1052754108 >> [v2:172.21.15.158:6838/1127200849,v1:172.21.15.158:6839/1127200849] conn(0x5595e2bc8900 0x5595e2bccd80 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f98f3bec6f1529fc server_cookie=416b8b64716bc09b in_seq=0 out_seq=0 2024-02-18T11:40:11.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.302+0000 7f8b23fff640 1 -- 172.21.15.12:0/1052754108 <== mds.0 v2:172.21.15.158:6838/1127200849 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5595e286a270 con 0x5595e2bc8900 2024-02-18T11:40:11.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.302+0000 7f8b23fff640 10 client.15072 handle_command_reply: tid=0 2024-02-18T11:40:11.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.302+0000 7f8b4da04740 1 client.15072 shutdown 2024-02-18T11:40:11.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.302+0000 7f8b4da04740 1 -- 172.21.15.12:0/1052754108 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5595e2bb2a60 msgr2=0x5595e2bbc760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:11.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.302+0000 7f8b4da04740 1 --2- 172.21.15.12:0/1052754108 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5595e2bb2a60 0x5595e2bbc760 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f8b340b9e70 tx=0x7f8b340bce00 comp rx=0 tx=0).stop 2024-02-18T11:40:11.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.302+0000 7f8b4da04740 1 -- 172.21.15.12:0/1052754108 shutdown_connections 2024-02-18T11:40:11.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.302+0000 7f8b4da04740 1 --2- 172.21.15.12:0/1052754108 >> [v2:172.21.15.158:6838/1127200849,v1:172.21.15.158:6839/1127200849] conn(0x5595e2bc8900 0x5595e2bccd80 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.302+0000 7f8b4da04740 1 --2- 172.21.15.12:0/1052754108 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595e2bb7bd0 0x5595e2bc37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.302+0000 7f8b4da04740 1 --2- 172.21.15.12:0/1052754108 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5595e2bb2a60 0x5595e2bbc760 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.302+0000 7f8b23fff640 0 client.15072 ms_handle_reset on v2:172.21.15.158:6838/1127200849 2024-02-18T11:40:11.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.302+0000 7f8b4da04740 1 --2- 172.21.15.12:0/1052754108 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5595e2bb2070 0x5595e2bbc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.302+0000 7f8b4da04740 1 -- 172.21.15.12:0/1052754108 >> 172.21.15.12:0/1052754108 conn(0x5595e2bad5e0 msgr2=0x5595e2bac8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:11.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.302+0000 7f8b4da04740 1 -- 172.21.15.12:0/1052754108 shutdown_connections 2024-02-18T11:40:11.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.302+0000 7f8b4da04740 1 -- 172.21.15.12:0/1052754108 wait complete. 2024-02-18T11:40:11.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.302+0000 7f8b4da04740 20 client.15072 trim_cache size 0 max 16384 2024-02-18T11:40:11.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.318+0000 7f8b4da04740 1 Processor -- start 2024-02-18T11:40:11.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.318+0000 7f8b4da04740 1 -- start start 2024-02-18T11:40:11.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.318+0000 7f8b4da04740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595e2bb2070 0x5595e2bbc220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:11.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.318+0000 7f8b4da04740 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5595e2bb2a60 0x5595e2bbc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:11.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.318+0000 7f8b4da04740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5595e2bb7bd0 0x5595e2bc3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:11.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.318+0000 7f8b4da04740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595e2a9f490 con 0x5595e2bb7bd0 2024-02-18T11:40:11.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.318+0000 7f8b4da04740 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595e2bc4420 con 0x5595e2bb2a60 2024-02-18T11:40:11.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.318+0000 7f8b4da04740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595e2bae030 con 0x5595e2bb2070 2024-02-18T11:40:11.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.318+0000 7f8b492ec640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5595e2bb7bd0 0x5595e2bc3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:11.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.318+0000 7f8b492ec640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5595e2bb7bd0 0x5595e2bc3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40248/0 (socket says 172.21.15.12:40248) 2024-02-18T11:40:11.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.318+0000 7f8b492ec640 1 -- 172.21.15.12:0/2506192688 learned_addr learned my addr 172.21.15.12:0/2506192688 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:40:11.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.318+0000 7f8b3bfff640 1 --2- 172.21.15.12:0/2506192688 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5595e2bb2a60 0x5595e2bbc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:11.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.318+0000 7f8b48aeb640 1 --2- 172.21.15.12:0/2506192688 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595e2bb2070 0x5595e2bbc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:11.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.318+0000 7f8b492ec640 1 -- 172.21.15.12:0/2506192688 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595e2bb2070 msgr2=0x5595e2bbc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:40:11.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.318+0000 7f8b492ec640 1 --2- 172.21.15.12:0/2506192688 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595e2bb2070 0x5595e2bbc220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.318+0000 7f8b492ec640 1 -- 172.21.15.12:0/2506192688 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5595e2bb2a60 msgr2=0x5595e2bbc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:40:11.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.318+0000 7f8b492ec640 1 --2- 172.21.15.12:0/2506192688 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5595e2bb2a60 0x5595e2bbc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.318+0000 7f8b492ec640 1 -- 172.21.15.12:0/2506192688 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5595e2ab4a20 con 0x5595e2bb7bd0 2024-02-18T11:40:11.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.318+0000 7f8b48aeb640 1 --2- 172.21.15.12:0/2506192688 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595e2bb2070 0x5595e2bbc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:40:11.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.318+0000 7f8b492ec640 1 --2- 172.21.15.12:0/2506192688 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5595e2bb7bd0 0x5595e2bc3760 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c011740 tx=0x7f8b3c013070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b7d8fc652dfd962 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:11.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.319+0000 7f8b2a7fc640 1 -- 172.21.15.12:0/2506192688 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c003b70 con 0x5595e2bb7bd0 2024-02-18T11:40:11.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.319+0000 7f8b2a7fc640 1 -- 172.21.15.12:0/2506192688 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8b3c003cd0 con 0x5595e2bb7bd0 2024-02-18T11:40:11.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.319+0000 7f8b2a7fc640 1 -- 172.21.15.12:0/2506192688 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c01eba0 con 0x5595e2bb7bd0 2024-02-18T11:40:11.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.319+0000 7f8b4da04740 1 -- 172.21.15.12:0/2506192688 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5595e2bb7bd0 msgr2=0x5595e2bc3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:11.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.319+0000 7f8b4da04740 1 --2- 172.21.15.12:0/2506192688 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5595e2bb7bd0 0x5595e2bc3760 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c011740 tx=0x7f8b3c013070 comp rx=0 tx=0).stop 2024-02-18T11:40:11.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.319+0000 7f8b4da04740 1 -- 172.21.15.12:0/2506192688 shutdown_connections 2024-02-18T11:40:11.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.319+0000 7f8b4da04740 1 --2- 172.21.15.12:0/2506192688 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5595e2bb7bd0 0x5595e2bc3760 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.319+0000 7f8b4da04740 1 --2- 172.21.15.12:0/2506192688 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5595e2bb2a60 0x5595e2bbc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.319+0000 7f8b4da04740 1 --2- 172.21.15.12:0/2506192688 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595e2bb2070 0x5595e2bbc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.319+0000 7f8b4da04740 1 -- 172.21.15.12:0/2506192688 >> 172.21.15.12:0/2506192688 conn(0x5595e2bad5e0 msgr2=0x5595e2bb3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:11.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.319+0000 7f8b4da04740 1 -- 172.21.15.12:0/2506192688 shutdown_connections 2024-02-18T11:40:11.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.319+0000 7f8b4da04740 1 -- 172.21.15.12:0/2506192688 wait complete. 2024-02-18T11:40:11.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.319+0000 7f8b4da04740 1 -- start start 2024-02-18T11:40:11.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.320+0000 7f8b4da04740 1 Processor -- start 2024-02-18T11:40:11.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.320+0000 7f8b4da04740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5595e2bb2070 0x5595e2bc0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:11.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.320+0000 7f8b4da04740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595e2bb2a60 0x5595e2bc0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:11.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.320+0000 7f8b4da04740 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5595e2bb7bd0 0x5595e2c1ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:11.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.320+0000 7f8b4da04740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595e2bb79b0 con 0x5595e2bb2070 2024-02-18T11:40:11.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.320+0000 7f8b4da04740 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595e2bc3a00 con 0x5595e2bb7bd0 2024-02-18T11:40:11.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.320+0000 7f8b4da04740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595e2bc3bc0 con 0x5595e2bb2a60 2024-02-18T11:40:11.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.320+0000 7f8b4da04740 10 client.0 _ll_register_callbacks cb 0x7f8b4000b4a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-18T11:40:11.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.320+0000 7f8b48aeb640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5595e2bb2070 0x5595e2bc0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:11.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.320+0000 7f8b48aeb640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5595e2bb2070 0x5595e2bc0540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40254/0 (socket says 172.21.15.12:40254) 2024-02-18T11:40:11.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.320+0000 7f8b48aeb640 1 -- 172.21.15.12:0/4124461125 learned_addr learned my addr 172.21.15.12:0/4124461125 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:40:11.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.320+0000 7f8b3bfff640 1 --2- 172.21.15.12:0/4124461125 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595e2bb2a60 0x5595e2bc0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:11.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.320+0000 7f8b492ec640 1 --2- 172.21.15.12:0/4124461125 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5595e2bb7bd0 0x5595e2c1ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:11.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.320+0000 7f8b48aeb640 1 -- 172.21.15.12:0/4124461125 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595e2bb2a60 msgr2=0x5595e2bc0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:40:11.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.320+0000 7f8b48aeb640 1 --2- 172.21.15.12:0/4124461125 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595e2bb2a60 0x5595e2bc0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.320+0000 7f8b48aeb640 1 -- 172.21.15.12:0/4124461125 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5595e2bb7bd0 msgr2=0x5595e2c1ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:40:11.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.320+0000 7f8b48aeb640 1 --2- 172.21.15.12:0/4124461125 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5595e2bb7bd0 0x5595e2c1ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.320+0000 7f8b48aeb640 1 -- 172.21.15.12:0/4124461125 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b3c011440 con 0x5595e2bb2070 2024-02-18T11:40:11.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.320+0000 7f8b492ec640 1 --2- 172.21.15.12:0/4124461125 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5595e2bb7bd0 0x5595e2c1ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:40:11.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.320+0000 7f8b48aeb640 1 --2- 172.21.15.12:0/4124461125 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5595e2bb2070 0x5595e2bc0540 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f8b4000bd10 tx=0x7f8b4000b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87439b6db2164c71 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:11.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.320+0000 7f8b23fff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-02-18T11:40:11.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.321+0000 7f8b23fff640 1 -- 172.21.15.12:0/4124461125 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b4001a770 con 0x5595e2bb2070 2024-02-18T11:40:11.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.321+0000 7f8b23fff640 1 -- 172.21.15.12:0/4124461125 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f8b4001a8d0 con 0x5595e2bb2070 2024-02-18T11:40:11.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.321+0000 7f8b4da04740 1 -- 172.21.15.12:0/4124461125 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5595e2bc8610 con 0x5595e2bb2070 2024-02-18T11:40:11.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.321+0000 7f8b23fff640 1 -- 172.21.15.12:0/4124461125 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b40027400 con 0x5595e2bb2070 2024-02-18T11:40:11.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.321+0000 7f8b4da04740 10 client.15078 fetch_fsmap learned FSMap version 20 2024-02-18T11:40:11.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.321+0000 7f8b4da04740 1 -- 172.21.15.12:0/4124461125 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x5595e2ada780 con 0x5595e2bb2070 2024-02-18T11:40:11.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.321+0000 7f8b23fff640 1 -- 172.21.15.12:0/4124461125 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5595e2bc8610 con 0x5595e2bb2070 2024-02-18T11:40:11.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.321+0000 7f8b23fff640 1 -- 172.21.15.12:0/4124461125 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8b40004810 con 0x5595e2bb2070 2024-02-18T11:40:11.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.321+0000 7f8b4da04740 10 client.15078 fetch_fsmap finished waiting for FSMap version 20 2024-02-18T11:40:11.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.321+0000 7f8b4da04740 10 client.15078 resolve_mds: resolved name 'b' to daemon mds.b 2024-02-18T11:40:11.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.321+0000 7f8b4da04740 20 client.15078 populate_metadata read hostname 'smithi012' 2024-02-18T11:40:11.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.321+0000 7f8b4da04740 1 --2- 172.21.15.12:0/4124461125 >> [v2:172.21.15.158:6838/1127200849,v1:172.21.15.158:6839/1127200849] conn(0x5595e2c24070 0x5595e2c204b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:11.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.321+0000 7f8b4da04740 4 client.15078 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "status"}] 2024-02-18T11:40:11.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.321+0000 7f8b4da04740 1 -- 172.21.15.12:0/4124461125 --> [v2:172.21.15.158:6838/1127200849,v1:172.21.15.158:6839/1127200849] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f8b4001aaa0 con 0x5595e2c24070 2024-02-18T11:40:11.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.321+0000 7f8b3bfff640 1 --2- 172.21.15.12:0/4124461125 >> [v2:172.21.15.158:6838/1127200849,v1:172.21.15.158:6839/1127200849] conn(0x5595e2c24070 0x5595e2c204b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:11.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.322+0000 7f8b23fff640 10 client.15078 ms_handle_connect on v2:172.21.15.158:6838/1127200849 2024-02-18T11:40:11.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.322+0000 7f8b3bfff640 1 --2- 172.21.15.12:0/4124461125 >> [v2:172.21.15.158:6838/1127200849,v1:172.21.15.158:6839/1127200849] conn(0x5595e2c24070 0x5595e2c204b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da03370daa1d842a server_cookie=8d2a3a16e9f0ebfe in_seq=0 out_seq=0 2024-02-18T11:40:11.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.322+0000 7f8b23fff640 1 -- 172.21.15.12:0/4124461125 <== mds.0 v2:172.21.15.158:6838/1127200849 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x7f8b4001aaa0 con 0x5595e2c24070 2024-02-18T11:40:11.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.322+0000 7f8b23fff640 10 client.15078 handle_command_reply: tid=0 2024-02-18T11:40:11.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.322+0000 7f8b4da04740 1 client.15078 shutdown 2024-02-18T11:40:11.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.322+0000 7f8b4da04740 1 -- 172.21.15.12:0/4124461125 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5595e2bb2070 msgr2=0x5595e2bc0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:11.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.322+0000 7f8b4da04740 1 --2- 172.21.15.12:0/4124461125 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5595e2bb2070 0x5595e2bc0540 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f8b4000bd10 tx=0x7f8b4000b690 comp rx=0 tx=0).stop 2024-02-18T11:40:11.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.322+0000 7f8b4da04740 1 -- 172.21.15.12:0/4124461125 shutdown_connections 2024-02-18T11:40:11.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.322+0000 7f8b4da04740 1 --2- 172.21.15.12:0/4124461125 >> [v2:172.21.15.158:6838/1127200849,v1:172.21.15.158:6839/1127200849] conn(0x5595e2c24070 0x5595e2c204b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.322+0000 7f8b4da04740 1 --2- 172.21.15.12:0/4124461125 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595e2bb2a60 0x5595e2bc0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.322+0000 7f8b4da04740 1 --2- 172.21.15.12:0/4124461125 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5595e2bb7bd0 0x5595e2c1ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.322+0000 7f8b4da04740 1 --2- 172.21.15.12:0/4124461125 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5595e2bb2070 0x5595e2bc0540 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.322+0000 7f8b23fff640 0 client.15078 ms_handle_reset on v2:172.21.15.158:6838/1127200849 2024-02-18T11:40:11.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.322+0000 7f8b4da04740 1 -- 172.21.15.12:0/4124461125 >> 172.21.15.12:0/4124461125 conn(0x5595e2bad5e0 msgr2=0x5595e2bb3f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:11.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.322+0000 7f8b4da04740 1 -- 172.21.15.12:0/4124461125 shutdown_connections 2024-02-18T11:40:11.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.322+0000 7f8b4da04740 1 -- 172.21.15.12:0/4124461125 wait complete. 2024-02-18T11:40:11.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.322+0000 7f8b4da04740 20 client.15078 trim_cache size 0 max 16384 2024-02-18T11:40:11.323 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-02-18T11:40:11.323 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_fsid": "0c8bb0de-ce51-11ee-95ba-87774f69a715", 2024-02-18T11:40:11.323 INFO:teuthology.orchestra.run.smithi012.stdout: "whoami": 0, 2024-02-18T11:40:11.323 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 24524, 2024-02-18T11:40:11.324 INFO:teuthology.orchestra.run.smithi012.stdout: "want_state": "up:active", 2024-02-18T11:40:11.324 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "up:active", 2024-02-18T11:40:11.324 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs", 2024-02-18T11:40:11.324 INFO:teuthology.orchestra.run.smithi012.stdout: "rank_uptime": 83.485212537999999, 2024-02-18T11:40:11.324 INFO:teuthology.orchestra.run.smithi012.stdout: "mdsmap_epoch": 20, 2024-02-18T11:40:11.324 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch": 89, 2024-02-18T11:40:11.324 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch_barrier": 87, 2024-02-18T11:40:11.324 INFO:teuthology.orchestra.run.smithi012.stdout: "uptime": 89.002159855000002 2024-02-18T11:40:11.324 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-02-18T11:40:11.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.323+0000 7f8b4ad4f640 1 -- 172.21.15.12:0/2317593650 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f8b2c0a27e0 msgr2=0x7f8b2c0a4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:11.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.323+0000 7f8b4ad4f640 1 --2- 172.21.15.12:0/2317593650 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f8b2c0a27e0 0x7f8b2c0a4ca0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c005fd0 tx=0x7f8b3c009500 comp rx=0 tx=0).stop 2024-02-18T11:40:11.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.323+0000 7f8b4ad4f640 1 -- 172.21.15.12:0/2317593650 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b441023c0 msgr2=0x7f8b440735b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:11.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.323+0000 7f8b4ad4f640 1 --2- 172.21.15.12:0/2317593650 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b441023c0 0x7f8b440735b0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8b34036040 tx=0x7f8b340040c0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.323+0000 7f8b4ad4f640 1 -- 172.21.15.12:0/2317593650 shutdown_connections 2024-02-18T11:40:11.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.323+0000 7f8b4ad4f640 1 --2- 172.21.15.12:0/2317593650 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f8b2c0a27e0 0x7f8b2c0a4ca0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.323+0000 7f8b4ad4f640 1 --2- 172.21.15.12:0/2317593650 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8b44102d80 0x7f8b44073af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.323+0000 7f8b4ad4f640 1 --2- 172.21.15.12:0/2317593650 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b441023c0 0x7f8b440735b0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.323+0000 7f8b4ad4f640 1 --2- 172.21.15.12:0/2317593650 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b441011d0 0x7f8b44074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:11.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.323+0000 7f8b4ad4f640 1 -- 172.21.15.12:0/2317593650 >> 172.21.15.12:0/2317593650 conn(0x7f8b440fc760 msgr2=0x7f8b441056c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:11.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.323+0000 7f8b4ad4f640 1 -- 172.21.15.12:0/2317593650 shutdown_connections 2024-02-18T11:40:11.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:11.323+0000 7f8b4ad4f640 1 -- 172.21.15.12:0/2317593650 wait complete. 2024-02-18T11:40:11.328 INFO:tasks.cephadm:Setup complete, yielding 2024-02-18T11:40:11.328 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-18T11:40:11.338 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi012.front.sepia.ceph.com 2024-02-18T11:40:11.339 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- bash -c 'ceph orch status' 2024-02-18T11:40:11.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:40:11 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/309864830' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-18T11:40:11.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:40:11 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/3363054188' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-18T11:40:11.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:40:11 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/4203948026' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-18T11:40:11.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:40:11 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/309864830' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-18T11:40:11.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:40:11 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/3363054188' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-18T11:40:11.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:40:11 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/4203948026' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-18T11:40:11.578 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T11:40:12.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:40:12 smithi012 ceph-mon[32460]: pgmap v329: 97 pgs: 97 active+clean; 639 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-18T11:40:12.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:40:12 smithi158 ceph-mon[36626]: pgmap v329: 97 pgs: 97 active+clean; 639 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-18T11:40:12.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:40:12 smithi188 ceph-mon[36925]: pgmap v329: 97 pgs: 97 active+clean; 639 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-18T11:40:13.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:13.126+0000 7fa7c69ba700 1 -- 172.21.15.12:0/2921779720 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7c0100f00 msgr2=0x7fa7c0101380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:13.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:13.126+0000 7fa7c69ba700 1 --2- 172.21.15.12:0/2921779720 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7c0100f00 0x7fa7c0101380 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4009990 tx=0x7fa7b4009ca0 comp rx=0 tx=0).stop 2024-02-18T11:40:13.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:13.126+0000 7fa7c69ba700 1 -- 172.21.15.12:0/2921779720 shutdown_connections 2024-02-18T11:40:13.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:13.126+0000 7fa7c69ba700 1 --2- 172.21.15.12:0/2921779720 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7c01018c0 0x7fa7c0101d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:13.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:13.126+0000 7fa7c69ba700 1 --2- 172.21.15.12:0/2921779720 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa7c01005a0 0x7fa7c01009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:13.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:13.126+0000 7fa7c69ba700 1 --2- 172.21.15.12:0/2921779720 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7c0100f00 0x7fa7c0101380 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:13.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:13.126+0000 7fa7c69ba700 1 -- 172.21.15.12:0/2921779720 >> 172.21.15.12:0/2921779720 conn(0x7fa7c00fc1a0 msgr2=0x7fa7c00fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:13.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:13.126+0000 7fa7c69ba700 1 -- 172.21.15.12:0/2921779720 shutdown_connections 2024-02-18T11:40:13.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:13.126+0000 7fa7c69ba700 1 -- 172.21.15.12:0/2921779720 wait complete. 2024-02-18T11:40:13.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:13.126+0000 7fa7c69ba700 1 Processor -- start 2024-02-18T11:40:13.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:13.126+0000 7fa7c69ba700 1 -- start start 2024-02-18T11:40:13.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:13.127+0000 7fa7c69ba700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7c01005a0 0x7fa7c0199e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:13.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:13.127+0000 7fa7c69ba700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7c0100f00 0x7fa7c019a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:13.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:13.127+0000 7fa7c69ba700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa7c01018c0 0x7fa7c01a13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:13.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:13.127+0000 7fa7c69ba700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c01a19b0 con 0x7fa7c0100f00 2024-02-18T11:40:13.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:13.127+0000 7fa7c69ba700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c01a1af0 con 0x7fa7c01018c0 2024-02-18T11:40:13.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:13.127+0000 7fa7c69ba700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c01a1c80 con 0x7fa7c01005a0 2024-02-18T11:40:13.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:13.127+0000 7fa7bf7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7c0100f00 0x7fa7c019a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:13.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:13.127+0000 7fa7bf7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7c0100f00 0x7fa7c019a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47830/0 (socket says 172.21.15.12:47830) 2024-02-18T11:40:13.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:13.127+0000 7fa7bf7fe700 1 -- 172.21.15.12:0/959276662 learned_addr learned my addr 172.21.15.12:0/959276662 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:40:13.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:13.127+0000 7fa7bffff700 1 --2- 172.21.15.12:0/959276662 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7c01005a0 0x7fa7c0199e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:13.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:13.127+0000 7fa7c4f57700 1 --2- 172.21.15.12:0/959276662 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa7c01018c0 0x7fa7c01a13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:13.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:13.127+0000 7fa7bf7fe700 1 -- 172.21.15.12:0/959276662 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7c01005a0 msgr2=0x7fa7c0199e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:13.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:13.127+0000 7fa7bf7fe700 1 --2- 172.21.15.12:0/959276662 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7c01005a0 0x7fa7c0199e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:13.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:13.127+0000 7fa7bf7fe700 1 -- 172.21.15.12:0/959276662 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa7c01018c0 msgr2=0x7fa7c01a13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:13.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:13.127+0000 7fa7bf7fe700 1 --2- 172.21.15.12:0/959276662 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa7c01018c0 0x7fa7c01a13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:13.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:13.127+0000 7fa7bf7fe700 1 -- 172.21.15.12:0/959276662 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7b40095f0 con 0x7fa7c0100f00 2024-02-18T11:40:13.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:13.127+0000 7fa7c4f57700 1 --2- 172.21.15.12:0/959276662 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa7c01018c0 0x7fa7c01a13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:40:13.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:13.127+0000 7fa7bf7fe700 1 --2- 172.21.15.12:0/959276662 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7c0100f00 0x7fa7c019a360 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4000c00 tx=0x7fa7b400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:13.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:13.127+0000 7fa7bd7fa700 1 -- 172.21.15.12:0/959276662 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b401d070 con 0x7fa7c0100f00 2024-02-18T11:40:13.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:13.127+0000 7fa7bd7fa700 1 -- 172.21.15.12:0/959276662 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa7b4005530 con 0x7fa7c0100f00 2024-02-18T11:40:13.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:13.127+0000 7fa7c69ba700 1 -- 172.21.15.12:0/959276662 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7c01a1f00 con 0x7fa7c0100f00 2024-02-18T11:40:13.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:13.128+0000 7fa7bd7fa700 1 -- 172.21.15.12:0/959276662 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b4004cb0 con 0x7fa7c0100f00 2024-02-18T11:40:13.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:13.128+0000 7fa7c69ba700 1 -- 172.21.15.12:0/959276662 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7c01a22b0 con 0x7fa7c0100f00 2024-02-18T11:40:13.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:13.129+0000 7fa7c69ba700 1 -- 172.21.15.12:0/959276662 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7c010b380 con 0x7fa7c0100f00 2024-02-18T11:40:13.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:13.130+0000 7fa7bd7fa700 1 -- 172.21.15.12:0/959276662 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 137296+0+0 (secure 0 0 0) 0x7fa7b400fc00 con 0x7fa7c0100f00 2024-02-18T11:40:13.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:13.130+0000 7fa7bd7fa700 1 --2- 172.21.15.12:0/959276662 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fa7a80a2870 0x7fa7a80a4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:13.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:13.130+0000 7fa7bffff700 1 --2- 172.21.15.12:0/959276662 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fa7a80a2870 0x7fa7a80a4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:13.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:13.130+0000 7fa7bd7fa700 1 -- 172.21.15.12:0/959276662 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa7b40cf7e0 con 0x7fa7c0100f00 2024-02-18T11:40:13.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:13.131+0000 7fa7bffff700 1 --2- 172.21.15.12:0/959276662 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fa7a80a2870 0x7fa7a80a4d30 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa7b0009a10 tx=0x7fa7b0009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:13.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:13.135+0000 7fa7bd7fa700 1 -- 172.21.15.12:0/959276662 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa7b40984b0 con 0x7fa7c0100f00 2024-02-18T11:40:13.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:13.356+0000 7fa7c69ba700 1 -- 172.21.15.12:0/959276662 --> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa7c002d920 con 0x7fa7a80a2870 2024-02-18T11:40:13.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:13.357+0000 7fa7bd7fa700 1 -- 172.21.15.12:0/959276662 <== mgr.14152 v2:172.21.15.12:6800/1153736685 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fa7c002d920 con 0x7fa7a80a2870 2024-02-18T11:40:13.351 INFO:teuthology.orchestra.run.smithi012.stdout:Backend: cephadm 2024-02-18T11:40:13.351 INFO:teuthology.orchestra.run.smithi012.stdout:Available: Yes 2024-02-18T11:40:13.351 INFO:teuthology.orchestra.run.smithi012.stdout:Paused: No 2024-02-18T11:40:13.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:13.359+0000 7fa7c69ba700 1 -- 172.21.15.12:0/959276662 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fa7a80a2870 msgr2=0x7fa7a80a4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:13.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:13.359+0000 7fa7c69ba700 1 --2- 172.21.15.12:0/959276662 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fa7a80a2870 0x7fa7a80a4d30 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa7b0009a10 tx=0x7fa7b0009190 comp rx=0 tx=0).stop 2024-02-18T11:40:13.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:13.359+0000 7fa7c69ba700 1 -- 172.21.15.12:0/959276662 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7c0100f00 msgr2=0x7fa7c019a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:13.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:13.359+0000 7fa7c69ba700 1 --2- 172.21.15.12:0/959276662 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7c0100f00 0x7fa7c019a360 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4000c00 tx=0x7fa7b400f690 comp rx=0 tx=0).stop 2024-02-18T11:40:13.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:13.359+0000 7fa7c69ba700 1 -- 172.21.15.12:0/959276662 shutdown_connections 2024-02-18T11:40:13.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:13.359+0000 7fa7c69ba700 1 --2- 172.21.15.12:0/959276662 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa7c01018c0 0x7fa7c01a13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:13.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:13.359+0000 7fa7c69ba700 1 --2- 172.21.15.12:0/959276662 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7c01005a0 0x7fa7c0199e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:13.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:13.359+0000 7fa7c69ba700 1 --2- 172.21.15.12:0/959276662 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fa7a80a2870 0x7fa7a80a4d30 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:13.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:13.359+0000 7fa7c69ba700 1 --2- 172.21.15.12:0/959276662 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7c0100f00 0x7fa7c019a360 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:13.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:13.359+0000 7fa7c69ba700 1 -- 172.21.15.12:0/959276662 >> 172.21.15.12:0/959276662 conn(0x7fa7c00fc1a0 msgr2=0x7fa7c00fdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:13.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:13.359+0000 7fa7c69ba700 1 -- 172.21.15.12:0/959276662 shutdown_connections 2024-02-18T11:40:13.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:13.359+0000 7fa7c69ba700 1 -- 172.21.15.12:0/959276662 wait complete. 2024-02-18T11:40:14.063 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- bash -c 'ceph orch ps' 2024-02-18T11:40:14.312 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T11:40:14.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:40:14 smithi012 ceph-mon[32460]: pgmap v330: 97 pgs: 97 active+clean; 639 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-18T11:40:14.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:40:14 smithi012 ceph-mon[32460]: from='client.15084 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:40:14.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:40:14 smithi158 ceph-mon[36626]: pgmap v330: 97 pgs: 97 active+clean; 639 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-18T11:40:14.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:40:14 smithi158 ceph-mon[36626]: from='client.15084 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:40:14.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:40:14 smithi188 ceph-mon[36925]: pgmap v330: 97 pgs: 97 active+clean; 639 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-18T11:40:14.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:40:14 smithi188 ceph-mon[36925]: from='client.15084 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:40:15.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:15.867+0000 7f0daae3a700 1 -- 172.21.15.12:0/2444958409 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0da4100f00 msgr2=0x7f0da4101360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:15.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:15.867+0000 7f0daae3a700 1 --2- 172.21.15.12:0/2444958409 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0da4100f00 0x7f0da4101360 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f0d90009990 tx=0x7f0d90009ca0 comp rx=0 tx=0).stop 2024-02-18T11:40:15.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:15.867+0000 7f0daae3a700 1 -- 172.21.15.12:0/2444958409 shutdown_connections 2024-02-18T11:40:15.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:15.867+0000 7f0daae3a700 1 --2- 172.21.15.12:0/2444958409 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0da41018a0 0x7f0da4109160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:15.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:15.867+0000 7f0daae3a700 1 --2- 172.21.15.12:0/2444958409 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0da41021c0 0x7f0da41025e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:15.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:15.867+0000 7f0daae3a700 1 --2- 172.21.15.12:0/2444958409 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0da4100f00 0x7f0da4101360 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:15.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:15.867+0000 7f0daae3a700 1 -- 172.21.15.12:0/2444958409 >> 172.21.15.12:0/2444958409 conn(0x7f0da40fb950 msgr2=0x7f0da40fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:15.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:15.867+0000 7f0daae3a700 1 -- 172.21.15.12:0/2444958409 shutdown_connections 2024-02-18T11:40:15.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:15.867+0000 7f0daae3a700 1 -- 172.21.15.12:0/2444958409 wait complete. 2024-02-18T11:40:15.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:15.867+0000 7f0daae3a700 1 Processor -- start 2024-02-18T11:40:15.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:15.867+0000 7f0daae3a700 1 -- start start 2024-02-18T11:40:15.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:15.868+0000 7f0daae3a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0da4100f00 0x7f0da4197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:15.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:15.868+0000 7f0daae3a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0da41018a0 0x7f0da41981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:15.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:15.868+0000 7f0daae3a700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0da41021c0 0x7f0da419f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:15.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:15.868+0000 7f0daae3a700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0da419f7f0 con 0x7f0da4100f00 2024-02-18T11:40:15.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:15.868+0000 7f0daae3a700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0da419f930 con 0x7f0da41021c0 2024-02-18T11:40:15.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:15.868+0000 7f0daae3a700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0da419faf0 con 0x7f0da41018a0 2024-02-18T11:40:15.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:15.868+0000 7f0da8bd6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0da4100f00 0x7f0da4197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:15.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:15.868+0000 7f0da8bd6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0da4100f00 0x7f0da4197c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47840/0 (socket says 172.21.15.12:47840) 2024-02-18T11:40:15.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:15.868+0000 7f0da8bd6700 1 -- 172.21.15.12:0/3929567811 learned_addr learned my addr 172.21.15.12:0/3929567811 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:40:15.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:15.868+0000 7f0da3fff700 1 --2- 172.21.15.12:0/3929567811 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0da41018a0 0x7f0da41981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:15.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:15.868+0000 7f0da8bd6700 1 -- 172.21.15.12:0/3929567811 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0da41018a0 msgr2=0x7f0da41981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:15.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:15.868+0000 7f0da93d7700 1 --2- 172.21.15.12:0/3929567811 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0da41021c0 0x7f0da419f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:15.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:15.868+0000 7f0da8bd6700 1 --2- 172.21.15.12:0/3929567811 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0da41018a0 0x7f0da41981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:15.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:15.868+0000 7f0da8bd6700 1 -- 172.21.15.12:0/3929567811 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0da41021c0 msgr2=0x7f0da419f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:15.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:15.868+0000 7f0da8bd6700 1 --2- 172.21.15.12:0/3929567811 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0da41021c0 0x7f0da419f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:15.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:15.868+0000 7f0da8bd6700 1 -- 172.21.15.12:0/3929567811 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d900095f0 con 0x7f0da4100f00 2024-02-18T11:40:15.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:15.868+0000 7f0da93d7700 1 --2- 172.21.15.12:0/3929567811 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0da41021c0 0x7f0da419f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:40:15.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:15.868+0000 7f0da8bd6700 1 --2- 172.21.15.12:0/3929567811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0da4100f00 0x7f0da4197c70 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0d9800d980 tx=0x7f0d9800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:15.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:15.868+0000 7f0da1ffb700 1 -- 172.21.15.12:0/3929567811 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d9800de60 con 0x7f0da4100f00 2024-02-18T11:40:15.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:15.868+0000 7f0da1ffb700 1 -- 172.21.15.12:0/3929567811 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0d98004500 con 0x7f0da4100f00 2024-02-18T11:40:15.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:15.868+0000 7f0daae3a700 1 -- 172.21.15.12:0/3929567811 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0da419fdd0 con 0x7f0da4100f00 2024-02-18T11:40:15.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:15.868+0000 7f0da1ffb700 1 -- 172.21.15.12:0/3929567811 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d98003ce0 con 0x7f0da4100f00 2024-02-18T11:40:15.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:15.869+0000 7f0daae3a700 1 -- 172.21.15.12:0/3929567811 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0da41a0210 con 0x7f0da4100f00 2024-02-18T11:40:15.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:15.870+0000 7f0daae3a700 1 -- 172.21.15.12:0/3929567811 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0da410c560 con 0x7f0da4100f00 2024-02-18T11:40:15.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:15.871+0000 7f0da1ffb700 1 -- 172.21.15.12:0/3929567811 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 137296+0+0 (secure 0 0 0) 0x7f0d98009cb0 con 0x7f0da4100f00 2024-02-18T11:40:15.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:15.871+0000 7f0da1ffb700 1 --2- 172.21.15.12:0/3929567811 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f0d940a2870 0x7f0d940a4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:15.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:15.871+0000 7f0da3fff700 1 --2- 172.21.15.12:0/3929567811 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f0d940a2870 0x7f0d940a4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:15.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:15.871+0000 7f0da1ffb700 1 -- 172.21.15.12:0/3929567811 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f0d980ce3c0 con 0x7f0da4100f00 2024-02-18T11:40:15.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:15.872+0000 7f0da3fff700 1 --2- 172.21.15.12:0/3929567811 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f0d940a2870 0x7f0d940a4d30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0d900038c0 tx=0x7f0d90005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:15.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:15.876+0000 7f0da1ffb700 1 -- 172.21.15.12:0/3929567811 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0d98097000 con 0x7f0da4100f00 2024-02-18T11:40:16.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:16.090+0000 7f0daae3a700 1 -- 172.21.15.12:0/3929567811 --> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f0da4100500 con 0x7f0d940a2870 2024-02-18T11:40:16.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:16.103+0000 7f0da1ffb700 1 -- 172.21.15.12:0/3929567811 <== mgr.14152 v2:172.21.15.12:6800/1153736685 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4185 (secure 0 0 0) 0x7f0da4100500 con 0x7f0d940a2870 2024-02-18T11:40:16.097 INFO:teuthology.orchestra.run.smithi012.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-18T11:40:16.097 INFO:teuthology.orchestra.run.smithi012.stdout:mds.a smithi012 running (97s) 15s ago 96s 12.8M - 19.0.0-1555-g1c920afa 53c33ee3bf62 651883f871d9 2024-02-18T11:40:16.097 INFO:teuthology.orchestra.run.smithi012.stdout:mds.b smithi158 running (94s) 27s ago 93s 28.2M - 19.0.0-1555-g1c920afa 53c33ee3bf62 d083ddecd08a 2024-02-18T11:40:16.098 INFO:teuthology.orchestra.run.smithi012.stdout:mds.c smithi188 running (108s) 27s ago 108s 19.5M - 19.0.0-1555-g1c920afa 53c33ee3bf62 b51fc279ee8a 2024-02-18T11:40:16.098 INFO:teuthology.orchestra.run.smithi012.stdout:mds.d smithi012 running (116s) 15s ago 116s 13.9M - 19.0.0-1555-g1c920afa 53c33ee3bf62 bb1d1e660e10 2024-02-18T11:40:16.098 INFO:teuthology.orchestra.run.smithi012.stdout:mds.e smithi158 running (114s) 27s ago 113s 15.9M - 19.0.0-1555-g1c920afa 53c33ee3bf62 b0f4c55b33cd 2024-02-18T11:40:16.098 INFO:teuthology.orchestra.run.smithi012.stdout:mds.f smithi188 running (111s) 27s ago 110s 18.2M - 19.0.0-1555-g1c920afa 53c33ee3bf62 38349ecccb97 2024-02-18T11:40:16.098 INFO:teuthology.orchestra.run.smithi012.stdout:mds.g smithi012 running (99s) 15s ago 99s 13.7M - 19.0.0-1555-g1c920afa 53c33ee3bf62 2d26a2be1b71 2024-02-18T11:40:16.098 INFO:teuthology.orchestra.run.smithi012.stdout:mds.h smithi158 running (105s) 27s ago 105s 19.2M - 19.0.0-1555-g1c920afa 53c33ee3bf62 93295aab3123 2024-02-18T11:40:16.098 INFO:teuthology.orchestra.run.smithi012.stdout:mds.i smithi188 running (103s) 27s ago 102s 17.6M - 19.0.0-1555-g1c920afa 53c33ee3bf62 7a8b1bd187ae 2024-02-18T11:40:16.098 INFO:teuthology.orchestra.run.smithi012.stdout:mds.j smithi012 running (119s) 15s ago 119s 14.0M - 19.0.0-1555-g1c920afa 53c33ee3bf62 b73432e950b9 2024-02-18T11:40:16.098 INFO:teuthology.orchestra.run.smithi012.stdout:mds.k smithi158 running (2m) 27s ago 2m 16.3M - 19.0.0-1555-g1c920afa 53c33ee3bf62 fa7b6fd22c88 2024-02-18T11:40:16.098 INFO:teuthology.orchestra.run.smithi012.stdout:mds.l smithi188 running (2m) 27s ago 2m 14.0M - 19.0.0-1555-g1c920afa 53c33ee3bf62 c3ce165c7789 2024-02-18T11:40:16.098 INFO:teuthology.orchestra.run.smithi012.stdout:mgr.x smithi012 *:9283,8765 running (9m) 15s ago 9m 513M - 19.0.0-1555-g1c920afa 53c33ee3bf62 d9409129c6e1 2024-02-18T11:40:16.098 INFO:teuthology.orchestra.run.smithi012.stdout:mgr.y smithi158 *:8443,8765 running (7m) 27s ago 7m 454M - 19.0.0-1555-g1c920afa 53c33ee3bf62 7fc0f5c3029b 2024-02-18T11:40:16.098 INFO:teuthology.orchestra.run.smithi012.stdout:mgr.z smithi188 *:8443,8765 running (7m) 27s ago 7m 452M - 19.0.0-1555-g1c920afa 53c33ee3bf62 9be720b49c12 2024-02-18T11:40:16.098 INFO:teuthology.orchestra.run.smithi012.stdout:mon.a smithi012 running (9m) 15s ago 9m 49.6M 2048M 19.0.0-1555-g1c920afa 53c33ee3bf62 74f2d580255e 2024-02-18T11:40:16.098 INFO:teuthology.orchestra.run.smithi012.stdout:mon.b smithi158 running (7m) 27s ago 7m 38.6M 2048M 19.0.0-1555-g1c920afa 53c33ee3bf62 bceb8ce2e0f3 2024-02-18T11:40:16.098 INFO:teuthology.orchestra.run.smithi012.stdout:mon.c smithi188 running (7m) 27s ago 7m 39.2M 2048M 19.0.0-1555-g1c920afa 53c33ee3bf62 7875018d620c 2024-02-18T11:40:16.099 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0 smithi012 running (6m) 15s ago 6m 55.3M 4273M 19.0.0-1555-g1c920afa 53c33ee3bf62 a7a69fd149bb 2024-02-18T11:40:16.099 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1 smithi158 running (6m) 27s ago 6m 52.5M 4270M 19.0.0-1555-g1c920afa 53c33ee3bf62 ae98728565fa 2024-02-18T11:40:16.099 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2 smithi188 running (6m) 27s ago 6m 49.4M 4270M 19.0.0-1555-g1c920afa 53c33ee3bf62 e3754460562c 2024-02-18T11:40:16.099 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3 smithi012 running (5m) 15s ago 5m 56.8M 4273M 19.0.0-1555-g1c920afa 53c33ee3bf62 fe248724d544 2024-02-18T11:40:16.099 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4 smithi158 running (5m) 27s ago 5m 54.4M 4270M 19.0.0-1555-g1c920afa 53c33ee3bf62 fd54360331ae 2024-02-18T11:40:16.099 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5 smithi188 running (4m) 27s ago 4m 53.6M 4270M 19.0.0-1555-g1c920afa 53c33ee3bf62 348268efc3b6 2024-02-18T11:40:16.099 INFO:teuthology.orchestra.run.smithi012.stdout:osd.6 smithi012 running (4m) 15s ago 4m 48.0M 4273M 19.0.0-1555-g1c920afa 53c33ee3bf62 23e6c62d1bea 2024-02-18T11:40:16.099 INFO:teuthology.orchestra.run.smithi012.stdout:osd.7 smithi158 running (4m) 27s ago 4m 53.4M 4270M 19.0.0-1555-g1c920afa 53c33ee3bf62 a821390f486c 2024-02-18T11:40:16.099 INFO:teuthology.orchestra.run.smithi012.stdout:osd.8 smithi188 running (3m) 27s ago 3m 53.1M 4270M 19.0.0-1555-g1c920afa 53c33ee3bf62 0af07b138499 2024-02-18T11:40:16.099 INFO:teuthology.orchestra.run.smithi012.stdout:osd.9 smithi012 running (3m) 15s ago 3m 43.9M 4273M 19.0.0-1555-g1c920afa 53c33ee3bf62 85fecb26ec7f 2024-02-18T11:40:16.099 INFO:teuthology.orchestra.run.smithi012.stdout:osd.10 smithi158 running (2m) 27s ago 2m 53.9M 4270M 19.0.0-1555-g1c920afa 53c33ee3bf62 ad4c95a3f804 2024-02-18T11:40:16.099 INFO:teuthology.orchestra.run.smithi012.stdout:osd.11 smithi188 running (2m) 27s ago 2m 53.6M 4270M 19.0.0-1555-g1c920afa 53c33ee3bf62 dce8e37b6a87 2024-02-18T11:40:16.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:16.106+0000 7f0daae3a700 1 -- 172.21.15.12:0/3929567811 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f0d940a2870 msgr2=0x7f0d940a4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:16.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:16.106+0000 7f0daae3a700 1 --2- 172.21.15.12:0/3929567811 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f0d940a2870 0x7f0d940a4d30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0d900038c0 tx=0x7f0d90005fb0 comp rx=0 tx=0).stop 2024-02-18T11:40:16.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:16.106+0000 7f0daae3a700 1 -- 172.21.15.12:0/3929567811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0da4100f00 msgr2=0x7f0da4197c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:16.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:16.106+0000 7f0daae3a700 1 --2- 172.21.15.12:0/3929567811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0da4100f00 0x7f0da4197c70 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0d9800d980 tx=0x7f0d9800dd40 comp rx=0 tx=0).stop 2024-02-18T11:40:16.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:16.106+0000 7f0daae3a700 1 -- 172.21.15.12:0/3929567811 shutdown_connections 2024-02-18T11:40:16.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:16.106+0000 7f0daae3a700 1 --2- 172.21.15.12:0/3929567811 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0da41021c0 0x7f0da419f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:16.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:16.106+0000 7f0daae3a700 1 --2- 172.21.15.12:0/3929567811 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f0d940a2870 0x7f0d940a4d30 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:16.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:16.106+0000 7f0daae3a700 1 --2- 172.21.15.12:0/3929567811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0da4100f00 0x7f0da4197c70 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:16.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:16.106+0000 7f0daae3a700 1 --2- 172.21.15.12:0/3929567811 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0da41018a0 0x7f0da41981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:16.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:16.106+0000 7f0daae3a700 1 -- 172.21.15.12:0/3929567811 >> 172.21.15.12:0/3929567811 conn(0x7f0da40fb950 msgr2=0x7f0da40ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:16.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:16.106+0000 7f0daae3a700 1 -- 172.21.15.12:0/3929567811 shutdown_connections 2024-02-18T11:40:16.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:16.106+0000 7f0daae3a700 1 -- 172.21.15.12:0/3929567811 wait complete. 2024-02-18T11:40:16.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:40:16 smithi012 ceph-mon[32460]: pgmap v331: 97 pgs: 97 active+clean; 639 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-18T11:40:16.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:40:16 smithi158 ceph-mon[36626]: pgmap v331: 97 pgs: 97 active+clean; 639 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-18T11:40:16.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:40:16 smithi188 ceph-mon[36925]: pgmap v331: 97 pgs: 97 active+clean; 639 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-18T11:40:16.745 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- bash -c 'ceph orch ls' 2024-02-18T11:40:16.992 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T11:40:17.396 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:40:17 smithi012 ceph-mon[32460]: from='client.15090 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:40:17.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:40:17 smithi158 ceph-mon[36626]: from='client.15090 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:40:17.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:40:17 smithi188 ceph-mon[36925]: from='client.15090 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:40:18.370 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:40:18 smithi012 ceph-mon[32460]: pgmap v332: 97 pgs: 97 active+clean; 639 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-02-18T11:40:18.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:40:18 smithi158 ceph-mon[36626]: pgmap v332: 97 pgs: 97 active+clean; 639 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-02-18T11:40:18.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:40:18 smithi188 ceph-mon[36925]: pgmap v332: 97 pgs: 97 active+clean; 639 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-02-18T11:40:18.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:18.629+0000 7f260973d700 1 -- 172.21.15.12:0/3885631767 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2604100c50 msgr2=0x7f2604101070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:18.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:18.629+0000 7f260973d700 1 --2- 172.21.15.12:0/3885631767 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2604100c50 0x7f2604101070 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f25f4009990 tx=0x7f25f4009ca0 comp rx=0 tx=0).stop 2024-02-18T11:40:18.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:18.629+0000 7f260973d700 1 -- 172.21.15.12:0/3885631767 shutdown_connections 2024-02-18T11:40:18.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:18.629+0000 7f260973d700 1 --2- 172.21.15.12:0/3885631767 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26041027f0 0x7f26041090a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:18.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:18.629+0000 7f260973d700 1 --2- 172.21.15.12:0/3885631767 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2604100c50 0x7f2604101070 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:18.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:18.629+0000 7f260973d700 1 --2- 172.21.15.12:0/3885631767 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2604101e50 0x7f26041022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:18.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:18.629+0000 7f260973d700 1 -- 172.21.15.12:0/3885631767 >> 172.21.15.12:0/3885631767 conn(0x7f26040fc210 msgr2=0x7f26040fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:18.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:18.629+0000 7f260973d700 1 -- 172.21.15.12:0/3885631767 shutdown_connections 2024-02-18T11:40:18.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:18.629+0000 7f260973d700 1 -- 172.21.15.12:0/3885631767 wait complete. 2024-02-18T11:40:18.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:18.630+0000 7f260973d700 1 Processor -- start 2024-02-18T11:40:18.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:18.630+0000 7f260973d700 1 -- start start 2024-02-18T11:40:18.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:18.630+0000 7f260973d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2604100c50 0x7f2604197d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:18.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:18.630+0000 7f260973d700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2604101e50 0x7f2604198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:18.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:18.630+0000 7f260973d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26041027f0 0x7f260419f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:18.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:18.630+0000 7f260973d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f260419f860 con 0x7f2604100c50 2024-02-18T11:40:18.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:18.630+0000 7f260973d700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f260419f9a0 con 0x7f2604101e50 2024-02-18T11:40:18.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:18.630+0000 7f260973d700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f260419fb60 con 0x7f26041027f0 2024-02-18T11:40:18.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:18.631+0000 7f2602ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2604100c50 0x7f2604197d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:18.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:18.631+0000 7f2602ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2604100c50 0x7f2604197d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47848/0 (socket says 172.21.15.12:47848) 2024-02-18T11:40:18.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:18.631+0000 7f2602ffd700 1 -- 172.21.15.12:0/1410503272 learned_addr learned my addr 172.21.15.12:0/1410503272 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:40:18.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:18.631+0000 7f26037fe700 1 --2- 172.21.15.12:0/1410503272 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26041027f0 0x7f260419f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:18.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:18.631+0000 7f26027fc700 1 --2- 172.21.15.12:0/1410503272 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2604101e50 0x7f2604198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:18.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:18.631+0000 7f2602ffd700 1 -- 172.21.15.12:0/1410503272 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26041027f0 msgr2=0x7f260419f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:18.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:18.631+0000 7f2602ffd700 1 --2- 172.21.15.12:0/1410503272 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26041027f0 0x7f260419f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:18.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:18.631+0000 7f2602ffd700 1 -- 172.21.15.12:0/1410503272 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2604101e50 msgr2=0x7f2604198250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:18.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:18.631+0000 7f2602ffd700 1 --2- 172.21.15.12:0/1410503272 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2604101e50 0x7f2604198250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:18.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:18.631+0000 7f2602ffd700 1 -- 172.21.15.12:0/1410503272 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25f40095f0 con 0x7f2604100c50 2024-02-18T11:40:18.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:18.631+0000 7f26037fe700 1 --2- 172.21.15.12:0/1410503272 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26041027f0 0x7f260419f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:40:18.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:18.631+0000 7f2602ffd700 1 --2- 172.21.15.12:0/1410503272 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2604100c50 0x7f2604197d10 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f25f40038c0 tx=0x7f25f400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:18.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:18.631+0000 7f25fbfff700 1 -- 172.21.15.12:0/1410503272 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f401d070 con 0x7f2604100c50 2024-02-18T11:40:18.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:18.631+0000 7f260973d700 1 -- 172.21.15.12:0/1410503272 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f260419fde0 con 0x7f2604100c50 2024-02-18T11:40:18.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:18.631+0000 7f260973d700 1 -- 172.21.15.12:0/1410503272 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26041a01a0 con 0x7f2604100c50 2024-02-18T11:40:18.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:18.631+0000 7f25fbfff700 1 -- 172.21.15.12:0/1410503272 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f25f4005510 con 0x7f2604100c50 2024-02-18T11:40:18.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:18.631+0000 7f25fbfff700 1 -- 172.21.15.12:0/1410503272 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f4004b30 con 0x7f2604100c50 2024-02-18T11:40:18.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:18.633+0000 7f260973d700 1 -- 172.21.15.12:0/1410503272 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2604108130 con 0x7f2604100c50 2024-02-18T11:40:18.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:18.634+0000 7f25fbfff700 1 -- 172.21.15.12:0/1410503272 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 137296+0+0 (secure 0 0 0) 0x7f25f4004c90 con 0x7f2604100c50 2024-02-18T11:40:18.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:18.634+0000 7f25fbfff700 1 --2- 172.21.15.12:0/1410503272 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f25f00a28f0 0x7f25f00a4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:18.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:18.634+0000 7f26027fc700 1 --2- 172.21.15.12:0/1410503272 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f25f00a28f0 0x7f25f00a4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:18.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:18.634+0000 7f25fbfff700 1 -- 172.21.15.12:0/1410503272 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f25f40cf510 con 0x7f2604100c50 2024-02-18T11:40:18.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:18.635+0000 7f26027fc700 1 --2- 172.21.15.12:0/1410503272 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f25f00a28f0 0x7f25f00a4db0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f25ec005950 tx=0x7f25ec00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:18.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:18.639+0000 7f25fbfff700 1 -- 172.21.15.12:0/1410503272 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f25f40981e0 con 0x7f2604100c50 2024-02-18T11:40:18.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:18.847+0000 7f260973d700 1 -- 172.21.15.12:0/1410503272 --> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f260402d950 con 0x7f25f00a28f0 2024-02-18T11:40:18.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:18.853+0000 7f25fbfff700 1 -- 172.21.15.12:0/1410503272 <== mgr.14152 v2:172.21.15.12:6800/1153736685 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7f260402d950 con 0x7f25f00a28f0 2024-02-18T11:40:18.847 INFO:teuthology.orchestra.run.smithi012.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-02-18T11:40:18.848 INFO:teuthology.orchestra.run.smithi012.stdout:mds.all 12/12 30s ago 2m smithi012=a;smithi012=d;smithi012=g;smithi012=j;smithi158=b;smithi158=e;smithi158=h;smithi158=k;smithi188=c;smithi188=f;smithi188=i;smithi188=l;count:12 2024-02-18T11:40:18.848 INFO:teuthology.orchestra.run.smithi012.stdout:mgr 3/3 30s ago 7m smithi012=x;smithi158=y;smithi188=z;count:3 2024-02-18T11:40:18.848 INFO:teuthology.orchestra.run.smithi012.stdout:mon 3/3 30s ago 7m smithi012:172.21.15.12=a;smithi158:172.21.15.158=b;smithi188:172.21.15.188=c;count:3 2024-02-18T11:40:18.848 INFO:teuthology.orchestra.run.smithi012.stdout:osd 12 30s ago - 2024-02-18T11:40:18.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:18.856+0000 7f260973d700 1 -- 172.21.15.12:0/1410503272 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f25f00a28f0 msgr2=0x7f25f00a4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:18.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:18.856+0000 7f260973d700 1 --2- 172.21.15.12:0/1410503272 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f25f00a28f0 0x7f25f00a4db0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f25ec005950 tx=0x7f25ec00b410 comp rx=0 tx=0).stop 2024-02-18T11:40:18.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:18.856+0000 7f260973d700 1 -- 172.21.15.12:0/1410503272 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2604100c50 msgr2=0x7f2604197d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:18.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:18.856+0000 7f260973d700 1 --2- 172.21.15.12:0/1410503272 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2604100c50 0x7f2604197d10 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f25f40038c0 tx=0x7f25f400f690 comp rx=0 tx=0).stop 2024-02-18T11:40:18.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:18.856+0000 7f260973d700 1 -- 172.21.15.12:0/1410503272 shutdown_connections 2024-02-18T11:40:18.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:18.856+0000 7f260973d700 1 --2- 172.21.15.12:0/1410503272 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26041027f0 0x7f260419f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:18.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:18.856+0000 7f260973d700 1 --2- 172.21.15.12:0/1410503272 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f25f00a28f0 0x7f25f00a4db0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:18.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:18.856+0000 7f260973d700 1 --2- 172.21.15.12:0/1410503272 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2604100c50 0x7f2604197d10 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:18.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:18.856+0000 7f260973d700 1 --2- 172.21.15.12:0/1410503272 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2604101e50 0x7f2604198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:18.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:18.856+0000 7f260973d700 1 -- 172.21.15.12:0/1410503272 >> 172.21.15.12:0/1410503272 conn(0x7f26040fc210 msgr2=0x7f2604105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:18.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:18.856+0000 7f260973d700 1 -- 172.21.15.12:0/1410503272 shutdown_connections 2024-02-18T11:40:18.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:18.856+0000 7f260973d700 1 -- 172.21.15.12:0/1410503272 wait complete. 2024-02-18T11:40:19.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:40:19 smithi012 ceph-mon[32460]: from='client.15096 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:40:19.460 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- bash -c 'ceph orch host ls' 2024-02-18T11:40:19.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:40:19 smithi158 ceph-mon[36626]: from='client.15096 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:40:19.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:40:19 smithi188 ceph-mon[36925]: from='client.15096 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:40:19.703 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T11:40:20.423 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:40:20 smithi012 ceph-mon[32460]: pgmap v333: 97 pgs: 97 active+clean; 639 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-18T11:40:20.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:40:20 smithi158 ceph-mon[36626]: pgmap v333: 97 pgs: 97 active+clean; 639 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-18T11:40:20.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:40:20 smithi188 ceph-mon[36925]: pgmap v333: 97 pgs: 97 active+clean; 639 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-18T11:40:21.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:21.184+0000 7fbcbf971700 1 -- 172.21.15.12:0/573836408 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbcb8101870 msgr2=0x7fbcb8109110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:21.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:21.184+0000 7fbcbf971700 1 --2- 172.21.15.12:0/573836408 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbcb8101870 0x7fbcb8109110 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fbcb000b0d0 tx=0x7fbcb000b3e0 comp rx=0 tx=0).stop 2024-02-18T11:40:21.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:21.185+0000 7fbcbf971700 1 -- 172.21.15.12:0/573836408 shutdown_connections 2024-02-18T11:40:21.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:21.185+0000 7fbcbf971700 1 --2- 172.21.15.12:0/573836408 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbcb8101870 0x7fbcb8109110 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:21.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:21.185+0000 7fbcbf971700 1 --2- 172.21.15.12:0/573836408 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbcb8102200 0x7fbcb8102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:21.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:21.185+0000 7fbcbf971700 1 --2- 172.21.15.12:0/573836408 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbcb8100eb0 0x7fbcb8101330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:21.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:21.185+0000 7fbcbf971700 1 -- 172.21.15.12:0/573836408 >> 172.21.15.12:0/573836408 conn(0x7fbcb80fb940 msgr2=0x7fbcb80fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:21.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:21.185+0000 7fbcbf971700 1 -- 172.21.15.12:0/573836408 shutdown_connections 2024-02-18T11:40:21.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:21.185+0000 7fbcbf971700 1 -- 172.21.15.12:0/573836408 wait complete. 2024-02-18T11:40:21.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:21.185+0000 7fbcbf971700 1 Processor -- start 2024-02-18T11:40:21.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:21.185+0000 7fbcbf971700 1 -- start start 2024-02-18T11:40:21.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:21.185+0000 7fbcbf971700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbcb8100eb0 0x7fbcb8197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:21.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:21.185+0000 7fbcbf971700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbcb8101870 0x7fbcb8198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:21.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:21.185+0000 7fbcbf971700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbcb8102200 0x7fbcb819f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:21.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:21.185+0000 7fbcbf971700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb819f980 con 0x7fbcb8100eb0 2024-02-18T11:40:21.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:21.185+0000 7fbcbf971700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb819fac0 con 0x7fbcb8102200 2024-02-18T11:40:21.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:21.185+0000 7fbcbf971700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb819fc50 con 0x7fbcb8101870 2024-02-18T11:40:21.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:21.186+0000 7fbcbd70d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbcb8100eb0 0x7fbcb8197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:21.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:21.186+0000 7fbcbd70d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbcb8100eb0 0x7fbcb8197df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47872/0 (socket says 172.21.15.12:47872) 2024-02-18T11:40:21.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:21.186+0000 7fbcbd70d700 1 -- 172.21.15.12:0/2525556601 learned_addr learned my addr 172.21.15.12:0/2525556601 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:40:21.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:21.186+0000 7fbcbcf0c700 1 --2- 172.21.15.12:0/2525556601 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbcb8101870 0x7fbcb8198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:21.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:21.186+0000 7fbcbdf0e700 1 --2- 172.21.15.12:0/2525556601 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbcb8102200 0x7fbcb819f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:21.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:21.186+0000 7fbcbd70d700 1 -- 172.21.15.12:0/2525556601 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbcb8101870 msgr2=0x7fbcb8198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:21.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:21.186+0000 7fbcbd70d700 1 --2- 172.21.15.12:0/2525556601 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbcb8101870 0x7fbcb8198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:21.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:21.186+0000 7fbcbd70d700 1 -- 172.21.15.12:0/2525556601 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbcb8102200 msgr2=0x7fbcb819f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:21.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:21.186+0000 7fbcbd70d700 1 --2- 172.21.15.12:0/2525556601 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbcb8102200 0x7fbcb819f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:21.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:21.186+0000 7fbcbd70d700 1 -- 172.21.15.12:0/2525556601 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcb0009d00 con 0x7fbcb8100eb0 2024-02-18T11:40:21.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:21.186+0000 7fbcbdf0e700 1 --2- 172.21.15.12:0/2525556601 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbcb8102200 0x7fbcb819f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:40:21.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:21.186+0000 7fbcbd70d700 1 --2- 172.21.15.12:0/2525556601 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbcb8100eb0 0x7fbcb8197df0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fbcb400d9b0 tx=0x7fbcb400dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:21.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:21.186+0000 7fbcae7fc700 1 -- 172.21.15.12:0/2525556601 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb400be80 con 0x7fbcb8100eb0 2024-02-18T11:40:21.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:21.186+0000 7fbcbf971700 1 -- 172.21.15.12:0/2525556601 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcb819ff30 con 0x7fbcb8100eb0 2024-02-18T11:40:21.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:21.186+0000 7fbcae7fc700 1 -- 172.21.15.12:0/2525556601 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbcb4004500 con 0x7fbcb8100eb0 2024-02-18T11:40:21.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:21.186+0000 7fbcae7fc700 1 -- 172.21.15.12:0/2525556601 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb4003be0 con 0x7fbcb8100eb0 2024-02-18T11:40:21.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:21.186+0000 7fbcbf971700 1 -- 172.21.15.12:0/2525556601 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcb81a0350 con 0x7fbcb8100eb0 2024-02-18T11:40:21.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:21.187+0000 7fbcbf971700 1 -- 172.21.15.12:0/2525556601 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcb80689a0 con 0x7fbcb8100eb0 2024-02-18T11:40:21.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:21.189+0000 7fbcae7fc700 1 -- 172.21.15.12:0/2525556601 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 137296+0+0 (secure 0 0 0) 0x7fbcb4009c00 con 0x7fbcb8100eb0 2024-02-18T11:40:21.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:21.190+0000 7fbcae7fc700 1 --2- 172.21.15.12:0/2525556601 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fbca40a28f0 0x7fbca40a4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:21.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:21.190+0000 7fbcbcf0c700 1 --2- 172.21.15.12:0/2525556601 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fbca40a28f0 0x7fbca40a4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:21.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:21.190+0000 7fbcae7fc700 1 -- 172.21.15.12:0/2525556601 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fbcb40ce5f0 con 0x7fbcb8100eb0 2024-02-18T11:40:21.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:21.190+0000 7fbcbcf0c700 1 --2- 172.21.15.12:0/2525556601 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fbca40a28f0 0x7fbca40a4db0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fbcb8101f30 tx=0x7fbca800bef0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:21.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:21.193+0000 7fbcae7fc700 1 -- 172.21.15.12:0/2525556601 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbcb40972b0 con 0x7fbcb8100eb0 2024-02-18T11:40:21.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:21.401+0000 7fbcbf971700 1 -- 172.21.15.12:0/2525556601 --> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fbcb81004b0 con 0x7fbca40a28f0 2024-02-18T11:40:21.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:21.402+0000 7fbcae7fc700 1 -- 172.21.15.12:0/2525556601 <== mgr.14152 v2:172.21.15.12:6800/1153736685 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7fbcb81004b0 con 0x7fbca40a28f0 2024-02-18T11:40:21.396 INFO:teuthology.orchestra.run.smithi012.stdout:HOST ADDR LABELS STATUS 2024-02-18T11:40:21.396 INFO:teuthology.orchestra.run.smithi012.stdout:smithi012 172.21.15.12 2024-02-18T11:40:21.396 INFO:teuthology.orchestra.run.smithi012.stdout:smithi158 172.21.15.158 2024-02-18T11:40:21.396 INFO:teuthology.orchestra.run.smithi012.stdout:smithi188 172.21.15.188 2024-02-18T11:40:21.396 INFO:teuthology.orchestra.run.smithi012.stdout:3 hosts in cluster 2024-02-18T11:40:21.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:21.405+0000 7fbcbf971700 1 -- 172.21.15.12:0/2525556601 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fbca40a28f0 msgr2=0x7fbca40a4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:21.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:21.405+0000 7fbcbf971700 1 --2- 172.21.15.12:0/2525556601 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fbca40a28f0 0x7fbca40a4db0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fbcb8101f30 tx=0x7fbca800bef0 comp rx=0 tx=0).stop 2024-02-18T11:40:21.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:21.405+0000 7fbcbf971700 1 -- 172.21.15.12:0/2525556601 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbcb8100eb0 msgr2=0x7fbcb8197df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:21.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:21.405+0000 7fbcbf971700 1 --2- 172.21.15.12:0/2525556601 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbcb8100eb0 0x7fbcb8197df0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fbcb400d9b0 tx=0x7fbcb400dd70 comp rx=0 tx=0).stop 2024-02-18T11:40:21.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:21.405+0000 7fbcbf971700 1 -- 172.21.15.12:0/2525556601 shutdown_connections 2024-02-18T11:40:21.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:21.405+0000 7fbcbf971700 1 --2- 172.21.15.12:0/2525556601 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbcb8102200 0x7fbcb819f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:21.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:21.405+0000 7fbcbf971700 1 --2- 172.21.15.12:0/2525556601 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fbca40a28f0 0x7fbca40a4db0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:21.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:21.405+0000 7fbcbf971700 1 --2- 172.21.15.12:0/2525556601 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbcb8100eb0 0x7fbcb8197df0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:21.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:21.405+0000 7fbcbf971700 1 --2- 172.21.15.12:0/2525556601 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbcb8101870 0x7fbcb8198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:21.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:21.405+0000 7fbcbf971700 1 -- 172.21.15.12:0/2525556601 >> 172.21.15.12:0/2525556601 conn(0x7fbcb80fb940 msgr2=0x7fbcb80ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:21.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:21.405+0000 7fbcbf971700 1 -- 172.21.15.12:0/2525556601 shutdown_connections 2024-02-18T11:40:21.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:21.405+0000 7fbcbf971700 1 -- 172.21.15.12:0/2525556601 wait complete. 2024-02-18T11:40:22.042 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- bash -c 'ceph orch device ls' 2024-02-18T11:40:22.293 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T11:40:22.415 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:40:22 smithi012 ceph-mon[32460]: pgmap v334: 97 pgs: 97 active+clean; 639 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-18T11:40:22.415 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:40:22 smithi012 ceph-mon[32460]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:40:22.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:40:22 smithi158 ceph-mon[36626]: pgmap v334: 97 pgs: 97 active+clean; 639 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-18T11:40:22.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:40:22 smithi158 ceph-mon[36626]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:40:22.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:40:22 smithi188 ceph-mon[36925]: pgmap v334: 97 pgs: 97 active+clean; 639 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-18T11:40:22.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:40:22 smithi188 ceph-mon[36925]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:40:23.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:40:23 smithi012 ceph-mon[32460]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-02-18T11:40:23.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:40:23 smithi158 ceph-mon[36626]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-02-18T11:40:23.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:40:23 smithi188 ceph-mon[36925]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-02-18T11:40:23.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:23.879+0000 7f86e53bb700 1 -- 172.21.15.12:0/2764984246 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86e01027f0 msgr2=0x7f86e0104be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:23.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:23.879+0000 7f86e53bb700 1 --2- 172.21.15.12:0/2764984246 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86e01027f0 0x7f86e0104be0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f86d800b0a0 tx=0x7f86d800b3b0 comp rx=0 tx=0).stop 2024-02-18T11:40:23.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:23.879+0000 7f86e53bb700 1 -- 172.21.15.12:0/2764984246 shutdown_connections 2024-02-18T11:40:23.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:23.879+0000 7f86e53bb700 1 --2- 172.21.15.12:0/2764984246 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f86e0107a50 0x7f86e0109e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:23.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:23.879+0000 7f86e53bb700 1 --2- 172.21.15.12:0/2764984246 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86e01027f0 0x7f86e0104be0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:23.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:23.879+0000 7f86e53bb700 1 --2- 172.21.15.12:0/2764984246 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f86e0105120 0x7f86e0107510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:23.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:23.879+0000 7f86e53bb700 1 -- 172.21.15.12:0/2764984246 >> 172.21.15.12:0/2764984246 conn(0x7f86e00fc180 msgr2=0x7f86e00fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:23.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:23.879+0000 7f86e53bb700 1 -- 172.21.15.12:0/2764984246 shutdown_connections 2024-02-18T11:40:23.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:23.879+0000 7f86e53bb700 1 -- 172.21.15.12:0/2764984246 wait complete. 2024-02-18T11:40:23.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:23.879+0000 7f86e53bb700 1 Processor -- start 2024-02-18T11:40:23.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:23.879+0000 7f86e53bb700 1 -- start start 2024-02-18T11:40:23.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:23.880+0000 7f86e53bb700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f86e01027f0 0x7f86e0198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:23.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:23.880+0000 7f86e53bb700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f86e0105120 0x7f86e0198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:23.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:23.880+0000 7f86e53bb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86e0107a50 0x7f86e019f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:23.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:23.880+0000 7f86e53bb700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e019fb90 con 0x7f86e0107a50 2024-02-18T11:40:23.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:23.880+0000 7f86e53bb700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e019fcd0 con 0x7f86e01027f0 2024-02-18T11:40:23.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:23.880+0000 7f86e53bb700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e019fe60 con 0x7f86e0105120 2024-02-18T11:40:23.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:23.880+0000 7f86df7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86e0107a50 0x7f86e019f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:23.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:23.880+0000 7f86df7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86e0107a50 0x7f86e019f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49840/0 (socket says 172.21.15.12:49840) 2024-02-18T11:40:23.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:23.880+0000 7f86df7fe700 1 -- 172.21.15.12:0/3332487392 learned_addr learned my addr 172.21.15.12:0/3332487392 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:40:23.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:23.880+0000 7f86deffd700 1 --2- 172.21.15.12:0/3332487392 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f86e01027f0 0x7f86e0198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:23.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:23.880+0000 7f86de7fc700 1 --2- 172.21.15.12:0/3332487392 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f86e0105120 0x7f86e0198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:23.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:23.880+0000 7f86df7fe700 1 -- 172.21.15.12:0/3332487392 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f86e0105120 msgr2=0x7f86e0198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:23.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:23.880+0000 7f86df7fe700 1 --2- 172.21.15.12:0/3332487392 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f86e0105120 0x7f86e0198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:23.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:23.880+0000 7f86df7fe700 1 -- 172.21.15.12:0/3332487392 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f86e01027f0 msgr2=0x7f86e0198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:23.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:23.880+0000 7f86df7fe700 1 --2- 172.21.15.12:0/3332487392 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f86e01027f0 0x7f86e0198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:23.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:23.880+0000 7f86df7fe700 1 -- 172.21.15.12:0/3332487392 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86d8009d00 con 0x7f86e0107a50 2024-02-18T11:40:23.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:23.880+0000 7f86de7fc700 1 --2- 172.21.15.12:0/3332487392 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f86e0105120 0x7f86e0198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:40:23.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:23.880+0000 7f86df7fe700 1 --2- 172.21.15.12:0/3332487392 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86e0107a50 0x7f86e019f5c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f86d000b4b0 tx=0x7f86d000b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:23.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:23.880+0000 7f86d7fff700 1 -- 172.21.15.12:0/3332487392 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d0011830 con 0x7f86e0107a50 2024-02-18T11:40:23.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:23.880+0000 7f86e53bb700 1 -- 172.21.15.12:0/3332487392 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86e01a0140 con 0x7f86e0107a50 2024-02-18T11:40:23.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:23.880+0000 7f86d7fff700 1 -- 172.21.15.12:0/3332487392 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f86d0011e70 con 0x7f86e0107a50 2024-02-18T11:40:23.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:23.880+0000 7f86d7fff700 1 -- 172.21.15.12:0/3332487392 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d000f2e0 con 0x7f86e0107a50 2024-02-18T11:40:23.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:23.881+0000 7f86e53bb700 1 -- 172.21.15.12:0/3332487392 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86e01a06c0 con 0x7f86e0107a50 2024-02-18T11:40:23.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:23.881+0000 7f86e53bb700 1 -- 172.21.15.12:0/3332487392 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86e00fdd00 con 0x7f86e0107a50 2024-02-18T11:40:23.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:23.883+0000 7f86d7fff700 1 -- 172.21.15.12:0/3332487392 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) v1 ==== 137245+0+0 (secure 0 0 0) 0x7f86d00105a0 con 0x7f86e0107a50 2024-02-18T11:40:23.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:23.883+0000 7f86d7fff700 1 --2- 172.21.15.12:0/3332487392 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f86cc0a2860 0x7f86cc0a4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:23.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:23.883+0000 7f86deffd700 1 --2- 172.21.15.12:0/3332487392 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f86cc0a2860 0x7f86cc0a4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:23.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:23.883+0000 7f86d7fff700 1 -- 172.21.15.12:0/3332487392 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f86d00ce310 con 0x7f86e0107a50 2024-02-18T11:40:23.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:23.884+0000 7f86deffd700 1 --2- 172.21.15.12:0/3332487392 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f86cc0a2860 0x7f86cc0a4d20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f86d8009fd0 tx=0x7f86d800b7b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:23.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:23.887+0000 7f86d7fff700 1 -- 172.21.15.12:0/3332487392 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f86d00970b0 con 0x7f86e0107a50 2024-02-18T11:40:24.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:24.099+0000 7f86e53bb700 1 -- 172.21.15.12:0/3332487392 --> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f86e002d840 con 0x7f86cc0a2860 2024-02-18T11:40:24.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:24.100+0000 7f86d7fff700 1 -- 172.21.15.12:0/3332487392 <== mgr.14152 v2:172.21.15.12:6800/1153736685 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7f86e002d840 con 0x7f86cc0a2860 2024-02-18T11:40:24.094 INFO:teuthology.orchestra.run.smithi012.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-02-18T11:40:24.094 INFO:teuthology.orchestra.run.smithi012.stdout:smithi012 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5331005J400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-02-18T11:40:24.094 INFO:teuthology.orchestra.run.smithi012.stdout:smithi158 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7475005A480DGN 447G No 2m ago Has a FileSystem, LVM detected 2024-02-18T11:40:24.095 INFO:teuthology.orchestra.run.smithi012.stdout:smithi188 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620500MP400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-02-18T11:40:24.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:24.102+0000 7f86e53bb700 1 -- 172.21.15.12:0/3332487392 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f86cc0a2860 msgr2=0x7f86cc0a4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:24.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:24.102+0000 7f86e53bb700 1 --2- 172.21.15.12:0/3332487392 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f86cc0a2860 0x7f86cc0a4d20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f86d8009fd0 tx=0x7f86d800b7b0 comp rx=0 tx=0).stop 2024-02-18T11:40:24.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:24.102+0000 7f86e53bb700 1 -- 172.21.15.12:0/3332487392 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86e0107a50 msgr2=0x7f86e019f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:24.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:24.102+0000 7f86e53bb700 1 --2- 172.21.15.12:0/3332487392 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86e0107a50 0x7f86e019f5c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f86d000b4b0 tx=0x7f86d000b7c0 comp rx=0 tx=0).stop 2024-02-18T11:40:24.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:24.102+0000 7f86e53bb700 1 -- 172.21.15.12:0/3332487392 shutdown_connections 2024-02-18T11:40:24.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:24.102+0000 7f86e53bb700 1 --2- 172.21.15.12:0/3332487392 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f86cc0a2860 0x7f86cc0a4d20 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:24.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:24.102+0000 7f86e53bb700 1 --2- 172.21.15.12:0/3332487392 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86e0107a50 0x7f86e019f5c0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:24.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:24.102+0000 7f86e53bb700 1 --2- 172.21.15.12:0/3332487392 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f86e01027f0 0x7f86e0198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:24.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:24.102+0000 7f86e53bb700 1 --2- 172.21.15.12:0/3332487392 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f86e0105120 0x7f86e0198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:24.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:24.102+0000 7f86e53bb700 1 -- 172.21.15.12:0/3332487392 >> 172.21.15.12:0/3332487392 conn(0x7f86e00fc180 msgr2=0x7f86e0103290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:24.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:24.103+0000 7f86e53bb700 1 -- 172.21.15.12:0/3332487392 shutdown_connections 2024-02-18T11:40:24.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:24.103+0000 7f86e53bb700 1 -- 172.21.15.12:0/3332487392 wait complete. 2024-02-18T11:40:24.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:40:24 smithi012 ceph-mon[32460]: pgmap v335: 97 pgs: 97 active+clean; 639 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 6 op/s 2024-02-18T11:40:24.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:40:24 smithi158 ceph-mon[36626]: pgmap v335: 97 pgs: 97 active+clean; 639 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 6 op/s 2024-02-18T11:40:24.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:40:24 smithi188 ceph-mon[36925]: pgmap v335: 97 pgs: 97 active+clean; 639 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 6 op/s 2024-02-18T11:40:24.782 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-18T11:40:24.792 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi012.front.sepia.ceph.com 2024-02-18T11:40:24.792 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- bash -c 'ceph fs dump' 2024-02-18T11:40:25.039 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T11:40:25.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:40:25 smithi012 ceph-mon[32460]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:40:25.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:40:25 smithi158 ceph-mon[36626]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:40:25.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:40:25 smithi188 ceph-mon[36925]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:40:26.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:40:26 smithi012 ceph-mon[32460]: pgmap v336: 97 pgs: 97 active+clean; 639 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-02-18T11:40:26.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:40:26 smithi158 ceph-mon[36626]: pgmap v336: 97 pgs: 97 active+clean; 639 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-02-18T11:40:26.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:40:26 smithi188 ceph-mon[36925]: pgmap v336: 97 pgs: 97 active+clean; 639 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-02-18T11:40:26.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:26.716+0000 7fc99249f700 1 -- 172.21.15.12:0/197960397 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc98c101ce0 msgr2=0x7fc98c102100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:26.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:26.716+0000 7fc99249f700 1 --2- 172.21.15.12:0/197960397 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc98c101ce0 0x7fc98c102100 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc97c0099c0 tx=0x7fc97c009cd0 comp rx=0 tx=0).stop 2024-02-18T11:40:26.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:26.716+0000 7fc99249f700 1 -- 172.21.15.12:0/197960397 shutdown_connections 2024-02-18T11:40:26.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:26.716+0000 7fc99249f700 1 --2- 172.21.15.12:0/197960397 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc98c103880 0x7fc98c10a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:26.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:26.716+0000 7fc99249f700 1 --2- 172.21.15.12:0/197960397 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc98c101ce0 0x7fc98c102100 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:26.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:26.716+0000 7fc99249f700 1 --2- 172.21.15.12:0/197960397 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc98c102ee0 0x7fc98c103340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:26.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:26.716+0000 7fc99249f700 1 -- 172.21.15.12:0/197960397 >> 172.21.15.12:0/197960397 conn(0x7fc98c0fd2c0 msgr2=0x7fc98c0ff6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:26.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:26.716+0000 7fc99249f700 1 -- 172.21.15.12:0/197960397 shutdown_connections 2024-02-18T11:40:26.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:26.716+0000 7fc99249f700 1 -- 172.21.15.12:0/197960397 wait complete. 2024-02-18T11:40:26.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:26.717+0000 7fc99249f700 1 Processor -- start 2024-02-18T11:40:26.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:26.717+0000 7fc99249f700 1 -- start start 2024-02-18T11:40:26.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:26.717+0000 7fc99249f700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc98c101ce0 0x7fc98c197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:26.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:26.717+0000 7fc99249f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc98c102ee0 0x7fc98c198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:26.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:26.717+0000 7fc99249f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc98c103880 0x7fc98c19f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:26.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:26.717+0000 7fc99249f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc98c19f890 con 0x7fc98c103880 2024-02-18T11:40:26.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:26.717+0000 7fc99249f700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc98c19f9d0 con 0x7fc98c101ce0 2024-02-18T11:40:26.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:26.717+0000 7fc99249f700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc98c19fb90 con 0x7fc98c102ee0 2024-02-18T11:40:26.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:26.717+0000 7fc990a3c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc98c103880 0x7fc98c19f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:26.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:26.718+0000 7fc990a3c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc98c103880 0x7fc98c19f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49856/0 (socket says 172.21.15.12:49856) 2024-02-18T11:40:26.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:26.718+0000 7fc990a3c700 1 -- 172.21.15.12:0/349760856 learned_addr learned my addr 172.21.15.12:0/349760856 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:40:26.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:26.718+0000 7fc98b7fe700 1 --2- 172.21.15.12:0/349760856 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc98c102ee0 0x7fc98c198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:26.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:26.718+0000 7fc98bfff700 1 --2- 172.21.15.12:0/349760856 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc98c101ce0 0x7fc98c197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:26.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:26.718+0000 7fc990a3c700 1 -- 172.21.15.12:0/349760856 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc98c102ee0 msgr2=0x7fc98c198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:26.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:26.718+0000 7fc990a3c700 1 --2- 172.21.15.12:0/349760856 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc98c102ee0 0x7fc98c198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:26.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:26.718+0000 7fc990a3c700 1 -- 172.21.15.12:0/349760856 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc98c101ce0 msgr2=0x7fc98c197d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:26.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:26.718+0000 7fc990a3c700 1 --2- 172.21.15.12:0/349760856 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc98c101ce0 0x7fc98c197d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:26.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:26.718+0000 7fc990a3c700 1 -- 172.21.15.12:0/349760856 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc97c0095f0 con 0x7fc98c103880 2024-02-18T11:40:26.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:26.718+0000 7fc990a3c700 1 --2- 172.21.15.12:0/349760856 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc98c103880 0x7fc98c19f2c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc984007ee0 tx=0x7fc98400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:26.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:26.718+0000 7fc9897fa700 1 -- 172.21.15.12:0/349760856 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc984017070 con 0x7fc98c103880 2024-02-18T11:40:26.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:26.718+0000 7fc9897fa700 1 -- 172.21.15.12:0/349760856 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc98400f040 con 0x7fc98c103880 2024-02-18T11:40:26.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:26.718+0000 7fc99249f700 1 -- 172.21.15.12:0/349760856 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc98c19fe70 con 0x7fc98c103880 2024-02-18T11:40:26.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:26.718+0000 7fc9897fa700 1 -- 172.21.15.12:0/349760856 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc984005760 con 0x7fc98c103880 2024-02-18T11:40:26.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:26.718+0000 7fc99249f700 1 -- 172.21.15.12:0/349760856 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc98c1a0280 con 0x7fc98c103880 2024-02-18T11:40:26.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:26.719+0000 7fc99249f700 1 -- 172.21.15.12:0/349760856 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc98c068970 con 0x7fc98c103880 2024-02-18T11:40:26.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:26.720+0000 7fc9897fa700 1 -- 172.21.15.12:0/349760856 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) v1 ==== 137245+0+0 (secure 0 0 0) 0x7fc9840058c0 con 0x7fc98c103880 2024-02-18T11:40:26.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:26.721+0000 7fc9897fa700 1 --2- 172.21.15.12:0/349760856 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fc9740a2860 0x7fc9740a4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:26.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:26.721+0000 7fc98bfff700 1 --2- 172.21.15.12:0/349760856 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fc9740a2860 0x7fc9740a4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:26.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:26.721+0000 7fc9897fa700 1 -- 172.21.15.12:0/349760856 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fc9840cdd50 con 0x7fc98c103880 2024-02-18T11:40:26.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:26.721+0000 7fc98bfff700 1 --2- 172.21.15.12:0/349760856 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fc9740a2860 0x7fc9740a4d20 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc97c009990 tx=0x7fc97c005ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:26.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:26.725+0000 7fc9897fa700 1 -- 172.21.15.12:0/349760856 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc984096a90 con 0x7fc98c103880 2024-02-18T11:40:26.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:26.972+0000 7fc99249f700 1 -- 172.21.15.12:0/349760856 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc98c109110 con 0x7fc98c103880 2024-02-18T11:40:26.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:26.973+0000 7fc9897fa700 1 -- 172.21.15.12:0/349760856 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+2945 (secure 0 0 0) 0x7fc984022250 con 0x7fc98c103880 2024-02-18T11:40:26.967 INFO:teuthology.orchestra.run.smithi012.stdout:e20 2024-02-18T11:40:26.967 INFO:teuthology.orchestra.run.smithi012.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-18T11:40:26.967 INFO:teuthology.orchestra.run.smithi012.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-18T11:40:26.967 INFO:teuthology.orchestra.run.smithi012.stdout:legacy client fscid: 1 2024-02-18T11:40:26.967 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:40:26.967 INFO:teuthology.orchestra.run.smithi012.stdout:Filesystem 'cephfs' (1) 2024-02-18T11:40:26.967 INFO:teuthology.orchestra.run.smithi012.stdout:fs_name cephfs 2024-02-18T11:40:26.968 INFO:teuthology.orchestra.run.smithi012.stdout:epoch 20 2024-02-18T11:40:26.968 INFO:teuthology.orchestra.run.smithi012.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-02-18T11:40:26.968 INFO:teuthology.orchestra.run.smithi012.stdout:created 2024-02-18T11:38:46.888173+0000 2024-02-18T11:40:26.968 INFO:teuthology.orchestra.run.smithi012.stdout:modified 2024-02-18T11:38:51.531552+0000 2024-02-18T11:40:26.968 INFO:teuthology.orchestra.run.smithi012.stdout:tableserver 0 2024-02-18T11:40:26.968 INFO:teuthology.orchestra.run.smithi012.stdout:root 0 2024-02-18T11:40:26.968 INFO:teuthology.orchestra.run.smithi012.stdout:session_timeout 300 2024-02-18T11:40:26.968 INFO:teuthology.orchestra.run.smithi012.stdout:session_autoclose 300 2024-02-18T11:40:26.968 INFO:teuthology.orchestra.run.smithi012.stdout:max_file_size 1099511627776 2024-02-18T11:40:26.968 INFO:teuthology.orchestra.run.smithi012.stdout:max_xattr_size 65536 2024-02-18T11:40:26.968 INFO:teuthology.orchestra.run.smithi012.stdout:required_client_features {} 2024-02-18T11:40:26.968 INFO:teuthology.orchestra.run.smithi012.stdout:last_failure 0 2024-02-18T11:40:26.969 INFO:teuthology.orchestra.run.smithi012.stdout:last_failure_osd_epoch 0 2024-02-18T11:40:26.969 INFO:teuthology.orchestra.run.smithi012.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-18T11:40:26.969 INFO:teuthology.orchestra.run.smithi012.stdout:max_mds 3 2024-02-18T11:40:26.969 INFO:teuthology.orchestra.run.smithi012.stdout:in 0,1,2 2024-02-18T11:40:26.969 INFO:teuthology.orchestra.run.smithi012.stdout:up {0=24524,1=24508,2=24478} 2024-02-18T11:40:26.969 INFO:teuthology.orchestra.run.smithi012.stdout:failed 2024-02-18T11:40:26.969 INFO:teuthology.orchestra.run.smithi012.stdout:damaged 2024-02-18T11:40:26.969 INFO:teuthology.orchestra.run.smithi012.stdout:stopped 2024-02-18T11:40:26.969 INFO:teuthology.orchestra.run.smithi012.stdout:data_pools [3] 2024-02-18T11:40:26.969 INFO:teuthology.orchestra.run.smithi012.stdout:metadata_pool 2 2024-02-18T11:40:26.969 INFO:teuthology.orchestra.run.smithi012.stdout:inline_data disabled 2024-02-18T11:40:26.969 INFO:teuthology.orchestra.run.smithi012.stdout:balancer 2024-02-18T11:40:26.969 INFO:teuthology.orchestra.run.smithi012.stdout:bal_rank_mask -1 2024-02-18T11:40:26.970 INFO:teuthology.orchestra.run.smithi012.stdout:standby_count_wanted 0 2024-02-18T11:40:26.970 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.b{0:24524} state up:active seq 3 addr [v2:172.21.15.158:6838/1127200849,v1:172.21.15.158:6839/1127200849] compat {c=[1],r=[1],i=[fff]}] 2024-02-18T11:40:26.970 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.h{0:24476} state up:standby-replay seq 1 addr [v2:172.21.15.158:6836/813251825,v1:172.21.15.158:6837/813251825] compat {c=[1],r=[1],i=[fff]}] 2024-02-18T11:40:26.970 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.i{1:24508} state up:active seq 6 addr [v2:172.21.15.188:6838/1342671190,v1:172.21.15.188:6839/1342671190] compat {c=[1],r=[1],i=[fff]}] 2024-02-18T11:40:26.970 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.f{1:24466} state up:standby-replay seq 1 addr [v2:172.21.15.188:6834/806709644,v1:172.21.15.188:6835/806709644] compat {c=[1],r=[1],i=[fff]}] 2024-02-18T11:40:26.970 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.c{2:24478} state up:active seq 7 addr [v2:172.21.15.188:6836/1036630143,v1:172.21.15.188:6837/1036630143] compat {c=[1],r=[1],i=[fff]}] 2024-02-18T11:40:26.970 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.e{2:24446} state up:standby-replay seq 1 addr [v2:172.21.15.158:6834/3673253642,v1:172.21.15.158:6835/3673253642] compat {c=[1],r=[1],i=[fff]}] 2024-02-18T11:40:26.970 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:40:26.970 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:40:26.970 INFO:teuthology.orchestra.run.smithi012.stdout:Standby daemons: 2024-02-18T11:40:26.970 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:40:26.970 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.j{-1:14526} state up:standby seq 1 addr [v2:172.21.15.12:6834/1788335302,v1:172.21.15.12:6835/1788335302] compat {c=[1],r=[1],i=[fff]}] 2024-02-18T11:40:26.970 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.d{-1:14544} state up:standby seq 1 addr [v2:172.21.15.12:6836/2144692455,v1:172.21.15.12:6837/2144692455] compat {c=[1],r=[1],i=[fff]}] 2024-02-18T11:40:26.970 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.g{-1:14610} state up:standby seq 1 addr [v2:172.21.15.12:6838/3569056036,v1:172.21.15.12:6839/3569056036] compat {c=[1],r=[1],i=[fff]}] 2024-02-18T11:40:26.971 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.a{-1:14622} state up:standby seq 1 addr [v2:172.21.15.12:6840/1973234486,v1:172.21.15.12:6841/1973234486] compat {c=[1],r=[1],i=[fff]}] 2024-02-18T11:40:26.971 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.l{-1:24409} state up:standby seq 1 addr [v2:172.21.15.188:6832/1207202465,v1:172.21.15.188:6833/1207202465] compat {c=[1],r=[1],i=[fff]}] 2024-02-18T11:40:26.971 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.k{-1:24410} state up:standby seq 1 addr [v2:172.21.15.158:6832/2636348323,v1:172.21.15.158:6833/2636348323] compat {c=[1],r=[1],i=[fff]}] 2024-02-18T11:40:26.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:26.975+0000 7fc99249f700 1 -- 172.21.15.12:0/349760856 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fc9740a2860 msgr2=0x7fc9740a4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:26.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:26.975+0000 7fc99249f700 1 --2- 172.21.15.12:0/349760856 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fc9740a2860 0x7fc9740a4d20 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc97c009990 tx=0x7fc97c005ad0 comp rx=0 tx=0).stop 2024-02-18T11:40:26.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:26.975+0000 7fc99249f700 1 -- 172.21.15.12:0/349760856 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc98c103880 msgr2=0x7fc98c19f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:26.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:26.975+0000 7fc99249f700 1 --2- 172.21.15.12:0/349760856 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc98c103880 0x7fc98c19f2c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc984007ee0 tx=0x7fc98400d3b0 comp rx=0 tx=0).stop 2024-02-18T11:40:26.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:26.975+0000 7fc99249f700 1 -- 172.21.15.12:0/349760856 shutdown_connections 2024-02-18T11:40:26.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:26.975+0000 7fc99249f700 1 --2- 172.21.15.12:0/349760856 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fc9740a2860 0x7fc9740a4d20 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:26.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:26.975+0000 7fc99249f700 1 --2- 172.21.15.12:0/349760856 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc98c103880 0x7fc98c19f2c0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:26.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:26.975+0000 7fc99249f700 1 --2- 172.21.15.12:0/349760856 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc98c101ce0 0x7fc98c197d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:26.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:26.975+0000 7fc99249f700 1 --2- 172.21.15.12:0/349760856 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc98c102ee0 0x7fc98c198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:26.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:26.975+0000 7fc99249f700 1 -- 172.21.15.12:0/349760856 >> 172.21.15.12:0/349760856 conn(0x7fc98c0fd2c0 msgr2=0x7fc98c106250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:26.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:26.975+0000 7fc99249f700 1 -- 172.21.15.12:0/349760856 shutdown_connections 2024-02-18T11:40:26.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:26.975+0000 7fc99249f700 1 -- 172.21.15.12:0/349760856 wait complete. 2024-02-18T11:40:26.972 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 20 2024-02-18T11:40:27.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:40:27 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/349760856' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-18T11:40:27.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:40:27 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/349760856' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-18T11:40:27.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:40:27 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/349760856' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-18T11:40:27.911 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- bash -c 'ceph osd dump' 2024-02-18T11:40:28.164 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T11:40:28.405 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:40:28 smithi012 ceph-mon[32460]: pgmap v337: 97 pgs: 97 active+clean; 640 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-02-18T11:40:28.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:40:28 smithi158 ceph-mon[36626]: pgmap v337: 97 pgs: 97 active+clean; 640 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-02-18T11:40:28.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:40:28 smithi188 ceph-mon[36925]: pgmap v337: 97 pgs: 97 active+clean; 640 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-02-18T11:40:29.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:29.787+0000 7fe24da1d700 1 -- 172.21.15.12:0/3400529909 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe248100be0 msgr2=0x7fe248101000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:29.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:29.787+0000 7fe24da1d700 1 --2- 172.21.15.12:0/3400529909 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe248100be0 0x7fe248101000 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fe2380099c0 tx=0x7fe238009cd0 comp rx=0 tx=0).stop 2024-02-18T11:40:29.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:29.787+0000 7fe24da1d700 1 -- 172.21.15.12:0/3400529909 shutdown_connections 2024-02-18T11:40:29.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:29.787+0000 7fe24da1d700 1 --2- 172.21.15.12:0/3400529909 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe248102780 0x7fe248109030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:29.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:29.787+0000 7fe24da1d700 1 --2- 172.21.15.12:0/3400529909 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe248100be0 0x7fe248101000 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:29.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:29.787+0000 7fe24da1d700 1 --2- 172.21.15.12:0/3400529909 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe248101de0 0x7fe248102240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:29.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:29.787+0000 7fe24da1d700 1 -- 172.21.15.12:0/3400529909 >> 172.21.15.12:0/3400529909 conn(0x7fe2480fc1a0 msgr2=0x7fe2480fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:29.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:29.787+0000 7fe24da1d700 1 -- 172.21.15.12:0/3400529909 shutdown_connections 2024-02-18T11:40:29.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:29.787+0000 7fe24da1d700 1 -- 172.21.15.12:0/3400529909 wait complete. 2024-02-18T11:40:29.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:29.788+0000 7fe24da1d700 1 Processor -- start 2024-02-18T11:40:29.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:29.788+0000 7fe24da1d700 1 -- start start 2024-02-18T11:40:29.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:29.788+0000 7fe24da1d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe248100be0 0x7fe24806af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:29.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:29.788+0000 7fe24da1d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe248101de0 0x7fe24819ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:29.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:29.788+0000 7fe24da1d700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe248102780 0x7fe24819f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:29.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:29.788+0000 7fe24da1d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe24819f850 con 0x7fe248101de0 2024-02-18T11:40:29.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:29.788+0000 7fe24da1d700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe24819f9c0 con 0x7fe248102780 2024-02-18T11:40:29.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:29.788+0000 7fe24da1d700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe24819fb80 con 0x7fe248100be0 2024-02-18T11:40:29.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:29.788+0000 7fe2467fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe248101de0 0x7fe24819ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:29.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:29.788+0000 7fe2467fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe248101de0 0x7fe24819ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49874/0 (socket says 172.21.15.12:49874) 2024-02-18T11:40:29.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:29.788+0000 7fe2467fc700 1 -- 172.21.15.12:0/932237400 learned_addr learned my addr 172.21.15.12:0/932237400 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:40:29.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:29.789+0000 7fe246ffd700 1 --2- 172.21.15.12:0/932237400 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe248100be0 0x7fe24806af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:29.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:29.789+0000 7fe2477fe700 1 --2- 172.21.15.12:0/932237400 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe248102780 0x7fe24819f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:29.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:29.789+0000 7fe2467fc700 1 -- 172.21.15.12:0/932237400 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe248100be0 msgr2=0x7fe24806af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:29.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:29.789+0000 7fe2467fc700 1 --2- 172.21.15.12:0/932237400 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe248100be0 0x7fe24806af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:29.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:29.789+0000 7fe2467fc700 1 -- 172.21.15.12:0/932237400 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe248102780 msgr2=0x7fe24819f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:29.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:29.789+0000 7fe2467fc700 1 --2- 172.21.15.12:0/932237400 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe248102780 0x7fe24819f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:29.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:29.789+0000 7fe2467fc700 1 -- 172.21.15.12:0/932237400 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2380095f0 con 0x7fe248101de0 2024-02-18T11:40:29.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:29.789+0000 7fe2477fe700 1 --2- 172.21.15.12:0/932237400 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe248102780 0x7fe24819f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:40:29.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:29.789+0000 7fe2467fc700 1 --2- 172.21.15.12:0/932237400 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe248101de0 0x7fe24819ca20 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe23000b530 tx=0x7fe23000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:29.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:29.789+0000 7fe24ca1b700 1 -- 172.21.15.12:0/932237400 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe230011830 con 0x7fe248101de0 2024-02-18T11:40:29.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:29.789+0000 7fe24ca1b700 1 -- 172.21.15.12:0/932237400 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe230011e70 con 0x7fe248101de0 2024-02-18T11:40:29.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:29.789+0000 7fe24da1d700 1 -- 172.21.15.12:0/932237400 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe24819fe60 con 0x7fe248101de0 2024-02-18T11:40:29.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:29.789+0000 7fe24ca1b700 1 -- 172.21.15.12:0/932237400 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23000f340 con 0x7fe248101de0 2024-02-18T11:40:29.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:29.789+0000 7fe24da1d700 1 -- 172.21.15.12:0/932237400 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2481a0320 con 0x7fe248101de0 2024-02-18T11:40:29.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:29.790+0000 7fe24da1d700 1 -- 172.21.15.12:0/932237400 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe248068970 con 0x7fe248101de0 2024-02-18T11:40:29.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:29.791+0000 7fe24ca1b700 1 -- 172.21.15.12:0/932237400 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) v1 ==== 137245+0+0 (secure 0 0 0) 0x7fe2300106d0 con 0x7fe248101de0 2024-02-18T11:40:29.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:29.792+0000 7fe24ca1b700 1 --2- 172.21.15.12:0/932237400 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fe2340a2860 0x7fe2340a4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:29.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:29.792+0000 7fe246ffd700 1 --2- 172.21.15.12:0/932237400 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fe2340a2860 0x7fe2340a4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:29.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:29.792+0000 7fe24ca1b700 1 -- 172.21.15.12:0/932237400 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe2300ce490 con 0x7fe248101de0 2024-02-18T11:40:29.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:29.792+0000 7fe246ffd700 1 --2- 172.21.15.12:0/932237400 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fe2340a2860 0x7fe2340a4d20 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe2380038c0 tx=0x7fe23800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:29.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:29.795+0000 7fe24ca1b700 1 -- 172.21.15.12:0/932237400 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe230097230 con 0x7fe248101de0 2024-02-18T11:40:29.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:29.996+0000 7fe24da1d700 1 -- 172.21.15.12:0/932237400 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7fe248108020 con 0x7fe248101de0 2024-02-18T11:40:29.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:29.997+0000 7fe24ca1b700 1 -- 172.21.15.12:0/932237400 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v89) v1 ==== 56+0+5285 (secure 0 0 0) 0x7fe230015070 con 0x7fe248101de0 2024-02-18T11:40:29.991 INFO:teuthology.orchestra.run.smithi012.stdout:epoch 89 2024-02-18T11:40:29.991 INFO:teuthology.orchestra.run.smithi012.stdout:fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 2024-02-18T11:40:29.991 INFO:teuthology.orchestra.run.smithi012.stdout:created 2024-02-18T11:30:49.110059+0000 2024-02-18T11:40:29.991 INFO:teuthology.orchestra.run.smithi012.stdout:modified 2024-02-18T11:39:52.478480+0000 2024-02-18T11:40:29.991 INFO:teuthology.orchestra.run.smithi012.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-02-18T11:40:29.991 INFO:teuthology.orchestra.run.smithi012.stdout:crush_version 26 2024-02-18T11:40:29.992 INFO:teuthology.orchestra.run.smithi012.stdout:full_ratio 0.95 2024-02-18T11:40:29.992 INFO:teuthology.orchestra.run.smithi012.stdout:backfillfull_ratio 0.9 2024-02-18T11:40:29.992 INFO:teuthology.orchestra.run.smithi012.stdout:nearfull_ratio 0.85 2024-02-18T11:40:29.992 INFO:teuthology.orchestra.run.smithi012.stdout:require_min_compat_client luminous 2024-02-18T11:40:29.992 INFO:teuthology.orchestra.run.smithi012.stdout:min_compat_client luminous 2024-02-18T11:40:29.992 INFO:teuthology.orchestra.run.smithi012.stdout:require_osd_release squid 2024-02-18T11:40:29.992 INFO:teuthology.orchestra.run.smithi012.stdout:stretch_mode_enabled false 2024-02-18T11:40:29.992 INFO:teuthology.orchestra.run.smithi012.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-02-18T11:40:29.992 INFO:teuthology.orchestra.run.smithi012.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 86 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-02-18T11:40:29.992 INFO:teuthology.orchestra.run.smithi012.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 87 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-02-18T11:40:29.992 INFO:teuthology.orchestra.run.smithi012.stdout:max_osd 12 2024-02-18T11:40:29.992 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0 up in weight 1 up_from 8 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.12:6802/536522426,v1:172.21.15.12:6803/536522426] [v2:172.21.15.12:6804/536522426,v1:172.21.15.12:6805/536522426] exists,up 35f15ea6-deb8-48a1-a7dd-ab649ec13533 2024-02-18T11:40:29.992 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1 up in weight 1 up_from 13 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.158:6800/4249111488,v1:172.21.15.158:6801/4249111488] [v2:172.21.15.158:6802/4249111488,v1:172.21.15.158:6803/4249111488] exists,up 2af937ef-d827-44e6-b805-70cffbfe1126 2024-02-18T11:40:29.992 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2 up in weight 1 up_from 18 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.188:6800/627976150,v1:172.21.15.188:6801/627976150] [v2:172.21.15.188:6802/627976150,v1:172.21.15.188:6803/627976150] exists,up 17b804dd-31df-45ae-a637-4a5f65bbaf26 2024-02-18T11:40:29.993 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3 up in weight 1 up_from 27 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.12:6810/3758512503,v1:172.21.15.12:6811/3758512503] [v2:172.21.15.12:6812/3758512503,v1:172.21.15.12:6813/3758512503] exists,up 3f9cc423-cdfa-4acf-aaa2-b58b32ccb9cf 2024-02-18T11:40:29.993 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4 up in weight 1 up_from 35 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.158:6808/3892820997,v1:172.21.15.158:6809/3892820997] [v2:172.21.15.158:6810/3892820997,v1:172.21.15.158:6811/3892820997] exists,up a92e20ec-af1c-4ef4-a0cb-caba5fcc71aa 2024-02-18T11:40:29.993 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5 up in weight 1 up_from 40 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.188:6808/3939495896,v1:172.21.15.188:6809/3939495896] [v2:172.21.15.188:6810/3939495896,v1:172.21.15.188:6811/3939495896] exists,up d9c2e0c8-85f5-4ae2-bf3e-125c1be3a2db 2024-02-18T11:40:29.993 INFO:teuthology.orchestra.run.smithi012.stdout:osd.6 up in weight 1 up_from 48 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.12:6818/1206337767,v1:172.21.15.12:6819/1206337767] [v2:172.21.15.12:6820/1206337767,v1:172.21.15.12:6821/1206337767] exists,up 4a523512-de6b-49f0-be81-00f2f00c1c10 2024-02-18T11:40:29.993 INFO:teuthology.orchestra.run.smithi012.stdout:osd.7 up in weight 1 up_from 53 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.158:6816/3283400032,v1:172.21.15.158:6817/3283400032] [v2:172.21.15.158:6818/3283400032,v1:172.21.15.158:6819/3283400032] exists,up 363271a6-df93-44c9-b0e5-3d29768aa96c 2024-02-18T11:40:29.993 INFO:teuthology.orchestra.run.smithi012.stdout:osd.8 up in weight 1 up_from 61 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.188:6816/2110170682,v1:172.21.15.188:6817/2110170682] [v2:172.21.15.188:6818/2110170682,v1:172.21.15.188:6819/2110170682] exists,up 9a089528-3e8e-45d3-9279-ddb54088b96a 2024-02-18T11:40:29.993 INFO:teuthology.orchestra.run.smithi012.stdout:osd.9 up in weight 1 up_from 69 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.12:6826/1825928986,v1:172.21.15.12:6827/1825928986] [v2:172.21.15.12:6828/1825928986,v1:172.21.15.12:6829/1825928986] exists,up b12e2727-d649-4464-90e1-fafbc1754832 2024-02-18T11:40:29.993 INFO:teuthology.orchestra.run.smithi012.stdout:osd.10 up in weight 1 up_from 74 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.158:6824/3239052167,v1:172.21.15.158:6825/3239052167] [v2:172.21.15.158:6826/3239052167,v1:172.21.15.158:6827/3239052167] exists,up 518b4cbf-9d90-4e9a-8ac9-b5cb9b3dee4a 2024-02-18T11:40:29.993 INFO:teuthology.orchestra.run.smithi012.stdout:osd.11 up in weight 1 up_from 82 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.188:6824/1135773350,v1:172.21.15.188:6825/1135773350] [v2:172.21.15.188:6826/1135773350,v1:172.21.15.188:6827/1135773350] exists,up 98aa3a0d-4b2f-46c8-9341-375d13794c3c 2024-02-18T11:40:29.993 INFO:teuthology.orchestra.run.smithi012.stdout:pg_upmap_items 2.4 [2,11] 2024-02-18T11:40:29.993 INFO:teuthology.orchestra.run.smithi012.stdout:pg_upmap_items 2.f [8,11] 2024-02-18T11:40:29.993 INFO:teuthology.orchestra.run.smithi012.stdout:pg_upmap_items 3.2 [3,9] 2024-02-18T11:40:29.993 INFO:teuthology.orchestra.run.smithi012.stdout:pg_upmap_items 3.13 [4,9] 2024-02-18T11:40:29.993 INFO:teuthology.orchestra.run.smithi012.stdout:pg_upmap_items 3.24 [4,9] 2024-02-18T11:40:29.993 INFO:teuthology.orchestra.run.smithi012.stdout:pg_upmap_items 3.2a [3,8] 2024-02-18T11:40:29.994 INFO:teuthology.orchestra.run.smithi012.stdout:pg_upmap_items 3.31 [4,6] 2024-02-18T11:40:29.994 INFO:teuthology.orchestra.run.smithi012.stdout:pg_upmap_items 3.37 [5,8] 2024-02-18T11:40:29.994 INFO:teuthology.orchestra.run.smithi012.stdout:blocklist 172.21.15.12:6801/4135274411 expires 2024-02-19T11:31:50.936928+0000 2024-02-18T11:40:29.994 INFO:teuthology.orchestra.run.smithi012.stdout:blocklist 172.21.15.12:6800/4135274411 expires 2024-02-19T11:31:50.936928+0000 2024-02-18T11:40:29.994 INFO:teuthology.orchestra.run.smithi012.stdout:blocklist 172.21.15.12:0/118428584 expires 2024-02-19T11:31:50.936928+0000 2024-02-18T11:40:29.994 INFO:teuthology.orchestra.run.smithi012.stdout:blocklist 172.21.15.12:0/4276810915 expires 2024-02-19T11:31:50.936928+0000 2024-02-18T11:40:29.994 INFO:teuthology.orchestra.run.smithi012.stdout:blocklist 172.21.15.12:0/3777545262 expires 2024-02-19T11:31:24.000830+0000 2024-02-18T11:40:29.994 INFO:teuthology.orchestra.run.smithi012.stdout:blocklist 172.21.15.12:0/114707743 expires 2024-02-19T11:31:24.000830+0000 2024-02-18T11:40:29.994 INFO:teuthology.orchestra.run.smithi012.stdout:blocklist 172.21.15.12:6800/3925204723 expires 2024-02-19T11:31:24.000830+0000 2024-02-18T11:40:29.994 INFO:teuthology.orchestra.run.smithi012.stdout:blocklist 172.21.15.12:6801/3925204723 expires 2024-02-19T11:31:24.000830+0000 2024-02-18T11:40:29.994 INFO:teuthology.orchestra.run.smithi012.stdout:blocklist 172.21.15.12:0/74881618 expires 2024-02-19T11:31:24.000830+0000 2024-02-18T11:40:29.994 INFO:teuthology.orchestra.run.smithi012.stdout:blocklist 172.21.15.12:0/1614805375 expires 2024-02-19T11:31:50.936928+0000 2024-02-18T11:40:29.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:29.999+0000 7fe24da1d700 1 -- 172.21.15.12:0/932237400 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fe2340a2860 msgr2=0x7fe2340a4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:29.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:29.999+0000 7fe24da1d700 1 --2- 172.21.15.12:0/932237400 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fe2340a2860 0x7fe2340a4d20 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe2380038c0 tx=0x7fe23800b540 comp rx=0 tx=0).stop 2024-02-18T11:40:29.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:29.999+0000 7fe24da1d700 1 -- 172.21.15.12:0/932237400 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe248101de0 msgr2=0x7fe24819ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:29.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:29.999+0000 7fe24da1d700 1 --2- 172.21.15.12:0/932237400 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe248101de0 0x7fe24819ca20 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe23000b530 tx=0x7fe23000b840 comp rx=0 tx=0).stop 2024-02-18T11:40:29.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:29.999+0000 7fe24da1d700 1 -- 172.21.15.12:0/932237400 shutdown_connections 2024-02-18T11:40:29.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:29.999+0000 7fe24da1d700 1 --2- 172.21.15.12:0/932237400 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe248102780 0x7fe24819f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:29.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:29.999+0000 7fe24da1d700 1 --2- 172.21.15.12:0/932237400 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe248100be0 0x7fe24806af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:29.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:29.999+0000 7fe24da1d700 1 --2- 172.21.15.12:0/932237400 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fe2340a2860 0x7fe2340a4d20 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:29.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:29.999+0000 7fe24da1d700 1 --2- 172.21.15.12:0/932237400 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe248101de0 0x7fe24819ca20 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:29.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:29.999+0000 7fe24da1d700 1 -- 172.21.15.12:0/932237400 >> 172.21.15.12:0/932237400 conn(0x7fe2480fc1a0 msgr2=0x7fe248105160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:29.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:29.999+0000 7fe24da1d700 1 -- 172.21.15.12:0/932237400 shutdown_connections 2024-02-18T11:40:29.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:29.999+0000 7fe24da1d700 1 -- 172.21.15.12:0/932237400 wait complete. 2024-02-18T11:40:30.365 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:40:30 smithi012 ceph-mon[32460]: pgmap v338: 97 pgs: 97 active+clean; 640 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-02-18T11:40:30.365 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:40:30 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/932237400' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-02-18T11:40:30.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:40:30 smithi158 ceph-mon[36626]: pgmap v338: 97 pgs: 97 active+clean; 640 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-02-18T11:40:30.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:40:30 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/932237400' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-02-18T11:40:30.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:40:30 smithi188 ceph-mon[36925]: pgmap v338: 97 pgs: 97 active+clean; 640 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-02-18T11:40:30.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:40:30 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/932237400' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-02-18T11:40:30.693 INFO:teuthology.run_tasks:Running task fs.ready... 2024-02-18T11:40:30.704 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-18T11:40:30.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:30.816+0000 7f6f9dbd7640 1 -- 172.21.15.12:0/738938241 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f98104610 msgr2=0x7f6f98106a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:30.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:30.816+0000 7f6f9dbd7640 1 --2- 172.21.15.12:0/738938241 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f98104610 0x7f6f98106a00 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c009900 tx=0x7f6f8c02bc30 comp rx=0 tx=0).stop 2024-02-18T11:40:30.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:30.816+0000 7f6f9dbd7640 1 -- 172.21.15.12:0/738938241 shutdown_connections 2024-02-18T11:40:30.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:30.816+0000 7f6f9dbd7640 1 --2- 172.21.15.12:0/738938241 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6f98106f40 0x7f6f98109390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:30.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:30.816+0000 7f6f9dbd7640 1 --2- 172.21.15.12:0/738938241 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f98104610 0x7f6f98106a00 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:30.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:30.816+0000 7f6f9dbd7640 1 --2- 172.21.15.12:0/738938241 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f98101ce0 0x7f6f981040d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:30.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:30.816+0000 7f6f9dbd7640 1 -- 172.21.15.12:0/738938241 >> 172.21.15.12:0/738938241 conn(0x7f6f980fb6f0 msgr2=0x7f6f980fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:30.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:30.816+0000 7f6f9dbd7640 1 -- 172.21.15.12:0/738938241 shutdown_connections 2024-02-18T11:40:30.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:30.816+0000 7f6f9dbd7640 1 -- 172.21.15.12:0/738938241 wait complete. 2024-02-18T11:40:30.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:30.817+0000 7f6f9dbd7640 1 Processor -- start 2024-02-18T11:40:30.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:30.817+0000 7f6f9dbd7640 1 -- start start 2024-02-18T11:40:30.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:30.817+0000 7f6f9dbd7640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f98101ce0 0x7f6f981972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:30.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:30.817+0000 7f6f9dbd7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f98104610 0x7f6f98197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:30.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:30.817+0000 7f6f9dbd7640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6f98106f40 0x7f6f9819e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:30.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:30.817+0000 7f6f9dbd7640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9819edd0 con 0x7f6f98101ce0 2024-02-18T11:40:30.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:30.817+0000 7f6f9dbd7640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9819ef40 con 0x7f6f98106f40 2024-02-18T11:40:30.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:30.817+0000 7f6f9dbd7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9819f100 con 0x7f6f98104610 2024-02-18T11:40:30.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:30.817+0000 7f6f977fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f98101ce0 0x7f6f981972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:30.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:30.817+0000 7f6f977fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f98101ce0 0x7f6f981972e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49888/0 (socket says 172.21.15.12:49888) 2024-02-18T11:40:30.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:30.817+0000 7f6f977fe640 1 -- 172.21.15.12:0/2350553153 learned_addr learned my addr 172.21.15.12:0/2350553153 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:40:30.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:30.817+0000 7f6f96ffd640 1 --2- 172.21.15.12:0/2350553153 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f98104610 0x7f6f98197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:30.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:30.817+0000 7f6f97fff640 1 --2- 172.21.15.12:0/2350553153 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6f98106f40 0x7f6f9819e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:30.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:30.817+0000 7f6f977fe640 1 -- 172.21.15.12:0/2350553153 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f98104610 msgr2=0x7f6f98197820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:30.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:30.817+0000 7f6f977fe640 1 --2- 172.21.15.12:0/2350553153 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f98104610 0x7f6f98197820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:30.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:30.817+0000 7f6f977fe640 1 -- 172.21.15.12:0/2350553153 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6f98106f40 msgr2=0x7f6f9819e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:30.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:30.817+0000 7f6f977fe640 1 --2- 172.21.15.12:0/2350553153 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6f98106f40 0x7f6f9819e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:30.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:30.817+0000 7f6f977fe640 1 -- 172.21.15.12:0/2350553153 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f8c009560 con 0x7f6f98101ce0 2024-02-18T11:40:30.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:30.817+0000 7f6f96ffd640 1 --2- 172.21.15.12:0/2350553153 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f98104610 0x7f6f98197820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:40:30.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:30.817+0000 7f6f97fff640 1 --2- 172.21.15.12:0/2350553153 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6f98106f40 0x7f6f9819e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:40:30.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:30.818+0000 7f6f977fe640 1 --2- 172.21.15.12:0/2350553153 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f98101ce0 0x7f6f981972e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6f8800c970 tx=0x7f6f8800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:30.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:30.818+0000 7f6f94ff9640 1 -- 172.21.15.12:0/2350553153 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f88007bb0 con 0x7f6f98101ce0 2024-02-18T11:40:30.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:30.818+0000 7f6f94ff9640 1 -- 172.21.15.12:0/2350553153 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6f88007d10 con 0x7f6f98101ce0 2024-02-18T11:40:30.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:30.818+0000 7f6f9dbd7640 1 -- 172.21.15.12:0/2350553153 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f9819f3e0 con 0x7f6f98101ce0 2024-02-18T11:40:30.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:30.818+0000 7f6f94ff9640 1 -- 172.21.15.12:0/2350553153 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f8800f450 con 0x7f6f98101ce0 2024-02-18T11:40:30.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:30.818+0000 7f6f9dbd7640 1 -- 172.21.15.12:0/2350553153 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f9819f8a0 con 0x7f6f98101ce0 2024-02-18T11:40:30.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:30.819+0000 7f6f9dbd7640 1 -- 172.21.15.12:0/2350553153 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f980fd1e0 con 0x7f6f98101ce0 2024-02-18T11:40:30.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:30.820+0000 7f6f94ff9640 1 -- 172.21.15.12:0/2350553153 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) v1 ==== 137245+0+0 (secure 0 0 0) 0x7f6f88004f70 con 0x7f6f98101ce0 2024-02-18T11:40:30.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:30.820+0000 7f6f94ff9640 1 --2- 172.21.15.12:0/2350553153 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f6f800a2750 0x7f6f800a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:30.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:30.820+0000 7f6f96ffd640 1 --2- 172.21.15.12:0/2350553153 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f6f800a2750 0x7f6f800a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:30.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:30.820+0000 7f6f94ff9640 1 -- 172.21.15.12:0/2350553153 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6f880ce320 con 0x7f6f98101ce0 2024-02-18T11:40:30.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:30.821+0000 7f6f96ffd640 1 --2- 172.21.15.12:0/2350553153 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f6f800a2750 0x7f6f800a4c10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c036040 tx=0x7f6f8c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:30.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:30.825+0000 7f6f94ff9640 1 -- 172.21.15.12:0/2350553153 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6f880970c0 con 0x7f6f98101ce0 2024-02-18T11:40:31.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.010+0000 7f6f9dbd7640 1 -- 172.21.15.12:0/2350553153 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6f9802ce80 con 0x7f6f98101ce0 2024-02-18T11:40:31.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.011+0000 7f6f94ff9640 1 -- 172.21.15.12:0/2350553153 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+22662 (secure 0 0 0) 0x7f6f88085020 con 0x7f6f98101ce0 2024-02-18T11:40:31.006 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:40:31.006 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":89,"fsid":"0c8bb0de-ce51-11ee-95ba-87774f69a715","created":"2024-02-18T11:30:49.110059+0000","modified":"2024-02-18T11:39:52.478480+0000","last_up_change":"2024-02-18T11:37:57.512337+0000","last_in_change":"2024-02-18T11:37:36.567969+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-18T11:34:15.281138+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-18T11:38:44.802134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-18T11:38:46.081188+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"35f15ea6-deb8-48a1-a7dd-ab649ec13533","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":536522426},{"type":"v1","addr":"172.21.15.12:6803","nonce":536522426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":536522426},{"type":"v1","addr":"172.21.15.12:6805","nonce":536522426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":536522426},{"type":"v1","addr":"172.21.15.12:6809","nonce":536522426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":536522426},{"type":"v1","addr":"172.21.15.12:6807","nonce":536522426}]},"public_addr":"172.21.15.12:6803/536522426","cluster_addr":"172.21.15.12:6805/536522426","heartbeat_back_addr":"172.21.15.12:6809/536522426","heartbeat_front_addr":"172.21.15.12:6807/536522426","state":["exists","up"]},{"osd":1,"uuid":"2af937ef-d827-44e6-b805-70cffbfe1126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":4249111488},{"type":"v1","addr":"172.21.15.158:6801","nonce":4249111488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":4249111488},{"type":"v1","addr":"172.21.15.158:6803","nonce":4249111488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":4249111488},{"type":"v1","addr":"172.21.15.158:6807","nonce":4249111488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":4249111488},{"type":"v1","addr":"172.21.15.158:6805","nonce":4249111488}]},"public_addr":"172.21.15.158:6801/4249111488","cluster_addr":"172.21.15.158:6803/4249111488","heartbeat_back_addr":"172.21.15.158:6807/4249111488","heartbeat_front_addr":"172.21.15.158:6805/4249111488","state":["exists","up"]},{"osd":2,"uuid":"17b804dd-31df-45ae-a637-4a5f65bbaf26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":627976150},{"type":"v1","addr":"172.21.15.188:6801","nonce":627976150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":627976150},{"type":"v1","addr":"172.21.15.188:6803","nonce":627976150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":627976150},{"type":"v1","addr":"172.21.15.188:6807","nonce":627976150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":627976150},{"type":"v1","addr":"172.21.15.188:6805","nonce":627976150}]},"public_addr":"172.21.15.188:6801/627976150","cluster_addr":"172.21.15.188:6803/627976150","heartbeat_back_addr":"172.21.15.188:6807/627976150","heartbeat_front_addr":"172.21.15.188:6805/627976150","state":["exists","up"]},{"osd":3,"uuid":"3f9cc423-cdfa-4acf-aaa2-b58b32ccb9cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":3758512503},{"type":"v1","addr":"172.21.15.12:6811","nonce":3758512503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":3758512503},{"type":"v1","addr":"172.21.15.12:6813","nonce":3758512503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3758512503},{"type":"v1","addr":"172.21.15.12:6817","nonce":3758512503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":3758512503},{"type":"v1","addr":"172.21.15.12:6815","nonce":3758512503}]},"public_addr":"172.21.15.12:6811/3758512503","cluster_addr":"172.21.15.12:6813/3758512503","heartbeat_back_addr":"172.21.15.12:6817/3758512503","heartbeat_front_addr":"172.21.15.12:6815/3758512503","state":["exists","up"]},{"osd":4,"uuid":"a92e20ec-af1c-4ef4-a0cb-caba5fcc71aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":3892820997},{"type":"v1","addr":"172.21.15.158:6809","nonce":3892820997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":3892820997},{"type":"v1","addr":"172.21.15.158:6811","nonce":3892820997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":3892820997},{"type":"v1","addr":"172.21.15.158:6815","nonce":3892820997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":3892820997},{"type":"v1","addr":"172.21.15.158:6813","nonce":3892820997}]},"public_addr":"172.21.15.158:6809/3892820997","cluster_addr":"172.21.15.158:6811/3892820997","heartbeat_back_addr":"172.21.15.158:6815/3892820997","heartbeat_front_addr":"172.21.15.158:6813/3892820997","state":["exists","up"]},{"osd":5,"uuid":"d9c2e0c8-85f5-4ae2-bf3e-125c1be3a2db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3939495896},{"type":"v1","addr":"172.21.15.188:6809","nonce":3939495896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3939495896},{"type":"v1","addr":"172.21.15.188:6811","nonce":3939495896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3939495896},{"type":"v1","addr":"172.21.15.188:6815","nonce":3939495896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3939495896},{"type":"v1","addr":"172.21.15.188:6813","nonce":3939495896}]},"public_addr":"172.21.15.188:6809/3939495896","cluster_addr":"172.21.15.188:6811/3939495896","heartbeat_back_addr":"172.21.15.188:6815/3939495896","heartbeat_front_addr":"172.21.15.188:6813/3939495896","state":["exists","up"]},{"osd":6,"uuid":"4a523512-de6b-49f0-be81-00f2f00c1c10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":1206337767},{"type":"v1","addr":"172.21.15.12:6819","nonce":1206337767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":1206337767},{"type":"v1","addr":"172.21.15.12:6821","nonce":1206337767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":1206337767},{"type":"v1","addr":"172.21.15.12:6825","nonce":1206337767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":1206337767},{"type":"v1","addr":"172.21.15.12:6823","nonce":1206337767}]},"public_addr":"172.21.15.12:6819/1206337767","cluster_addr":"172.21.15.12:6821/1206337767","heartbeat_back_addr":"172.21.15.12:6825/1206337767","heartbeat_front_addr":"172.21.15.12:6823/1206337767","state":["exists","up"]},{"osd":7,"uuid":"363271a6-df93-44c9-b0e5-3d29768aa96c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":3283400032},{"type":"v1","addr":"172.21.15.158:6817","nonce":3283400032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":3283400032},{"type":"v1","addr":"172.21.15.158:6819","nonce":3283400032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":3283400032},{"type":"v1","addr":"172.21.15.158:6823","nonce":3283400032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":3283400032},{"type":"v1","addr":"172.21.15.158:6821","nonce":3283400032}]},"public_addr":"172.21.15.158:6817/3283400032","cluster_addr":"172.21.15.158:6819/3283400032","heartbeat_back_addr":"172.21.15.158:6823/3283400032","heartbeat_front_addr":"172.21.15.158:6821/3283400032","state":["exists","up"]},{"osd":8,"uuid":"9a089528-3e8e-45d3-9279-ddb54088b96a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2110170682},{"type":"v1","addr":"172.21.15.188:6817","nonce":2110170682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2110170682},{"type":"v1","addr":"172.21.15.188:6819","nonce":2110170682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":2110170682},{"type":"v1","addr":"172.21.15.188:6823","nonce":2110170682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2110170682},{"type":"v1","addr":"172.21.15.188:6821","nonce":2110170682}]},"public_addr":"172.21.15.188:6817/2110170682","cluster_addr":"172.21.15.188:6819/2110170682","heartbeat_back_addr":"172.21.15.188:6823/2110170682","heartbeat_front_addr":"172.21.15.188:6821/2110170682","state":["exists","up"]},{"osd":9,"uuid":"b12e2727-d649-4464-90e1-fafbc1754832","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1825928986},{"type":"v1","addr":"172.21.15.12:6827","nonce":1825928986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1825928986},{"type":"v1","addr":"172.21.15.12:6829","nonce":1825928986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":1825928986},{"type":"v1","addr":"172.21.15.12:6833","nonce":1825928986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1825928986},{"type":"v1","addr":"172.21.15.12:6831","nonce":1825928986}]},"public_addr":"172.21.15.12:6827/1825928986","cluster_addr":"172.21.15.12:6829/1825928986","heartbeat_back_addr":"172.21.15.12:6833/1825928986","heartbeat_front_addr":"172.21.15.12:6831/1825928986","state":["exists","up"]},{"osd":10,"uuid":"518b4cbf-9d90-4e9a-8ac9-b5cb9b3dee4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":3239052167},{"type":"v1","addr":"172.21.15.158:6825","nonce":3239052167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":3239052167},{"type":"v1","addr":"172.21.15.158:6827","nonce":3239052167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":3239052167},{"type":"v1","addr":"172.21.15.158:6831","nonce":3239052167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":3239052167},{"type":"v1","addr":"172.21.15.158:6829","nonce":3239052167}]},"public_addr":"172.21.15.158:6825/3239052167","cluster_addr":"172.21.15.158:6827/3239052167","heartbeat_back_addr":"172.21.15.158:6831/3239052167","heartbeat_front_addr":"172.21.15.158:6829/3239052167","state":["exists","up"]},{"osd":11,"uuid":"98aa3a0d-4b2f-46c8-9341-375d13794c3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1135773350},{"type":"v1","addr":"172.21.15.188:6825","nonce":1135773350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1135773350},{"type":"v1","addr":"172.21.15.188:6827","nonce":1135773350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1135773350},{"type":"v1","addr":"172.21.15.188:6831","nonce":1135773350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1135773350},{"type":"v1","addr":"172.21.15.188:6829","nonce":1135773350}]},"public_addr":"172.21.15.188:6825/1135773350","cluster_addr":"172.21.15.188:6827/1135773350","heartbeat_back_addr":"172.21.15.188:6831/1135773350","heartbeat_front_addr":"172.21.15.188:6829/1135773350","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:33:24.280879+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:33:47.859461+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:34:11.725283+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:34:35.707629+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:35:00.024197+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:35:24.499803+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:35:48.953870+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:36:13.642456+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:36:38.936457+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:37:04.021466+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:37:29.073178+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:37:54.863919+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.37","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:6801/4135274411":"2024-02-19T11:31:50.936928+0000","172.21.15.12:6800/4135274411":"2024-02-19T11:31:50.936928+0000","172.21.15.12:0/118428584":"2024-02-19T11:31:50.936928+0000","172.21.15.12:0/4276810915":"2024-02-19T11:31:50.936928+0000","172.21.15.12:0/3777545262":"2024-02-19T11:31:24.000830+0000","172.21.15.12:0/114707743":"2024-02-19T11:31:24.000830+0000","172.21.15.12:6800/3925204723":"2024-02-19T11:31:24.000830+0000","172.21.15.12:6801/3925204723":"2024-02-19T11:31:24.000830+0000","172.21.15.12:0/74881618":"2024-02-19T11:31:24.000830+0000","172.21.15.12:0/1614805375":"2024-02-19T11:31:50.936928+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-18T11:40:31.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.014+0000 7f6f9dbd7640 1 -- 172.21.15.12:0/2350553153 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f6f800a2750 msgr2=0x7f6f800a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:31.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.014+0000 7f6f9dbd7640 1 --2- 172.21.15.12:0/2350553153 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f6f800a2750 0x7f6f800a4c10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c036040 tx=0x7f6f8c000bc0 comp rx=0 tx=0).stop 2024-02-18T11:40:31.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.014+0000 7f6f9dbd7640 1 -- 172.21.15.12:0/2350553153 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f98101ce0 msgr2=0x7f6f981972e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:31.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.014+0000 7f6f9dbd7640 1 --2- 172.21.15.12:0/2350553153 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f98101ce0 0x7f6f981972e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6f8800c970 tx=0x7f6f8800ce30 comp rx=0 tx=0).stop 2024-02-18T11:40:31.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.014+0000 7f6f9dbd7640 1 -- 172.21.15.12:0/2350553153 shutdown_connections 2024-02-18T11:40:31.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.014+0000 7f6f9dbd7640 1 --2- 172.21.15.12:0/2350553153 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f6f800a2750 0x7f6f800a4c10 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:31.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.014+0000 7f6f9dbd7640 1 --2- 172.21.15.12:0/2350553153 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f98104610 0x7f6f98197820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:31.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.014+0000 7f6f9dbd7640 1 --2- 172.21.15.12:0/2350553153 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6f98106f40 0x7f6f9819e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:31.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.014+0000 7f6f9dbd7640 1 --2- 172.21.15.12:0/2350553153 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f98101ce0 0x7f6f981972e0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:31.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.014+0000 7f6f9dbd7640 1 -- 172.21.15.12:0/2350553153 >> 172.21.15.12:0/2350553153 conn(0x7f6f980fb6f0 msgr2=0x7f6f98102620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:31.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.014+0000 7f6f9dbd7640 1 -- 172.21.15.12:0/2350553153 shutdown_connections 2024-02-18T11:40:31.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.014+0000 7f6f9dbd7640 1 -- 172.21.15.12:0/2350553153 wait complete. 2024-02-18T11:40:31.019 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-18T11:34:15.281138+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-18T11:38:44.802134+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-18T11:38:46.081188+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-18T11:40:31.020 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-18T11:40:31.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.132+0000 7f2b7dba1640 1 -- 172.21.15.12:0/3576049960 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b781012c0 msgr2=0x7f2b78101740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:31.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.132+0000 7f2b7dba1640 1 --2- 172.21.15.12:0/3576049960 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b781012c0 0x7f2b78101740 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2b60009900 tx=0x7f2b6002bc30 comp rx=0 tx=0).stop 2024-02-18T11:40:31.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.132+0000 7f2b7dba1640 1 -- 172.21.15.12:0/3576049960 shutdown_connections 2024-02-18T11:40:31.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.132+0000 7f2b7dba1640 1 --2- 172.21.15.12:0/3576049960 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b78101c80 0x7f2b78108510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:31.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.132+0000 7f2b7dba1640 1 --2- 172.21.15.12:0/3576049960 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b781012c0 0x7f2b78101740 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:31.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.132+0000 7f2b7dba1640 1 --2- 172.21.15.12:0/3576049960 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2b781000d0 0x7f2b781004f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:31.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.132+0000 7f2b7dba1640 1 -- 172.21.15.12:0/3576049960 >> 172.21.15.12:0/3576049960 conn(0x7f2b780fb660 msgr2=0x7f2b780fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:31.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.132+0000 7f2b7dba1640 1 -- 172.21.15.12:0/3576049960 shutdown_connections 2024-02-18T11:40:31.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.132+0000 7f2b7dba1640 1 -- 172.21.15.12:0/3576049960 wait complete. 2024-02-18T11:40:31.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.132+0000 7f2b7dba1640 1 Processor -- start 2024-02-18T11:40:31.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.132+0000 7f2b7dba1640 1 -- start start 2024-02-18T11:40:31.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.133+0000 7f2b7dba1640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2b781000d0 0x7f2b78197260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:31.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.133+0000 7f2b7dba1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b781012c0 0x7f2b781977a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:31.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.133+0000 7f2b7dba1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b78101c80 0x7f2b7819e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:31.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.133+0000 7f2b7dba1640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7819ed50 con 0x7f2b78101c80 2024-02-18T11:40:31.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.133+0000 7f2b7dba1640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7819eec0 con 0x7f2b781000d0 2024-02-18T11:40:31.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.133+0000 7f2b7dba1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7819f080 con 0x7f2b781012c0 2024-02-18T11:40:31.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.133+0000 7f2b77fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b78101c80 0x7f2b7819e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:31.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.133+0000 7f2b77fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b78101c80 0x7f2b7819e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49910/0 (socket says 172.21.15.12:49910) 2024-02-18T11:40:31.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.133+0000 7f2b77fff640 1 -- 172.21.15.12:0/1932041949 learned_addr learned my addr 172.21.15.12:0/1932041949 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:40:31.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.133+0000 7f2b777fe640 1 --2- 172.21.15.12:0/1932041949 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2b781000d0 0x7f2b78197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:31.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.133+0000 7f2b6effd640 1 --2- 172.21.15.12:0/1932041949 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b781012c0 0x7f2b781977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:31.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.133+0000 7f2b77fff640 1 -- 172.21.15.12:0/1932041949 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b781012c0 msgr2=0x7f2b781977a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:31.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.133+0000 7f2b77fff640 1 --2- 172.21.15.12:0/1932041949 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b781012c0 0x7f2b781977a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:31.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.133+0000 7f2b77fff640 1 -- 172.21.15.12:0/1932041949 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2b781000d0 msgr2=0x7f2b78197260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:31.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.133+0000 7f2b77fff640 1 --2- 172.21.15.12:0/1932041949 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2b781000d0 0x7f2b78197260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:31.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.133+0000 7f2b77fff640 1 -- 172.21.15.12:0/1932041949 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b60009560 con 0x7f2b78101c80 2024-02-18T11:40:31.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.133+0000 7f2b77fff640 1 --2- 172.21.15.12:0/1932041949 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b78101c80 0x7f2b7819e810 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f2b7000eaa0 tx=0x7f2b7000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:31.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.133+0000 7f2b757fa640 1 -- 172.21.15.12:0/1932041949 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b7000cdb0 con 0x7f2b78101c80 2024-02-18T11:40:31.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.133+0000 7f2b7dba1640 1 -- 172.21.15.12:0/1932041949 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b7819f360 con 0x7f2b78101c80 2024-02-18T11:40:31.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.133+0000 7f2b757fa640 1 -- 172.21.15.12:0/1932041949 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2b70005020 con 0x7f2b78101c80 2024-02-18T11:40:31.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.133+0000 7f2b757fa640 1 -- 172.21.15.12:0/1932041949 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b700116d0 con 0x7f2b78101c80 2024-02-18T11:40:31.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.133+0000 7f2b7dba1640 1 -- 172.21.15.12:0/1932041949 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b7819f8c0 con 0x7f2b78101c80 2024-02-18T11:40:31.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.134+0000 7f2b7dba1640 1 -- 172.21.15.12:0/1932041949 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b78107490 con 0x7f2b78101c80 2024-02-18T11:40:31.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.135+0000 7f2b757fa640 1 -- 172.21.15.12:0/1932041949 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) v1 ==== 137245+0+0 (secure 0 0 0) 0x7f2b70005490 con 0x7f2b78101c80 2024-02-18T11:40:31.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.136+0000 7f2b757fa640 1 --2- 172.21.15.12:0/1932041949 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f2b640a2750 0x7f2b640a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:31.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.136+0000 7f2b777fe640 1 --2- 172.21.15.12:0/1932041949 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f2b640a2750 0x7f2b640a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:31.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.136+0000 7f2b757fa640 1 -- 172.21.15.12:0/1932041949 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2b700ce710 con 0x7f2b78101c80 2024-02-18T11:40:31.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.137+0000 7f2b777fe640 1 --2- 172.21.15.12:0/1932041949 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f2b640a2750 0x7f2b640a4c10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2b78101120 tx=0x7f2b6800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:31.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.141+0000 7f2b757fa640 1 -- 172.21.15.12:0/1932041949 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2b700974b0 con 0x7f2b78101c80 2024-02-18T11:40:31.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.323+0000 7f2b7dba1640 1 -- 172.21.15.12:0/1932041949 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2b7810cbe0 con 0x7f2b78101c80 2024-02-18T11:40:31.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.323+0000 7f2b757fa640 1 -- 172.21.15.12:0/1932041949 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2b7810cbe0 con 0x7f2b78101c80 2024-02-18T11:40:31.317 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-02-18T11:40:31.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.325+0000 7f2b7dba1640 1 -- 172.21.15.12:0/1932041949 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f2b640a2750 msgr2=0x7f2b640a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:31.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.325+0000 7f2b7dba1640 1 --2- 172.21.15.12:0/1932041949 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f2b640a2750 0x7f2b640a4c10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2b78101120 tx=0x7f2b6800b390 comp rx=0 tx=0).stop 2024-02-18T11:40:31.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.325+0000 7f2b7dba1640 1 -- 172.21.15.12:0/1932041949 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b78101c80 msgr2=0x7f2b7819e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:31.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.325+0000 7f2b7dba1640 1 --2- 172.21.15.12:0/1932041949 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b78101c80 0x7f2b7819e810 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f2b7000eaa0 tx=0x7f2b7000ef60 comp rx=0 tx=0).stop 2024-02-18T11:40:31.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.325+0000 7f2b7dba1640 1 -- 172.21.15.12:0/1932041949 shutdown_connections 2024-02-18T11:40:31.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.325+0000 7f2b7dba1640 1 --2- 172.21.15.12:0/1932041949 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f2b640a2750 0x7f2b640a4c10 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:31.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.325+0000 7f2b7dba1640 1 --2- 172.21.15.12:0/1932041949 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b781012c0 0x7f2b781977a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:31.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.325+0000 7f2b7dba1640 1 --2- 172.21.15.12:0/1932041949 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b78101c80 0x7f2b7819e810 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:31.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.325+0000 7f2b7dba1640 1 --2- 172.21.15.12:0/1932041949 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2b781000d0 0x7f2b78197260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:31.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.325+0000 7f2b7dba1640 1 -- 172.21.15.12:0/1932041949 >> 172.21.15.12:0/1932041949 conn(0x7f2b780fb660 msgr2=0x7f2b78104660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:31.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.325+0000 7f2b7dba1640 1 -- 172.21.15.12:0/1932041949 shutdown_connections 2024-02-18T11:40:31.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.325+0000 7f2b7dba1640 1 -- 172.21.15.12:0/1932041949 wait complete. 2024-02-18T11:40:31.329 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:40:31 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/2350553153' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-18T11:40:31.331 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-18T11:40:31.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.443+0000 7efe872ae640 1 -- 172.21.15.12:0/1232862835 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efe80101c90 msgr2=0x7efe80108520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:31.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.443+0000 7efe872ae640 1 --2- 172.21.15.12:0/1232862835 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efe80101c90 0x7efe80108520 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7efe78009ca0 tx=0x7efe7802c0f0 comp rx=0 tx=0).stop 2024-02-18T11:40:31.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.443+0000 7efe872ae640 1 -- 172.21.15.12:0/1232862835 shutdown_connections 2024-02-18T11:40:31.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.443+0000 7efe872ae640 1 --2- 172.21.15.12:0/1232862835 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe801012d0 0x7efe80101750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:31.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.443+0000 7efe872ae640 1 --2- 172.21.15.12:0/1232862835 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efe80101c90 0x7efe80108520 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:31.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.443+0000 7efe872ae640 1 --2- 172.21.15.12:0/1232862835 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efe801000e0 0x7efe80100500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:31.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.443+0000 7efe872ae640 1 -- 172.21.15.12:0/1232862835 >> 172.21.15.12:0/1232862835 conn(0x7efe800fb670 msgr2=0x7efe800fda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:31.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.443+0000 7efe872ae640 1 -- 172.21.15.12:0/1232862835 shutdown_connections 2024-02-18T11:40:31.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.443+0000 7efe872ae640 1 -- 172.21.15.12:0/1232862835 wait complete. 2024-02-18T11:40:31.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.443+0000 7efe872ae640 1 Processor -- start 2024-02-18T11:40:31.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.443+0000 7efe872ae640 1 -- start start 2024-02-18T11:40:31.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.443+0000 7efe872ae640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe801000e0 0x7efe80197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:31.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.443+0000 7efe872ae640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efe801012d0 0x7efe80197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:31.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.443+0000 7efe872ae640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efe80101c90 0x7efe8019e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:31.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.443+0000 7efe872ae640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe8019ecf0 con 0x7efe801012d0 2024-02-18T11:40:31.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.443+0000 7efe872ae640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe8019ee60 con 0x7efe80101c90 2024-02-18T11:40:31.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.443+0000 7efe872ae640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe8019f020 con 0x7efe801000e0 2024-02-18T11:40:31.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.443+0000 7efe84849640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efe801012d0 0x7efe80197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:31.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.444+0000 7efe84849640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efe801012d0 0x7efe80197740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49932/0 (socket says 172.21.15.12:49932) 2024-02-18T11:40:31.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.444+0000 7efe84849640 1 -- 172.21.15.12:0/2375984210 learned_addr learned my addr 172.21.15.12:0/2375984210 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:40:31.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.444+0000 7efe8584b640 1 --2- 172.21.15.12:0/2375984210 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efe80101c90 0x7efe8019e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:31.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.444+0000 7efe8504a640 1 --2- 172.21.15.12:0/2375984210 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe801000e0 0x7efe80197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:31.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.444+0000 7efe84849640 1 -- 172.21.15.12:0/2375984210 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe801000e0 msgr2=0x7efe80197200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:31.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.444+0000 7efe84849640 1 --2- 172.21.15.12:0/2375984210 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe801000e0 0x7efe80197200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:31.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.444+0000 7efe84849640 1 -- 172.21.15.12:0/2375984210 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efe80101c90 msgr2=0x7efe8019e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:31.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.444+0000 7efe84849640 1 --2- 172.21.15.12:0/2375984210 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efe80101c90 0x7efe8019e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:31.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.444+0000 7efe84849640 1 -- 172.21.15.12:0/2375984210 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe78009900 con 0x7efe801012d0 2024-02-18T11:40:31.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.444+0000 7efe8584b640 1 --2- 172.21.15.12:0/2375984210 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efe80101c90 0x7efe8019e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:40:31.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.444+0000 7efe8504a640 1 --2- 172.21.15.12:0/2375984210 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe801000e0 0x7efe80197200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:40:31.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.444+0000 7efe84849640 1 --2- 172.21.15.12:0/2375984210 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efe801012d0 0x7efe80197740 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7efe7000c970 tx=0x7efe7000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:31.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.444+0000 7efe767fc640 1 -- 172.21.15.12:0/2375984210 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe70007bb0 con 0x7efe801012d0 2024-02-18T11:40:31.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.444+0000 7efe872ae640 1 -- 172.21.15.12:0/2375984210 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe8019f300 con 0x7efe801012d0 2024-02-18T11:40:31.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.444+0000 7efe767fc640 1 -- 172.21.15.12:0/2375984210 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efe70007d10 con 0x7efe801012d0 2024-02-18T11:40:31.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.444+0000 7efe767fc640 1 -- 172.21.15.12:0/2375984210 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe7000f450 con 0x7efe801012d0 2024-02-18T11:40:31.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.444+0000 7efe872ae640 1 -- 172.21.15.12:0/2375984210 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe8019f810 con 0x7efe801012d0 2024-02-18T11:40:31.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.445+0000 7efe872ae640 1 -- 172.21.15.12:0/2375984210 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe80107400 con 0x7efe801012d0 2024-02-18T11:40:31.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.446+0000 7efe767fc640 1 -- 172.21.15.12:0/2375984210 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) v1 ==== 137245+0+0 (secure 0 0 0) 0x7efe70004f70 con 0x7efe801012d0 2024-02-18T11:40:31.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.447+0000 7efe767fc640 1 --2- 172.21.15.12:0/2375984210 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7efe6c0a2750 0x7efe6c0a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:31.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.447+0000 7efe8504a640 1 --2- 172.21.15.12:0/2375984210 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7efe6c0a2750 0x7efe6c0a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:31.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.447+0000 7efe767fc640 1 -- 172.21.15.12:0/2375984210 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8211+0+0 (secure 0 0 0) 0x7efe700ce380 con 0x7efe801012d0 2024-02-18T11:40:31.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.447+0000 7efe8504a640 1 --2- 172.21.15.12:0/2375984210 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7efe6c0a2750 0x7efe6c0a4c10 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7efe80101130 tx=0x7efe7c005e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:31.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.451+0000 7efe767fc640 1 -- 172.21.15.12:0/2375984210 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7efe700970c0 con 0x7efe801012d0 2024-02-18T11:40:31.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:40:31 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/2350553153' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-18T11:40:31.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:40:31 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/2350553153' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-18T11:40:31.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.631+0000 7efe872ae640 1 -- 172.21.15.12:0/2375984210 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7efe8010cbf0 con 0x7efe801012d0 2024-02-18T11:40:31.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.632+0000 7efe767fc640 1 -- 172.21.15.12:0/2375984210 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7efe8010cbf0 con 0x7efe801012d0 2024-02-18T11:40:31.626 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-02-18T11:40:31.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.634+0000 7efe872ae640 1 -- 172.21.15.12:0/2375984210 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7efe6c0a2750 msgr2=0x7efe6c0a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:31.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.634+0000 7efe872ae640 1 --2- 172.21.15.12:0/2375984210 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7efe6c0a2750 0x7efe6c0a4c10 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7efe80101130 tx=0x7efe7c005e70 comp rx=0 tx=0).stop 2024-02-18T11:40:31.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.634+0000 7efe872ae640 1 -- 172.21.15.12:0/2375984210 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efe801012d0 msgr2=0x7efe80197740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:31.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.634+0000 7efe872ae640 1 --2- 172.21.15.12:0/2375984210 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efe801012d0 0x7efe80197740 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7efe7000c970 tx=0x7efe7000ce30 comp rx=0 tx=0).stop 2024-02-18T11:40:31.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.634+0000 7efe872ae640 1 -- 172.21.15.12:0/2375984210 shutdown_connections 2024-02-18T11:40:31.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.634+0000 7efe872ae640 1 --2- 172.21.15.12:0/2375984210 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7efe6c0a2750 0x7efe6c0a4c10 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:31.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.634+0000 7efe872ae640 1 --2- 172.21.15.12:0/2375984210 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efe80101c90 0x7efe8019e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:31.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.634+0000 7efe872ae640 1 --2- 172.21.15.12:0/2375984210 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efe801012d0 0x7efe80197740 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:31.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.634+0000 7efe872ae640 1 --2- 172.21.15.12:0/2375984210 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe801000e0 0x7efe80197200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:31.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.634+0000 7efe872ae640 1 -- 172.21.15.12:0/2375984210 >> 172.21.15.12:0/2375984210 conn(0x7efe800fb670 msgr2=0x7efe801045d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:31.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.634+0000 7efe872ae640 1 -- 172.21.15.12:0/2375984210 shutdown_connections 2024-02-18T11:40:31.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.634+0000 7efe872ae640 1 -- 172.21.15.12:0/2375984210 wait complete. 2024-02-18T11:40:31.639 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-18T11:40:31.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.751+0000 7f56e0902640 1 -- 172.21.15.12:0/1126063319 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56dc1011d0 msgr2=0x7f56dc1015f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:31.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.751+0000 7f56e0902640 1 --2- 172.21.15.12:0/1126063319 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56dc1011d0 0x7f56dc1015f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f56cc009900 tx=0x7f56cc02bc30 comp rx=0 tx=0).stop 2024-02-18T11:40:31.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.751+0000 7f56e0902640 1 -- 172.21.15.12:0/1126063319 shutdown_connections 2024-02-18T11:40:31.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.751+0000 7f56e0902640 1 --2- 172.21.15.12:0/1126063319 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56dc1023c0 0x7f56dc102840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:31.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.751+0000 7f56e0902640 1 --2- 172.21.15.12:0/1126063319 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f56dc102d80 0x7f56dc109610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:31.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.751+0000 7f56e0902640 1 --2- 172.21.15.12:0/1126063319 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56dc1011d0 0x7f56dc1015f0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:31.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.751+0000 7f56e0902640 1 -- 172.21.15.12:0/1126063319 >> 172.21.15.12:0/1126063319 conn(0x7f56dc0fc760 msgr2=0x7f56dc0feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:31.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.751+0000 7f56e0902640 1 -- 172.21.15.12:0/1126063319 shutdown_connections 2024-02-18T11:40:31.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.751+0000 7f56e0902640 1 -- 172.21.15.12:0/1126063319 wait complete. 2024-02-18T11:40:31.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.752+0000 7f56e0902640 1 Processor -- start 2024-02-18T11:40:31.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.752+0000 7f56e0902640 1 -- start start 2024-02-18T11:40:31.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.752+0000 7f56e0902640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56dc1011d0 0x7f56dc197460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:31.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.752+0000 7f56e0902640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56dc1023c0 0x7f56dc1979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:31.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.752+0000 7f56e0902640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f56dc102d80 0x7f56dc19ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:31.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.752+0000 7f56e0902640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56dc19ef50 con 0x7f56dc1011d0 2024-02-18T11:40:31.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.752+0000 7f56e0902640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56dc19f0c0 con 0x7f56dc102d80 2024-02-18T11:40:31.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.752+0000 7f56e0902640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56dc19f280 con 0x7f56dc1023c0 2024-02-18T11:40:31.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.752+0000 7f56da59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56dc1011d0 0x7f56dc197460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:31.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.752+0000 7f56da59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56dc1011d0 0x7f56dc197460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48550/0 (socket says 172.21.15.12:48550) 2024-02-18T11:40:31.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.752+0000 7f56da59c640 1 -- 172.21.15.12:0/2764103234 learned_addr learned my addr 172.21.15.12:0/2764103234 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:40:31.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.752+0000 7f56d9d9b640 1 --2- 172.21.15.12:0/2764103234 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56dc1023c0 0x7f56dc1979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:31.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.752+0000 7f56dad9d640 1 --2- 172.21.15.12:0/2764103234 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f56dc102d80 0x7f56dc19ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:31.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.752+0000 7f56da59c640 1 -- 172.21.15.12:0/2764103234 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56dc1023c0 msgr2=0x7f56dc1979a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:31.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.752+0000 7f56da59c640 1 --2- 172.21.15.12:0/2764103234 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56dc1023c0 0x7f56dc1979a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:31.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.752+0000 7f56da59c640 1 -- 172.21.15.12:0/2764103234 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f56dc102d80 msgr2=0x7f56dc19ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:31.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.752+0000 7f56da59c640 1 --2- 172.21.15.12:0/2764103234 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f56dc102d80 0x7f56dc19ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:31.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.752+0000 7f56da59c640 1 -- 172.21.15.12:0/2764103234 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56cc009560 con 0x7f56dc1011d0 2024-02-18T11:40:31.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.752+0000 7f56d9d9b640 1 --2- 172.21.15.12:0/2764103234 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56dc1023c0 0x7f56dc1979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:40:31.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.752+0000 7f56da59c640 1 --2- 172.21.15.12:0/2764103234 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56dc1011d0 0x7f56dc197460 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f56cc036040 tx=0x7f56cc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:31.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.752+0000 7f56cb7fe640 1 -- 172.21.15.12:0/2764103234 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56cc0044e0 con 0x7f56dc1011d0 2024-02-18T11:40:31.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.752+0000 7f56cb7fe640 1 -- 172.21.15.12:0/2764103234 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f56cc0054e0 con 0x7f56dc1011d0 2024-02-18T11:40:31.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.753+0000 7f56e0902640 1 -- 172.21.15.12:0/2764103234 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56dc19f500 con 0x7f56dc1011d0 2024-02-18T11:40:31.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.753+0000 7f56cb7fe640 1 -- 172.21.15.12:0/2764103234 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56cc03e450 con 0x7f56dc1011d0 2024-02-18T11:40:31.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.753+0000 7f56e0902640 1 -- 172.21.15.12:0/2764103234 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56dc19fa90 con 0x7f56dc1011d0 2024-02-18T11:40:31.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.753+0000 7f56e0902640 1 -- 172.21.15.12:0/2764103234 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56dc1085e0 con 0x7f56dc1011d0 2024-02-18T11:40:31.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.754+0000 7f56cb7fe640 1 -- 172.21.15.12:0/2764103234 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) v1 ==== 137245+0+0 (secure 0 0 0) 0x7f56cc003710 con 0x7f56dc1011d0 2024-02-18T11:40:31.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.755+0000 7f56cb7fe640 1 --2- 172.21.15.12:0/2764103234 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f56c40a2750 0x7f56c40a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:31.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.755+0000 7f56d9d9b640 1 --2- 172.21.15.12:0/2764103234 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f56c40a2750 0x7f56c40a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:31.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.755+0000 7f56cb7fe640 1 -- 172.21.15.12:0/2764103234 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f56cc0f0a50 con 0x7f56dc1011d0 2024-02-18T11:40:31.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.755+0000 7f56d9d9b640 1 --2- 172.21.15.12:0/2764103234 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f56c40a2750 0x7f56c40a4c10 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f56d0006fd0 tx=0x7f56d0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:31.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.759+0000 7f56cb7fe640 1 -- 172.21.15.12:0/2764103234 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f56cc0b97f0 con 0x7f56dc1011d0 2024-02-18T11:40:31.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.944+0000 7f56e0902640 1 -- 172.21.15.12:0/2764103234 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f56dc02ce80 con 0x7f56dc1011d0 2024-02-18T11:40:31.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.944+0000 7f56cb7fe640 1 -- 172.21.15.12:0/2764103234 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+11 (secure 0 0 0) 0x7f56cc0bceb0 con 0x7f56dc1011d0 2024-02-18T11:40:31.938 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2024-02-18T11:40:31.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.946+0000 7f56e0902640 1 -- 172.21.15.12:0/2764103234 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f56c40a2750 msgr2=0x7f56c40a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:31.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.946+0000 7f56e0902640 1 --2- 172.21.15.12:0/2764103234 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f56c40a2750 0x7f56c40a4c10 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f56d0006fd0 tx=0x7f56d0008040 comp rx=0 tx=0).stop 2024-02-18T11:40:31.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.946+0000 7f56e0902640 1 -- 172.21.15.12:0/2764103234 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56dc1011d0 msgr2=0x7f56dc197460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:31.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.946+0000 7f56e0902640 1 --2- 172.21.15.12:0/2764103234 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56dc1011d0 0x7f56dc197460 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f56cc036040 tx=0x7f56cc0042f0 comp rx=0 tx=0).stop 2024-02-18T11:40:31.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.946+0000 7f56e0902640 1 -- 172.21.15.12:0/2764103234 shutdown_connections 2024-02-18T11:40:31.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.946+0000 7f56e0902640 1 --2- 172.21.15.12:0/2764103234 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f56c40a2750 0x7f56c40a4c10 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:31.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.946+0000 7f56e0902640 1 --2- 172.21.15.12:0/2764103234 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56dc1023c0 0x7f56dc1979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:31.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.946+0000 7f56e0902640 1 --2- 172.21.15.12:0/2764103234 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f56dc102d80 0x7f56dc19ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:31.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.946+0000 7f56e0902640 1 --2- 172.21.15.12:0/2764103234 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56dc1011d0 0x7f56dc197460 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:31.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.946+0000 7f56e0902640 1 -- 172.21.15.12:0/2764103234 >> 172.21.15.12:0/2764103234 conn(0x7f56dc0fc760 msgr2=0x7f56dc1057b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:31.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.946+0000 7f56e0902640 1 -- 172.21.15.12:0/2764103234 shutdown_connections 2024-02-18T11:40:31.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:31.946+0000 7f56e0902640 1 -- 172.21.15.12:0/2764103234 wait complete. 2024-02-18T11:40:31.951 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-18T11:40:32.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.061+0000 7ff959977640 1 -- 172.21.15.12:0/2313857621 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9541012d0 msgr2=0x7ff954101750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:32.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.061+0000 7ff959977640 1 --2- 172.21.15.12:0/2313857621 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9541012d0 0x7ff954101750 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff948009900 tx=0x7ff94802bc30 comp rx=0 tx=0).stop 2024-02-18T11:40:32.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.061+0000 7ff959977640 1 -- 172.21.15.12:0/2313857621 shutdown_connections 2024-02-18T11:40:32.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.061+0000 7ff959977640 1 --2- 172.21.15.12:0/2313857621 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff954101c90 0x7ff954108520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:32.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.061+0000 7ff959977640 1 --2- 172.21.15.12:0/2313857621 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9541012d0 0x7ff954101750 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:32.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.061+0000 7ff959977640 1 --2- 172.21.15.12:0/2313857621 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff9541000e0 0x7ff954100500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:32.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.061+0000 7ff959977640 1 -- 172.21.15.12:0/2313857621 >> 172.21.15.12:0/2313857621 conn(0x7ff9540fb670 msgr2=0x7ff9540fda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:32.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.061+0000 7ff959977640 1 -- 172.21.15.12:0/2313857621 shutdown_connections 2024-02-18T11:40:32.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.061+0000 7ff959977640 1 -- 172.21.15.12:0/2313857621 wait complete. 2024-02-18T11:40:32.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.062+0000 7ff959977640 1 Processor -- start 2024-02-18T11:40:32.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.062+0000 7ff959977640 1 -- start start 2024-02-18T11:40:32.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.062+0000 7ff959977640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9541000e0 0x7ff954196180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:32.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.062+0000 7ff959977640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9541012d0 0x7ff9541966c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:32.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.062+0000 7ff959977640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff954101c90 0x7ff95419d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:32.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.062+0000 7ff959977640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff95419dc70 con 0x7ff9541000e0 2024-02-18T11:40:32.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.062+0000 7ff959977640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff95419dde0 con 0x7ff954101c90 2024-02-18T11:40:32.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.062+0000 7ff959977640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff95419dfa0 con 0x7ff9541012d0 2024-02-18T11:40:32.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.062+0000 7ff952ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9541000e0 0x7ff954196180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:32.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.062+0000 7ff952ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9541000e0 0x7ff954196180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48566/0 (socket says 172.21.15.12:48566) 2024-02-18T11:40:32.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.062+0000 7ff952ffd640 1 -- 172.21.15.12:0/662843696 learned_addr learned my addr 172.21.15.12:0/662843696 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:40:32.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.062+0000 7ff9527fc640 1 --2- 172.21.15.12:0/662843696 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9541012d0 0x7ff9541966c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:32.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.062+0000 7ff9537fe640 1 --2- 172.21.15.12:0/662843696 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff954101c90 0x7ff95419d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:32.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.063+0000 7ff952ffd640 1 -- 172.21.15.12:0/662843696 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9541012d0 msgr2=0x7ff9541966c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:32.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.063+0000 7ff952ffd640 1 --2- 172.21.15.12:0/662843696 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9541012d0 0x7ff9541966c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:32.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.063+0000 7ff952ffd640 1 -- 172.21.15.12:0/662843696 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff954101c90 msgr2=0x7ff95419d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:32.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.063+0000 7ff952ffd640 1 --2- 172.21.15.12:0/662843696 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff954101c90 0x7ff95419d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:32.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.063+0000 7ff952ffd640 1 -- 172.21.15.12:0/662843696 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff948009560 con 0x7ff9541000e0 2024-02-18T11:40:32.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.063+0000 7ff952ffd640 1 --2- 172.21.15.12:0/662843696 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9541000e0 0x7ff954196180 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff94400d770 tx=0x7ff94400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:32.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.063+0000 7ff958975640 1 -- 172.21.15.12:0/662843696 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff944009c30 con 0x7ff9541000e0 2024-02-18T11:40:32.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.063+0000 7ff959977640 1 -- 172.21.15.12:0/662843696 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff95419e280 con 0x7ff9541000e0 2024-02-18T11:40:32.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.063+0000 7ff958975640 1 -- 172.21.15.12:0/662843696 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff94400de80 con 0x7ff9541000e0 2024-02-18T11:40:32.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.063+0000 7ff958975640 1 -- 172.21.15.12:0/662843696 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff944010460 con 0x7ff9541000e0 2024-02-18T11:40:32.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.063+0000 7ff959977640 1 -- 172.21.15.12:0/662843696 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff95419e7b0 con 0x7ff9541000e0 2024-02-18T11:40:32.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.064+0000 7ff959977640 1 -- 172.21.15.12:0/662843696 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff954107480 con 0x7ff9541000e0 2024-02-18T11:40:32.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.065+0000 7ff958975640 1 -- 172.21.15.12:0/662843696 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) v1 ==== 137245+0+0 (secure 0 0 0) 0x7ff944003f60 con 0x7ff9541000e0 2024-02-18T11:40:32.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.066+0000 7ff958975640 1 --2- 172.21.15.12:0/662843696 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7ff93c0a2750 0x7ff93c0a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:32.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.066+0000 7ff9527fc640 1 --2- 172.21.15.12:0/662843696 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7ff93c0a2750 0x7ff93c0a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:32.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.066+0000 7ff958975640 1 -- 172.21.15.12:0/662843696 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff9440ce190 con 0x7ff9541000e0 2024-02-18T11:40:32.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.066+0000 7ff9527fc640 1 --2- 172.21.15.12:0/662843696 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7ff93c0a2750 0x7ff93c0a4c10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff948009230 tx=0x7ff948005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:32.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.071+0000 7ff958975640 1 -- 172.21.15.12:0/662843696 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff944096fe0 con 0x7ff9541000e0 2024-02-18T11:40:32.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.292+0000 7ff959977640 1 -- 172.21.15.12:0/662843696 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff95402ce80 con 0x7ff9541000e0 2024-02-18T11:40:32.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.292+0000 7ff958975640 1 -- 172.21.15.12:0/662843696 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11349 (secure 0 0 0) 0x7ff94409a6a0 con 0x7ff9541000e0 2024-02-18T11:40:32.286 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:40:32.287 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1788335302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1788335302},{"type":"v1","addr":"172.21.15.12:6835","nonce":1788335302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/2144692455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":2144692455},{"type":"v1","addr":"172.21.15.12:6837","nonce":2144692455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3569056036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3569056036},{"type":"v1","addr":"172.21.15.12:6839","nonce":3569056036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14622,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/1973234486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":1973234486},{"type":"v1","addr":"172.21.15.12:6841","nonce":1973234486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1207202465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1207202465},{"type":"v1","addr":"172.21.15.188:6833","nonce":1207202465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6833/2636348323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6832","nonce":2636348323},{"type":"v1","addr":"172.21.15.158:6833","nonce":2636348323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-18T11:38:46.888173+0000","modified":"2024-02-18T11:38:51.531552+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24524,"mds_1":24508,"mds_2":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.158:6835/3673253642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":3673253642},{"type":"v1","addr":"172.21.15.158:6835","nonce":3673253642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/806709644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":806709644},{"type":"v1","addr":"172.21.15.188:6835","nonce":806709644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.158:6837/813251825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6836","nonce":813251825},{"type":"v1","addr":"172.21.15.158:6837","nonce":813251825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.188:6837/1036630143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1036630143},{"type":"v1","addr":"172.21.15.188:6837","nonce":1036630143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.188:6839/1342671190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":1342671190},{"type":"v1","addr":"172.21.15.188:6839","nonce":1342671190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.158:6839/1127200849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6838","nonce":1127200849},{"type":"v1","addr":"172.21.15.158:6839","nonce":1127200849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-18T11:40:32.287 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 20 2024-02-18T11:40:32.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.294+0000 7ff959977640 1 -- 172.21.15.12:0/662843696 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7ff93c0a2750 msgr2=0x7ff93c0a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:32.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.294+0000 7ff959977640 1 --2- 172.21.15.12:0/662843696 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7ff93c0a2750 0x7ff93c0a4c10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff948009230 tx=0x7ff948005fd0 comp rx=0 tx=0).stop 2024-02-18T11:40:32.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.294+0000 7ff959977640 1 -- 172.21.15.12:0/662843696 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9541000e0 msgr2=0x7ff954196180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:32.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.294+0000 7ff959977640 1 --2- 172.21.15.12:0/662843696 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9541000e0 0x7ff954196180 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff94400d770 tx=0x7ff94400dc30 comp rx=0 tx=0).stop 2024-02-18T11:40:32.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.294+0000 7ff959977640 1 -- 172.21.15.12:0/662843696 shutdown_connections 2024-02-18T11:40:32.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.294+0000 7ff959977640 1 --2- 172.21.15.12:0/662843696 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7ff93c0a2750 0x7ff93c0a4c10 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:32.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.294+0000 7ff959977640 1 --2- 172.21.15.12:0/662843696 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9541012d0 0x7ff9541966c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:32.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.294+0000 7ff959977640 1 --2- 172.21.15.12:0/662843696 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff954101c90 0x7ff95419d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:32.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.294+0000 7ff959977640 1 --2- 172.21.15.12:0/662843696 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9541000e0 0x7ff954196180 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:32.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.294+0000 7ff959977640 1 -- 172.21.15.12:0/662843696 >> 172.21.15.12:0/662843696 conn(0x7ff9540fb670 msgr2=0x7ff954104650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:32.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.295+0000 7ff959977640 1 -- 172.21.15.12:0/662843696 shutdown_connections 2024-02-18T11:40:32.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.295+0000 7ff959977640 1 -- 172.21.15.12:0/662843696 wait complete. 2024-02-18T11:40:32.299 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:40:32 smithi012 ceph-mon[32460]: pgmap v339: 97 pgs: 97 active+clean; 640 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-02-18T11:40:32.299 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:40:32 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/1932041949' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-18T11:40:32.299 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:40:32 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/2375984210' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-18T11:40:32.299 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:40:32 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/2764103234' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-18T11:40:32.300 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-18T11:40:32.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.408+0000 7f0ba6a2e640 1 -- 172.21.15.12:0/3173165767 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ba0102d90 msgr2=0x7f0ba0109620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:32.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.408+0000 7f0ba6a2e640 1 --2- 172.21.15.12:0/3173165767 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ba0102d90 0x7f0ba0109620 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0b98009ca0 tx=0x7f0b9802c0f0 comp rx=0 tx=0).stop 2024-02-18T11:40:32.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.408+0000 7f0ba6a2e640 1 -- 172.21.15.12:0/3173165767 shutdown_connections 2024-02-18T11:40:32.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.408+0000 7f0ba6a2e640 1 --2- 172.21.15.12:0/3173165767 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ba01023d0 0x7f0ba0102850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:32.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.408+0000 7f0ba6a2e640 1 --2- 172.21.15.12:0/3173165767 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ba0102d90 0x7f0ba0109620 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:32.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.408+0000 7f0ba6a2e640 1 --2- 172.21.15.12:0/3173165767 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0ba01011e0 0x7f0ba0101600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:32.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.408+0000 7f0ba6a2e640 1 -- 172.21.15.12:0/3173165767 >> 172.21.15.12:0/3173165767 conn(0x7f0ba00fc770 msgr2=0x7f0ba00feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:32.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.408+0000 7f0ba6a2e640 1 -- 172.21.15.12:0/3173165767 shutdown_connections 2024-02-18T11:40:32.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.408+0000 7f0ba6a2e640 1 -- 172.21.15.12:0/3173165767 wait complete. 2024-02-18T11:40:32.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.409+0000 7f0ba6a2e640 1 Processor -- start 2024-02-18T11:40:32.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.409+0000 7f0ba6a2e640 1 -- start start 2024-02-18T11:40:32.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.409+0000 7f0ba6a2e640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0ba01011e0 0x7f0ba01972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:32.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.409+0000 7f0ba6a2e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ba01023d0 0x7f0ba0197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:32.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.409+0000 7f0ba6a2e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ba0102d90 0x7f0ba019e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:32.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.409+0000 7f0ba6a2e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba019edd0 con 0x7f0ba01023d0 2024-02-18T11:40:32.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.409+0000 7f0ba6a2e640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba019ef40 con 0x7f0ba01011e0 2024-02-18T11:40:32.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.409+0000 7f0ba6a2e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba019f100 con 0x7f0ba0102d90 2024-02-18T11:40:32.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.409+0000 7f0b9f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ba01023d0 0x7f0ba0197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:32.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.409+0000 7f0b9f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ba01023d0 0x7f0ba0197820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48578/0 (socket says 172.21.15.12:48578) 2024-02-18T11:40:32.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.409+0000 7f0b9f7fe640 1 -- 172.21.15.12:0/994386901 learned_addr learned my addr 172.21.15.12:0/994386901 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:40:32.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.409+0000 7f0ba4fcb640 1 --2- 172.21.15.12:0/994386901 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ba0102d90 0x7f0ba019e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:32.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.410+0000 7f0b9ffff640 1 --2- 172.21.15.12:0/994386901 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0ba01011e0 0x7f0ba01972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:32.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.410+0000 7f0b9f7fe640 1 -- 172.21.15.12:0/994386901 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ba0102d90 msgr2=0x7f0ba019e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:32.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.410+0000 7f0b9f7fe640 1 --2- 172.21.15.12:0/994386901 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ba0102d90 0x7f0ba019e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:32.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.410+0000 7f0b9f7fe640 1 -- 172.21.15.12:0/994386901 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0ba01011e0 msgr2=0x7f0ba01972e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:32.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.410+0000 7f0b9f7fe640 1 --2- 172.21.15.12:0/994386901 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0ba01011e0 0x7f0ba01972e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:32.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.410+0000 7f0b9f7fe640 1 -- 172.21.15.12:0/994386901 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b98009900 con 0x7f0ba01023d0 2024-02-18T11:40:32.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.410+0000 7f0ba4fcb640 1 --2- 172.21.15.12:0/994386901 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ba0102d90 0x7f0ba019e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:40:32.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.410+0000 7f0b9f7fe640 1 --2- 172.21.15.12:0/994386901 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ba01023d0 0x7f0ba0197820 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f0b9400c970 tx=0x7f0b9400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:32.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.410+0000 7f0b9d7fa640 1 -- 172.21.15.12:0/994386901 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b94007bb0 con 0x7f0ba01023d0 2024-02-18T11:40:32.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.410+0000 7f0b9d7fa640 1 -- 172.21.15.12:0/994386901 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0b94007d10 con 0x7f0ba01023d0 2024-02-18T11:40:32.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.410+0000 7f0ba6a2e640 1 -- 172.21.15.12:0/994386901 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ba019f3e0 con 0x7f0ba01023d0 2024-02-18T11:40:32.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.410+0000 7f0b9d7fa640 1 -- 172.21.15.12:0/994386901 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b9400f450 con 0x7f0ba01023d0 2024-02-18T11:40:32.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.410+0000 7f0ba6a2e640 1 -- 172.21.15.12:0/994386901 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ba019f8f0 con 0x7f0ba01023d0 2024-02-18T11:40:32.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.411+0000 7f0ba6a2e640 1 -- 172.21.15.12:0/994386901 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ba01084a0 con 0x7f0ba01023d0 2024-02-18T11:40:32.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.412+0000 7f0b9d7fa640 1 -- 172.21.15.12:0/994386901 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) v1 ==== 137245+0+0 (secure 0 0 0) 0x7f0b94004f70 con 0x7f0ba01023d0 2024-02-18T11:40:32.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.413+0000 7f0b9d7fa640 1 --2- 172.21.15.12:0/994386901 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f0b880a27d0 0x7f0b880a4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:32.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.413+0000 7f0b9ffff640 1 --2- 172.21.15.12:0/994386901 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f0b880a27d0 0x7f0b880a4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:32.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.413+0000 7f0b9d7fa640 1 -- 172.21.15.12:0/994386901 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f0b940ce330 con 0x7f0ba01023d0 2024-02-18T11:40:32.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.413+0000 7f0b9ffff640 1 --2- 172.21.15.12:0/994386901 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f0b880a27d0 0x7f0b880a4c90 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0ba0102230 tx=0x7f0b90008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:32.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.417+0000 7f0b9d7fa640 1 -- 172.21.15.12:0/994386901 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0b94097070 con 0x7f0ba01023d0 2024-02-18T11:40:32.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:40:32 smithi158 ceph-mon[36626]: pgmap v339: 97 pgs: 97 active+clean; 640 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-02-18T11:40:32.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:40:32 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/1932041949' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-18T11:40:32.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:40:32 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/2375984210' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-18T11:40:32.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:40:32 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/2764103234' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-18T11:40:32.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.600+0000 7f0ba6a2e640 1 -- 172.21.15.12:0/994386901 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0ba010cc70 con 0x7f0ba01023d0 2024-02-18T11:40:32.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.601+0000 7f0b9d7fa640 1 -- 172.21.15.12:0/994386901 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+22662 (secure 0 0 0) 0x7f0b9409a730 con 0x7f0ba01023d0 2024-02-18T11:40:32.596 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:40:32.596 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":89,"fsid":"0c8bb0de-ce51-11ee-95ba-87774f69a715","created":"2024-02-18T11:30:49.110059+0000","modified":"2024-02-18T11:39:52.478480+0000","last_up_change":"2024-02-18T11:37:57.512337+0000","last_in_change":"2024-02-18T11:37:36.567969+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-18T11:34:15.281138+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-18T11:38:44.802134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-18T11:38:46.081188+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"35f15ea6-deb8-48a1-a7dd-ab649ec13533","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":536522426},{"type":"v1","addr":"172.21.15.12:6803","nonce":536522426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":536522426},{"type":"v1","addr":"172.21.15.12:6805","nonce":536522426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":536522426},{"type":"v1","addr":"172.21.15.12:6809","nonce":536522426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":536522426},{"type":"v1","addr":"172.21.15.12:6807","nonce":536522426}]},"public_addr":"172.21.15.12:6803/536522426","cluster_addr":"172.21.15.12:6805/536522426","heartbeat_back_addr":"172.21.15.12:6809/536522426","heartbeat_front_addr":"172.21.15.12:6807/536522426","state":["exists","up"]},{"osd":1,"uuid":"2af937ef-d827-44e6-b805-70cffbfe1126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":4249111488},{"type":"v1","addr":"172.21.15.158:6801","nonce":4249111488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":4249111488},{"type":"v1","addr":"172.21.15.158:6803","nonce":4249111488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":4249111488},{"type":"v1","addr":"172.21.15.158:6807","nonce":4249111488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":4249111488},{"type":"v1","addr":"172.21.15.158:6805","nonce":4249111488}]},"public_addr":"172.21.15.158:6801/4249111488","cluster_addr":"172.21.15.158:6803/4249111488","heartbeat_back_addr":"172.21.15.158:6807/4249111488","heartbeat_front_addr":"172.21.15.158:6805/4249111488","state":["exists","up"]},{"osd":2,"uuid":"17b804dd-31df-45ae-a637-4a5f65bbaf26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":627976150},{"type":"v1","addr":"172.21.15.188:6801","nonce":627976150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":627976150},{"type":"v1","addr":"172.21.15.188:6803","nonce":627976150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":627976150},{"type":"v1","addr":"172.21.15.188:6807","nonce":627976150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":627976150},{"type":"v1","addr":"172.21.15.188:6805","nonce":627976150}]},"public_addr":"172.21.15.188:6801/627976150","cluster_addr":"172.21.15.188:6803/627976150","heartbeat_back_addr":"172.21.15.188:6807/627976150","heartbeat_front_addr":"172.21.15.188:6805/627976150","state":["exists","up"]},{"osd":3,"uuid":"3f9cc423-cdfa-4acf-aaa2-b58b32ccb9cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":3758512503},{"type":"v1","addr":"172.21.15.12:6811","nonce":3758512503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":3758512503},{"type":"v1","addr":"172.21.15.12:6813","nonce":3758512503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3758512503},{"type":"v1","addr":"172.21.15.12:6817","nonce":3758512503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":3758512503},{"type":"v1","addr":"172.21.15.12:6815","nonce":3758512503}]},"public_addr":"172.21.15.12:6811/3758512503","cluster_addr":"172.21.15.12:6813/3758512503","heartbeat_back_addr":"172.21.15.12:6817/3758512503","heartbeat_front_addr":"172.21.15.12:6815/3758512503","state":["exists","up"]},{"osd":4,"uuid":"a92e20ec-af1c-4ef4-a0cb-caba5fcc71aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":3892820997},{"type":"v1","addr":"172.21.15.158:6809","nonce":3892820997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":3892820997},{"type":"v1","addr":"172.21.15.158:6811","nonce":3892820997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":3892820997},{"type":"v1","addr":"172.21.15.158:6815","nonce":3892820997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":3892820997},{"type":"v1","addr":"172.21.15.158:6813","nonce":3892820997}]},"public_addr":"172.21.15.158:6809/3892820997","cluster_addr":"172.21.15.158:6811/3892820997","heartbeat_back_addr":"172.21.15.158:6815/3892820997","heartbeat_front_addr":"172.21.15.158:6813/3892820997","state":["exists","up"]},{"osd":5,"uuid":"d9c2e0c8-85f5-4ae2-bf3e-125c1be3a2db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3939495896},{"type":"v1","addr":"172.21.15.188:6809","nonce":3939495896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3939495896},{"type":"v1","addr":"172.21.15.188:6811","nonce":3939495896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3939495896},{"type":"v1","addr":"172.21.15.188:6815","nonce":3939495896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3939495896},{"type":"v1","addr":"172.21.15.188:6813","nonce":3939495896}]},"public_addr":"172.21.15.188:6809/3939495896","cluster_addr":"172.21.15.188:6811/3939495896","heartbeat_back_addr":"172.21.15.188:6815/3939495896","heartbeat_front_addr":"172.21.15.188:6813/3939495896","state":["exists","up"]},{"osd":6,"uuid":"4a523512-de6b-49f0-be81-00f2f00c1c10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":1206337767},{"type":"v1","addr":"172.21.15.12:6819","nonce":1206337767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":1206337767},{"type":"v1","addr":"172.21.15.12:6821","nonce":1206337767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":1206337767},{"type":"v1","addr":"172.21.15.12:6825","nonce":1206337767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":1206337767},{"type":"v1","addr":"172.21.15.12:6823","nonce":1206337767}]},"public_addr":"172.21.15.12:6819/1206337767","cluster_addr":"172.21.15.12:6821/1206337767","heartbeat_back_addr":"172.21.15.12:6825/1206337767","heartbeat_front_addr":"172.21.15.12:6823/1206337767","state":["exists","up"]},{"osd":7,"uuid":"363271a6-df93-44c9-b0e5-3d29768aa96c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":3283400032},{"type":"v1","addr":"172.21.15.158:6817","nonce":3283400032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":3283400032},{"type":"v1","addr":"172.21.15.158:6819","nonce":3283400032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":3283400032},{"type":"v1","addr":"172.21.15.158:6823","nonce":3283400032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":3283400032},{"type":"v1","addr":"172.21.15.158:6821","nonce":3283400032}]},"public_addr":"172.21.15.158:6817/3283400032","cluster_addr":"172.21.15.158:6819/3283400032","heartbeat_back_addr":"172.21.15.158:6823/3283400032","heartbeat_front_addr":"172.21.15.158:6821/3283400032","state":["exists","up"]},{"osd":8,"uuid":"9a089528-3e8e-45d3-9279-ddb54088b96a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2110170682},{"type":"v1","addr":"172.21.15.188:6817","nonce":2110170682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2110170682},{"type":"v1","addr":"172.21.15.188:6819","nonce":2110170682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":2110170682},{"type":"v1","addr":"172.21.15.188:6823","nonce":2110170682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2110170682},{"type":"v1","addr":"172.21.15.188:6821","nonce":2110170682}]},"public_addr":"172.21.15.188:6817/2110170682","cluster_addr":"172.21.15.188:6819/2110170682","heartbeat_back_addr":"172.21.15.188:6823/2110170682","heartbeat_front_addr":"172.21.15.188:6821/2110170682","state":["exists","up"]},{"osd":9,"uuid":"b12e2727-d649-4464-90e1-fafbc1754832","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1825928986},{"type":"v1","addr":"172.21.15.12:6827","nonce":1825928986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1825928986},{"type":"v1","addr":"172.21.15.12:6829","nonce":1825928986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":1825928986},{"type":"v1","addr":"172.21.15.12:6833","nonce":1825928986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1825928986},{"type":"v1","addr":"172.21.15.12:6831","nonce":1825928986}]},"public_addr":"172.21.15.12:6827/1825928986","cluster_addr":"172.21.15.12:6829/1825928986","heartbeat_back_addr":"172.21.15.12:6833/1825928986","heartbeat_front_addr":"172.21.15.12:6831/1825928986","state":["exists","up"]},{"osd":10,"uuid":"518b4cbf-9d90-4e9a-8ac9-b5cb9b3dee4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":3239052167},{"type":"v1","addr":"172.21.15.158:6825","nonce":3239052167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":3239052167},{"type":"v1","addr":"172.21.15.158:6827","nonce":3239052167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":3239052167},{"type":"v1","addr":"172.21.15.158:6831","nonce":3239052167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":3239052167},{"type":"v1","addr":"172.21.15.158:6829","nonce":3239052167}]},"public_addr":"172.21.15.158:6825/3239052167","cluster_addr":"172.21.15.158:6827/3239052167","heartbeat_back_addr":"172.21.15.158:6831/3239052167","heartbeat_front_addr":"172.21.15.158:6829/3239052167","state":["exists","up"]},{"osd":11,"uuid":"98aa3a0d-4b2f-46c8-9341-375d13794c3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1135773350},{"type":"v1","addr":"172.21.15.188:6825","nonce":1135773350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1135773350},{"type":"v1","addr":"172.21.15.188:6827","nonce":1135773350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1135773350},{"type":"v1","addr":"172.21.15.188:6831","nonce":1135773350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1135773350},{"type":"v1","addr":"172.21.15.188:6829","nonce":1135773350}]},"public_addr":"172.21.15.188:6825/1135773350","cluster_addr":"172.21.15.188:6827/1135773350","heartbeat_back_addr":"172.21.15.188:6831/1135773350","heartbeat_front_addr":"172.21.15.188:6829/1135773350","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:33:24.280879+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:33:47.859461+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:34:11.725283+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:34:35.707629+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:35:00.024197+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:35:24.499803+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:35:48.953870+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:36:13.642456+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:36:38.936457+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:37:04.021466+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:37:29.073178+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:37:54.863919+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.37","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:6801/4135274411":"2024-02-19T11:31:50.936928+0000","172.21.15.12:6800/4135274411":"2024-02-19T11:31:50.936928+0000","172.21.15.12:0/118428584":"2024-02-19T11:31:50.936928+0000","172.21.15.12:0/4276810915":"2024-02-19T11:31:50.936928+0000","172.21.15.12:0/3777545262":"2024-02-19T11:31:24.000830+0000","172.21.15.12:0/114707743":"2024-02-19T11:31:24.000830+0000","172.21.15.12:6800/3925204723":"2024-02-19T11:31:24.000830+0000","172.21.15.12:6801/3925204723":"2024-02-19T11:31:24.000830+0000","172.21.15.12:0/74881618":"2024-02-19T11:31:24.000830+0000","172.21.15.12:0/1614805375":"2024-02-19T11:31:50.936928+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-18T11:40:32.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.604+0000 7f0ba6a2e640 1 -- 172.21.15.12:0/994386901 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f0b880a27d0 msgr2=0x7f0b880a4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:32.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.604+0000 7f0ba6a2e640 1 --2- 172.21.15.12:0/994386901 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f0b880a27d0 0x7f0b880a4c90 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0ba0102230 tx=0x7f0b90008040 comp rx=0 tx=0).stop 2024-02-18T11:40:32.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.604+0000 7f0ba6a2e640 1 -- 172.21.15.12:0/994386901 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ba01023d0 msgr2=0x7f0ba0197820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:32.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.604+0000 7f0ba6a2e640 1 --2- 172.21.15.12:0/994386901 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ba01023d0 0x7f0ba0197820 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f0b9400c970 tx=0x7f0b9400ce30 comp rx=0 tx=0).stop 2024-02-18T11:40:32.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.604+0000 7f0ba6a2e640 1 -- 172.21.15.12:0/994386901 shutdown_connections 2024-02-18T11:40:32.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.604+0000 7f0ba6a2e640 1 --2- 172.21.15.12:0/994386901 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f0b880a27d0 0x7f0b880a4c90 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:32.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.604+0000 7f0ba6a2e640 1 --2- 172.21.15.12:0/994386901 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ba0102d90 0x7f0ba019e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:32.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.604+0000 7f0ba6a2e640 1 --2- 172.21.15.12:0/994386901 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ba01023d0 0x7f0ba0197820 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:32.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.604+0000 7f0ba6a2e640 1 --2- 172.21.15.12:0/994386901 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0ba01011e0 0x7f0ba01972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:32.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.604+0000 7f0ba6a2e640 1 -- 172.21.15.12:0/994386901 >> 172.21.15.12:0/994386901 conn(0x7f0ba00fc770 msgr2=0x7f0ba0105670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:32.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.604+0000 7f0ba6a2e640 1 -- 172.21.15.12:0/994386901 shutdown_connections 2024-02-18T11:40:32.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.604+0000 7f0ba6a2e640 1 -- 172.21.15.12:0/994386901 wait complete. 2024-02-18T11:40:32.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:40:32 smithi188 ceph-mon[36925]: pgmap v339: 97 pgs: 97 active+clean; 640 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-02-18T11:40:32.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:40:32 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/1932041949' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-18T11:40:32.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:40:32 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/2375984210' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-18T11:40:32.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:40:32 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/2764103234' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-18T11:40:32.609 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-18T11:34:15.281138+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-18T11:38:44.802134+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-18T11:38:46.081188+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-18T11:40:32.610 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-18T11:40:32.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.722+0000 7fdcd7c64640 1 -- 172.21.15.12:0/3699938773 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdcd0106f10 msgr2=0x7fdcd0109300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:32.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.722+0000 7fdcd7c64640 1 --2- 172.21.15.12:0/3699938773 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdcd0106f10 0x7fdcd0109300 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8009ca0 tx=0x7fdcc802c0f0 comp rx=0 tx=0).stop 2024-02-18T11:40:32.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.722+0000 7fdcd7c64640 1 -- 172.21.15.12:0/3699938773 shutdown_connections 2024-02-18T11:40:32.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.722+0000 7fdcd7c64640 1 --2- 172.21.15.12:0/3699938773 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdcd0106f10 0x7fdcd0109300 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:32.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.722+0000 7fdcd7c64640 1 --2- 172.21.15.12:0/3699938773 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdcd01045e0 0x7fdcd01069d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:32.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.722+0000 7fdcd7c64640 1 --2- 172.21.15.12:0/3699938773 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdcd0101cb0 0x7fdcd01040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:32.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.722+0000 7fdcd7c64640 1 -- 172.21.15.12:0/3699938773 >> 172.21.15.12:0/3699938773 conn(0x7fdcd00fb660 msgr2=0x7fdcd00fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:32.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.722+0000 7fdcd7c64640 1 -- 172.21.15.12:0/3699938773 shutdown_connections 2024-02-18T11:40:32.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.723+0000 7fdcd7c64640 1 -- 172.21.15.12:0/3699938773 wait complete. 2024-02-18T11:40:32.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.723+0000 7fdcd7c64640 1 Processor -- start 2024-02-18T11:40:32.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.723+0000 7fdcd7c64640 1 -- start start 2024-02-18T11:40:32.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.723+0000 7fdcd7c64640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdcd0101cb0 0x7fdcd0197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:32.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.723+0000 7fdcd7c64640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdcd01045e0 0x7fdcd0197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:32.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.723+0000 7fdcd7c64640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdcd0106f10 0x7fdcd019e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:32.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.723+0000 7fdcd7c64640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd019ed40 con 0x7fdcd0106f10 2024-02-18T11:40:32.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.723+0000 7fdcd7c64640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd019eeb0 con 0x7fdcd01045e0 2024-02-18T11:40:32.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.723+0000 7fdcd7c64640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd019f070 con 0x7fdcd0101cb0 2024-02-18T11:40:32.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.723+0000 7fdcd6201640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdcd0106f10 0x7fdcd019e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:32.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.723+0000 7fdcd6201640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdcd0106f10 0x7fdcd019e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48602/0 (socket says 172.21.15.12:48602) 2024-02-18T11:40:32.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.723+0000 7fdcd6201640 1 -- 172.21.15.12:0/3197125675 learned_addr learned my addr 172.21.15.12:0/3197125675 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:40:32.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.723+0000 7fdcd51ff640 1 --2- 172.21.15.12:0/3197125675 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdcd01045e0 0x7fdcd0197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:32.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.723+0000 7fdcd5a00640 1 --2- 172.21.15.12:0/3197125675 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdcd0101cb0 0x7fdcd0197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:32.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.724+0000 7fdcd6201640 1 -- 172.21.15.12:0/3197125675 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdcd0101cb0 msgr2=0x7fdcd0197250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:32.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.724+0000 7fdcd6201640 1 --2- 172.21.15.12:0/3197125675 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdcd0101cb0 0x7fdcd0197250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:32.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.724+0000 7fdcd6201640 1 -- 172.21.15.12:0/3197125675 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdcd01045e0 msgr2=0x7fdcd0197790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:32.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.724+0000 7fdcd6201640 1 --2- 172.21.15.12:0/3197125675 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdcd01045e0 0x7fdcd0197790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:32.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.724+0000 7fdcd6201640 1 -- 172.21.15.12:0/3197125675 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcc8009900 con 0x7fdcd0106f10 2024-02-18T11:40:32.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.724+0000 7fdcd51ff640 1 --2- 172.21.15.12:0/3197125675 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdcd01045e0 0x7fdcd0197790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:40:32.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.724+0000 7fdcd6201640 1 --2- 172.21.15.12:0/3197125675 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdcd0106f10 0x7fdcd019e800 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8037040 tx=0x7fdcc8034a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:32.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.724+0000 7fdcc2ffd640 1 -- 172.21.15.12:0/3197125675 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc80329b0 con 0x7fdcd0106f10 2024-02-18T11:40:32.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.724+0000 7fdcc2ffd640 1 -- 172.21.15.12:0/3197125675 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdcc8003e50 con 0x7fdcd0106f10 2024-02-18T11:40:32.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.724+0000 7fdcd7c64640 1 -- 172.21.15.12:0/3197125675 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcd019f2f0 con 0x7fdcd0106f10 2024-02-18T11:40:32.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.724+0000 7fdcc2ffd640 1 -- 172.21.15.12:0/3197125675 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc8040450 con 0x7fdcd0106f10 2024-02-18T11:40:32.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.724+0000 7fdcd7c64640 1 -- 172.21.15.12:0/3197125675 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcd019f7a0 con 0x7fdcd0106f10 2024-02-18T11:40:32.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.725+0000 7fdcd7c64640 1 -- 172.21.15.12:0/3197125675 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcd00fd150 con 0x7fdcd0106f10 2024-02-18T11:40:32.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.726+0000 7fdcc2ffd640 1 -- 172.21.15.12:0/3197125675 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) v1 ==== 137245+0+0 (secure 0 0 0) 0x7fdcc800a040 con 0x7fdcd0106f10 2024-02-18T11:40:32.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.726+0000 7fdcc2ffd640 1 --2- 172.21.15.12:0/3197125675 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fdcbc0a2750 0x7fdcbc0a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:32.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.727+0000 7fdcd5a00640 1 --2- 172.21.15.12:0/3197125675 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fdcbc0a2750 0x7fdcbc0a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:32.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.727+0000 7fdcc2ffd640 1 -- 172.21.15.12:0/3197125675 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fdcc80f1400 con 0x7fdcd0106f10 2024-02-18T11:40:32.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.727+0000 7fdcd5a00640 1 --2- 172.21.15.12:0/3197125675 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fdcbc0a2750 0x7fdcbc0a4c10 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fdcc4009bd0 tx=0x7fdcc4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:32.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.731+0000 7fdcc2ffd640 1 -- 172.21.15.12:0/3197125675 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdcc80ba1a0 con 0x7fdcd0106f10 2024-02-18T11:40:32.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.911+0000 7fdcd7c64640 1 -- 172.21.15.12:0/3197125675 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdcd002cde0 con 0x7fdcd0106f10 2024-02-18T11:40:32.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.912+0000 7fdcc2ffd640 1 -- 172.21.15.12:0/3197125675 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdcc80bd860 con 0x7fdcd0106f10 2024-02-18T11:40:32.906 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-02-18T11:40:32.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.914+0000 7fdcd7c64640 1 -- 172.21.15.12:0/3197125675 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fdcbc0a2750 msgr2=0x7fdcbc0a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:32.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.914+0000 7fdcd7c64640 1 --2- 172.21.15.12:0/3197125675 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fdcbc0a2750 0x7fdcbc0a4c10 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fdcc4009bd0 tx=0x7fdcc4005f90 comp rx=0 tx=0).stop 2024-02-18T11:40:32.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.914+0000 7fdcd7c64640 1 -- 172.21.15.12:0/3197125675 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdcd0106f10 msgr2=0x7fdcd019e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:32.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.914+0000 7fdcd7c64640 1 --2- 172.21.15.12:0/3197125675 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdcd0106f10 0x7fdcd019e800 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8037040 tx=0x7fdcc8034a90 comp rx=0 tx=0).stop 2024-02-18T11:40:32.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.914+0000 7fdcd7c64640 1 -- 172.21.15.12:0/3197125675 shutdown_connections 2024-02-18T11:40:32.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.914+0000 7fdcd7c64640 1 --2- 172.21.15.12:0/3197125675 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fdcbc0a2750 0x7fdcbc0a4c10 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:32.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.914+0000 7fdcd7c64640 1 --2- 172.21.15.12:0/3197125675 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdcd0106f10 0x7fdcd019e800 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:32.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.914+0000 7fdcd7c64640 1 --2- 172.21.15.12:0/3197125675 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdcd01045e0 0x7fdcd0197790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:32.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.914+0000 7fdcd7c64640 1 --2- 172.21.15.12:0/3197125675 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdcd0101cb0 0x7fdcd0197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:32.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.914+0000 7fdcd7c64640 1 -- 172.21.15.12:0/3197125675 >> 172.21.15.12:0/3197125675 conn(0x7fdcd00fb660 msgr2=0x7fdcd01025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:32.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.914+0000 7fdcd7c64640 1 -- 172.21.15.12:0/3197125675 shutdown_connections 2024-02-18T11:40:32.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:32.914+0000 7fdcd7c64640 1 -- 172.21.15.12:0/3197125675 wait complete. 2024-02-18T11:40:32.919 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-18T11:40:33.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.030+0000 7fde08e2e640 1 -- 172.21.15.12:0/2497693811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fde04101cb0 msgr2=0x7fde041040a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:33.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.030+0000 7fde08e2e640 1 --2- 172.21.15.12:0/2497693811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fde04101cb0 0x7fde041040a0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fddf4009900 tx=0x7fddf402bc30 comp rx=0 tx=0).stop 2024-02-18T11:40:33.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.030+0000 7fde08e2e640 1 -- 172.21.15.12:0/2497693811 shutdown_connections 2024-02-18T11:40:33.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.030+0000 7fde08e2e640 1 --2- 172.21.15.12:0/2497693811 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde041045e0 0x7fde041069d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:33.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.030+0000 7fde08e2e640 1 --2- 172.21.15.12:0/2497693811 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fde04106f10 0x7fde04109300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:33.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.030+0000 7fde08e2e640 1 --2- 172.21.15.12:0/2497693811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fde04101cb0 0x7fde041040a0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:33.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.030+0000 7fde08e2e640 1 -- 172.21.15.12:0/2497693811 >> 172.21.15.12:0/2497693811 conn(0x7fde040fb660 msgr2=0x7fde040fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:33.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.030+0000 7fde08e2e640 1 -- 172.21.15.12:0/2497693811 shutdown_connections 2024-02-18T11:40:33.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.030+0000 7fde08e2e640 1 -- 172.21.15.12:0/2497693811 wait complete. 2024-02-18T11:40:33.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.031+0000 7fde08e2e640 1 Processor -- start 2024-02-18T11:40:33.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.031+0000 7fde08e2e640 1 -- start start 2024-02-18T11:40:33.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.031+0000 7fde08e2e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde04101cb0 0x7fde04101620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:33.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.031+0000 7fde08e2e640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fde041045e0 0x7fde040ffc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:33.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.031+0000 7fde08e2e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fde04106f10 0x7fde041001a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:33.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.031+0000 7fde08e2e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde04100900 con 0x7fde04106f10 2024-02-18T11:40:33.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.031+0000 7fde08e2e640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde04100a70 con 0x7fde041045e0 2024-02-18T11:40:33.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.031+0000 7fde08e2e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde041a1390 con 0x7fde04101cb0 2024-02-18T11:40:33.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.031+0000 7fde02d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fde04106f10 0x7fde041001a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:33.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.031+0000 7fde02d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fde04106f10 0x7fde041001a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48620/0 (socket says 172.21.15.12:48620) 2024-02-18T11:40:33.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.031+0000 7fde02d9d640 1 -- 172.21.15.12:0/1059862397 learned_addr learned my addr 172.21.15.12:0/1059862397 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:40:33.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.031+0000 7fde0259c640 1 --2- 172.21.15.12:0/1059862397 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde04101cb0 0x7fde04101620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:33.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.031+0000 7fde01d9b640 1 --2- 172.21.15.12:0/1059862397 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fde041045e0 0x7fde040ffc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:33.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.031+0000 7fde02d9d640 1 -- 172.21.15.12:0/1059862397 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde04101cb0 msgr2=0x7fde04101620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:33.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.031+0000 7fde02d9d640 1 --2- 172.21.15.12:0/1059862397 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde04101cb0 0x7fde04101620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:33.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.031+0000 7fde02d9d640 1 -- 172.21.15.12:0/1059862397 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fde041045e0 msgr2=0x7fde040ffc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:33.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.031+0000 7fde02d9d640 1 --2- 172.21.15.12:0/1059862397 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fde041045e0 0x7fde040ffc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:33.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.031+0000 7fde02d9d640 1 -- 172.21.15.12:0/1059862397 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddf4009560 con 0x7fde04106f10 2024-02-18T11:40:33.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.031+0000 7fde01d9b640 1 --2- 172.21.15.12:0/1059862397 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fde041045e0 0x7fde040ffc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:40:33.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.032+0000 7fde02d9d640 1 --2- 172.21.15.12:0/1059862397 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fde04106f10 0x7fde041001a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fddfc00b9a0 tx=0x7fddfc00be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:33.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.032+0000 7fddf37fe640 1 -- 172.21.15.12:0/1059862397 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddfc00cba0 con 0x7fde04106f10 2024-02-18T11:40:33.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.032+0000 7fddf37fe640 1 -- 172.21.15.12:0/1059862397 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fddfc00cd00 con 0x7fde04106f10 2024-02-18T11:40:33.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.032+0000 7fddf37fe640 1 -- 172.21.15.12:0/1059862397 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddfc0076f0 con 0x7fde04106f10 2024-02-18T11:40:33.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.032+0000 7fde08e2e640 1 -- 172.21.15.12:0/1059862397 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde041a1640 con 0x7fde04106f10 2024-02-18T11:40:33.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.032+0000 7fde08e2e640 1 -- 172.21.15.12:0/1059862397 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde041a1bd0 con 0x7fde04106f10 2024-02-18T11:40:33.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.033+0000 7fde08e2e640 1 -- 172.21.15.12:0/1059862397 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde040fd150 con 0x7fde04106f10 2024-02-18T11:40:33.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.034+0000 7fddf37fe640 1 -- 172.21.15.12:0/1059862397 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) v1 ==== 137245+0+0 (secure 0 0 0) 0x7fddfc014a20 con 0x7fde04106f10 2024-02-18T11:40:33.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.034+0000 7fddf37fe640 1 --2- 172.21.15.12:0/1059862397 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fddec0a27d0 0x7fddec0a4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:33.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.034+0000 7fde0259c640 1 --2- 172.21.15.12:0/1059862397 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fddec0a27d0 0x7fddec0a4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:33.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.034+0000 7fddf37fe640 1 -- 172.21.15.12:0/1059862397 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fddfc0cdd60 con 0x7fde04106f10 2024-02-18T11:40:33.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.035+0000 7fde0259c640 1 --2- 172.21.15.12:0/1059862397 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fddec0a27d0 0x7fddec0a4c90 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fddf4006010 tx=0x7fddf4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:33.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.039+0000 7fddf37fe640 1 -- 172.21.15.12:0/1059862397 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fddfc04fd20 con 0x7fde04106f10 2024-02-18T11:40:33.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.220+0000 7fde08e2e640 1 -- 172.21.15.12:0/1059862397 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fde0402ce80 con 0x7fde04106f10 2024-02-18T11:40:33.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.220+0000 7fddf37fe640 1 -- 172.21.15.12:0/1059862397 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7fddfc096bb0 con 0x7fde04106f10 2024-02-18T11:40:33.214 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-02-18T11:40:33.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.222+0000 7fde08e2e640 1 -- 172.21.15.12:0/1059862397 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fddec0a27d0 msgr2=0x7fddec0a4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:33.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.222+0000 7fde08e2e640 1 --2- 172.21.15.12:0/1059862397 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fddec0a27d0 0x7fddec0a4c90 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fddf4006010 tx=0x7fddf4000bc0 comp rx=0 tx=0).stop 2024-02-18T11:40:33.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.222+0000 7fde08e2e640 1 -- 172.21.15.12:0/1059862397 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fde04106f10 msgr2=0x7fde041001a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:33.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.222+0000 7fde08e2e640 1 --2- 172.21.15.12:0/1059862397 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fde04106f10 0x7fde041001a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fddfc00b9a0 tx=0x7fddfc00be60 comp rx=0 tx=0).stop 2024-02-18T11:40:33.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.222+0000 7fde08e2e640 1 -- 172.21.15.12:0/1059862397 shutdown_connections 2024-02-18T11:40:33.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.222+0000 7fde08e2e640 1 --2- 172.21.15.12:0/1059862397 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fddec0a27d0 0x7fddec0a4c90 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:33.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.222+0000 7fde08e2e640 1 --2- 172.21.15.12:0/1059862397 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fde04106f10 0x7fde041001a0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:33.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.222+0000 7fde08e2e640 1 --2- 172.21.15.12:0/1059862397 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fde041045e0 0x7fde040ffc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:33.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.222+0000 7fde08e2e640 1 --2- 172.21.15.12:0/1059862397 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde04101cb0 0x7fde04101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:33.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.222+0000 7fde08e2e640 1 -- 172.21.15.12:0/1059862397 >> 172.21.15.12:0/1059862397 conn(0x7fde040fb660 msgr2=0x7fde041025f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:33.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.222+0000 7fde08e2e640 1 -- 172.21.15.12:0/1059862397 shutdown_connections 2024-02-18T11:40:33.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.222+0000 7fde08e2e640 1 -- 172.21.15.12:0/1059862397 wait complete. 2024-02-18T11:40:33.227 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-18T11:40:33.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.338+0000 7f06954df640 1 -- 172.21.15.12:0/3571536390 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f06901045f0 msgr2=0x7f06901069e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:33.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.338+0000 7f06954df640 1 --2- 172.21.15.12:0/3571536390 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f06901045f0 0x7f06901069e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f0684009900 tx=0x7f068402bc30 comp rx=0 tx=0).stop 2024-02-18T11:40:33.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.338+0000 7f06954df640 1 -- 172.21.15.12:0/3571536390 shutdown_connections 2024-02-18T11:40:33.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.338+0000 7f06954df640 1 --2- 172.21.15.12:0/3571536390 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0690106f20 0x7f0690109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:33.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.338+0000 7f06954df640 1 --2- 172.21.15.12:0/3571536390 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f06901045f0 0x7f06901069e0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:33.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.338+0000 7f06954df640 1 --2- 172.21.15.12:0/3571536390 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0690101cc0 0x7f06901040b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:33.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.338+0000 7f06954df640 1 -- 172.21.15.12:0/3571536390 >> 172.21.15.12:0/3571536390 conn(0x7f06900fb650 msgr2=0x7f06900fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:33.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.338+0000 7f06954df640 1 -- 172.21.15.12:0/3571536390 shutdown_connections 2024-02-18T11:40:33.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.338+0000 7f06954df640 1 -- 172.21.15.12:0/3571536390 wait complete. 2024-02-18T11:40:33.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.339+0000 7f06954df640 1 Processor -- start 2024-02-18T11:40:33.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.339+0000 7f06954df640 1 -- start start 2024-02-18T11:40:33.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.339+0000 7f06954df640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0690101cc0 0x7f0690101630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:33.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.339+0000 7f06954df640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f06901045f0 0x7f06900ffc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:33.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.339+0000 7f06954df640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0690106f20 0x7f06901001b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:33.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.339+0000 7f06954df640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0690100910 con 0x7f06901045f0 2024-02-18T11:40:33.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.339+0000 7f06954df640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0690100a80 con 0x7f0690101cc0 2024-02-18T11:40:33.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.339+0000 7f06954df640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06901a13a0 con 0x7f0690106f20 2024-02-18T11:40:33.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.339+0000 7f068e7fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f06901045f0 0x7f06900ffc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:33.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.339+0000 7f068e7fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f06901045f0 0x7f06900ffc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48634/0 (socket says 172.21.15.12:48634) 2024-02-18T11:40:33.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.339+0000 7f068e7fc640 1 -- 172.21.15.12:0/876170320 learned_addr learned my addr 172.21.15.12:0/876170320 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:40:33.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.340+0000 7f068f7fe640 1 --2- 172.21.15.12:0/876170320 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0690106f20 0x7f06901001b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:33.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.340+0000 7f068e7fc640 1 -- 172.21.15.12:0/876170320 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0690106f20 msgr2=0x7f06901001b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:33.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.340+0000 7f068e7fc640 1 --2- 172.21.15.12:0/876170320 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0690106f20 0x7f06901001b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:33.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.340+0000 7f068e7fc640 1 -- 172.21.15.12:0/876170320 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0690101cc0 msgr2=0x7f0690101630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:33.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.340+0000 7f068effd640 1 --2- 172.21.15.12:0/876170320 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0690101cc0 0x7f0690101630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:33.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.340+0000 7f068e7fc640 1 --2- 172.21.15.12:0/876170320 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0690101cc0 0x7f0690101630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:33.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.340+0000 7f068e7fc640 1 -- 172.21.15.12:0/876170320 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0684009560 con 0x7f06901045f0 2024-02-18T11:40:33.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.340+0000 7f068e7fc640 1 --2- 172.21.15.12:0/876170320 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f06901045f0 0x7f06900ffc70 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f0684036040 tx=0x7f0684004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:33.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.340+0000 7f0677fff640 1 -- 172.21.15.12:0/876170320 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0684004df0 con 0x7f06901045f0 2024-02-18T11:40:33.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.340+0000 7f0677fff640 1 -- 172.21.15.12:0/876170320 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0684005430 con 0x7f06901045f0 2024-02-18T11:40:33.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.340+0000 7f06954df640 1 -- 172.21.15.12:0/876170320 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06901a15f0 con 0x7f06901045f0 2024-02-18T11:40:33.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.340+0000 7f0677fff640 1 -- 172.21.15.12:0/876170320 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f068403e450 con 0x7f06901045f0 2024-02-18T11:40:33.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.340+0000 7f06954df640 1 -- 172.21.15.12:0/876170320 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06901a1b00 con 0x7f06901045f0 2024-02-18T11:40:33.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.341+0000 7f06954df640 1 -- 172.21.15.12:0/876170320 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06900fd140 con 0x7f06901045f0 2024-02-18T11:40:33.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.342+0000 7f0677fff640 1 -- 172.21.15.12:0/876170320 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) v1 ==== 137245+0+0 (secure 0 0 0) 0x7f0684003680 con 0x7f06901045f0 2024-02-18T11:40:33.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.342+0000 7f0677fff640 1 --2- 172.21.15.12:0/876170320 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f06780a2750 0x7f06780a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:33.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.343+0000 7f068effd640 1 --2- 172.21.15.12:0/876170320 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f06780a2750 0x7f06780a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:33.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.343+0000 7f0677fff640 1 -- 172.21.15.12:0/876170320 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f06840f0b40 con 0x7f06901045f0 2024-02-18T11:40:33.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.343+0000 7f068effd640 1 --2- 172.21.15.12:0/876170320 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f06780a2750 0x7f06780a4c10 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f068800a810 tx=0x7f0688008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:33.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.347+0000 7f0677fff640 1 -- 172.21.15.12:0/876170320 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f06840b98e0 con 0x7f06901045f0 2024-02-18T11:40:33.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:40:33 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/662843696' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-18T11:40:33.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:40:33 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/994386901' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-18T11:40:33.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:40:33 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/3197125675' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-18T11:40:33.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.530+0000 7f06954df640 1 -- 172.21.15.12:0/876170320 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f069002ce10 con 0x7f06901045f0 2024-02-18T11:40:33.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.531+0000 7f0677fff640 1 -- 172.21.15.12:0/876170320 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+11 (secure 0 0 0) 0x7f06840bcfa0 con 0x7f06901045f0 2024-02-18T11:40:33.525 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2024-02-18T11:40:33.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.533+0000 7f06954df640 1 -- 172.21.15.12:0/876170320 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f06780a2750 msgr2=0x7f06780a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:33.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.533+0000 7f06954df640 1 --2- 172.21.15.12:0/876170320 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f06780a2750 0x7f06780a4c10 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f068800a810 tx=0x7f0688008040 comp rx=0 tx=0).stop 2024-02-18T11:40:33.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.533+0000 7f06954df640 1 -- 172.21.15.12:0/876170320 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f06901045f0 msgr2=0x7f06900ffc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:33.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.533+0000 7f06954df640 1 --2- 172.21.15.12:0/876170320 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f06901045f0 0x7f06900ffc70 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f0684036040 tx=0x7f0684004270 comp rx=0 tx=0).stop 2024-02-18T11:40:33.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.533+0000 7f06954df640 1 -- 172.21.15.12:0/876170320 shutdown_connections 2024-02-18T11:40:33.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.533+0000 7f06954df640 1 --2- 172.21.15.12:0/876170320 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f06780a2750 0x7f06780a4c10 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:33.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.533+0000 7f06954df640 1 --2- 172.21.15.12:0/876170320 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0690106f20 0x7f06901001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:33.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.533+0000 7f06954df640 1 --2- 172.21.15.12:0/876170320 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f06901045f0 0x7f06900ffc70 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:33.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.533+0000 7f06954df640 1 --2- 172.21.15.12:0/876170320 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0690101cc0 0x7f0690101630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:33.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.533+0000 7f06954df640 1 -- 172.21.15.12:0/876170320 >> 172.21.15.12:0/876170320 conn(0x7f06900fb650 msgr2=0x7f0690102760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:33.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.533+0000 7f06954df640 1 -- 172.21.15.12:0/876170320 shutdown_connections 2024-02-18T11:40:33.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.533+0000 7f06954df640 1 -- 172.21.15.12:0/876170320 wait complete. 2024-02-18T11:40:33.537 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:40:33 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/662843696' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-18T11:40:33.537 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:40:33 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/994386901' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-18T11:40:33.537 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:40:33 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/3197125675' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-18T11:40:33.538 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-18T11:40:33.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:40:33 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/662843696' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-18T11:40:33.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:40:33 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/994386901' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-18T11:40:33.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:40:33 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/3197125675' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-18T11:40:33.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.645+0000 7fb7b50e1640 1 -- 172.21.15.12:0/3270586318 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7b01012d0 msgr2=0x7fb7b0101750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:33.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.645+0000 7fb7b50e1640 1 --2- 172.21.15.12:0/3270586318 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7b01012d0 0x7fb7b0101750 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb798009900 tx=0x7fb79802bc30 comp rx=0 tx=0).stop 2024-02-18T11:40:33.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.645+0000 7fb7b50e1640 1 -- 172.21.15.12:0/3270586318 shutdown_connections 2024-02-18T11:40:33.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.645+0000 7fb7b50e1640 1 --2- 172.21.15.12:0/3270586318 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb7b0101c90 0x7fb7b0108520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:33.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.645+0000 7fb7b50e1640 1 --2- 172.21.15.12:0/3270586318 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7b01012d0 0x7fb7b0101750 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:33.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.645+0000 7fb7b50e1640 1 --2- 172.21.15.12:0/3270586318 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb7b01000e0 0x7fb7b0100500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:33.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.645+0000 7fb7b50e1640 1 -- 172.21.15.12:0/3270586318 >> 172.21.15.12:0/3270586318 conn(0x7fb7b00fb650 msgr2=0x7fb7b00fda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:33.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.645+0000 7fb7b50e1640 1 -- 172.21.15.12:0/3270586318 shutdown_connections 2024-02-18T11:40:33.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.645+0000 7fb7b50e1640 1 -- 172.21.15.12:0/3270586318 wait complete. 2024-02-18T11:40:33.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.646+0000 7fb7b50e1640 1 Processor -- start 2024-02-18T11:40:33.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.646+0000 7fb7b50e1640 1 -- start start 2024-02-18T11:40:33.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.646+0000 7fb7b50e1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7b01000e0 0x7fb7b0197390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:33.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.646+0000 7fb7b50e1640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb7b01012d0 0x7fb7b01978d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:33.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.646+0000 7fb7b50e1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb7b0101c90 0x7fb7b019e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:33.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.646+0000 7fb7b50e1640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b019ee80 con 0x7fb7b01000e0 2024-02-18T11:40:33.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.646+0000 7fb7b50e1640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b019eff0 con 0x7fb7b01012d0 2024-02-18T11:40:33.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.646+0000 7fb7b50e1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b019f1b0 con 0x7fb7b0101c90 2024-02-18T11:40:33.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.646+0000 7fb7aed9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7b01000e0 0x7fb7b0197390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:33.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.646+0000 7fb7aed9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7b01000e0 0x7fb7b0197390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48662/0 (socket says 172.21.15.12:48662) 2024-02-18T11:40:33.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.646+0000 7fb7aed9d640 1 -- 172.21.15.12:0/2818017034 learned_addr learned my addr 172.21.15.12:0/2818017034 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:40:33.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.647+0000 7fb7aed9d640 1 -- 172.21.15.12:0/2818017034 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb7b0101c90 msgr2=0x7fb7b019e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:33.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.647+0000 7fb7aed9d640 1 --2- 172.21.15.12:0/2818017034 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb7b0101c90 0x7fb7b019e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:33.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.647+0000 7fb7aed9d640 1 -- 172.21.15.12:0/2818017034 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb7b01012d0 msgr2=0x7fb7b01978d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:33.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.647+0000 7fb7aed9d640 1 --2- 172.21.15.12:0/2818017034 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb7b01012d0 0x7fb7b01978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:33.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.647+0000 7fb7aed9d640 1 -- 172.21.15.12:0/2818017034 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb798009560 con 0x7fb7b01000e0 2024-02-18T11:40:33.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.647+0000 7fb7aed9d640 1 --2- 172.21.15.12:0/2818017034 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7b01000e0 0x7fb7b0197390 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb7a800b450 tx=0x7fb7a800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:33.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.647+0000 7fb7a7fff640 1 -- 172.21.15.12:0/2818017034 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7a8011890 con 0x7fb7b01000e0 2024-02-18T11:40:33.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.647+0000 7fb7a7fff640 1 -- 172.21.15.12:0/2818017034 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb7a80103e0 con 0x7fb7b01000e0 2024-02-18T11:40:33.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.647+0000 7fb7b50e1640 1 -- 172.21.15.12:0/2818017034 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7b019f490 con 0x7fb7b01000e0 2024-02-18T11:40:33.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.647+0000 7fb7a7fff640 1 -- 172.21.15.12:0/2818017034 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7a800f580 con 0x7fb7b01000e0 2024-02-18T11:40:33.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.647+0000 7fb7b50e1640 1 -- 172.21.15.12:0/2818017034 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7b019fa20 con 0x7fb7b01000e0 2024-02-18T11:40:33.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.648+0000 7fb7b50e1640 1 -- 172.21.15.12:0/2818017034 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7b010bb70 con 0x7fb7b01000e0 2024-02-18T11:40:33.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.649+0000 7fb7a7fff640 1 -- 172.21.15.12:0/2818017034 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) v1 ==== 137245+0+0 (secure 0 0 0) 0x7fb7a80108e0 con 0x7fb7b01000e0 2024-02-18T11:40:33.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.650+0000 7fb7a7fff640 1 --2- 172.21.15.12:0/2818017034 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fb79c0a2750 0x7fb79c0a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:33.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.650+0000 7fb7ae59c640 1 --2- 172.21.15.12:0/2818017034 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fb79c0a2750 0x7fb79c0a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:33.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.650+0000 7fb7a7fff640 1 -- 172.21.15.12:0/2818017034 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb7a80ce3d0 con 0x7fb7b01000e0 2024-02-18T11:40:33.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.650+0000 7fb7ae59c640 1 --2- 172.21.15.12:0/2818017034 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fb79c0a2750 0x7fb79c0a4c10 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb79802c6a0 tx=0x7fb79802c740 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:33.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.654+0000 7fb7a7fff640 1 -- 172.21.15.12:0/2818017034 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb7a8097110 con 0x7fb7b01000e0 2024-02-18T11:40:33.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.871+0000 7fb7b50e1640 1 -- 172.21.15.12:0/2818017034 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb7b019fca0 con 0x7fb7b01000e0 2024-02-18T11:40:33.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.872+0000 7fb7a7fff640 1 -- 172.21.15.12:0/2818017034 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11349 (secure 0 0 0) 0x7fb7a809a7d0 con 0x7fb7b01000e0 2024-02-18T11:40:33.866 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:40:33.866 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1788335302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1788335302},{"type":"v1","addr":"172.21.15.12:6835","nonce":1788335302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/2144692455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":2144692455},{"type":"v1","addr":"172.21.15.12:6837","nonce":2144692455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3569056036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3569056036},{"type":"v1","addr":"172.21.15.12:6839","nonce":3569056036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14622,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/1973234486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":1973234486},{"type":"v1","addr":"172.21.15.12:6841","nonce":1973234486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1207202465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1207202465},{"type":"v1","addr":"172.21.15.188:6833","nonce":1207202465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6833/2636348323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6832","nonce":2636348323},{"type":"v1","addr":"172.21.15.158:6833","nonce":2636348323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-18T11:38:46.888173+0000","modified":"2024-02-18T11:38:51.531552+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24524,"mds_1":24508,"mds_2":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.158:6835/3673253642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":3673253642},{"type":"v1","addr":"172.21.15.158:6835","nonce":3673253642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/806709644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":806709644},{"type":"v1","addr":"172.21.15.188:6835","nonce":806709644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.158:6837/813251825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6836","nonce":813251825},{"type":"v1","addr":"172.21.15.158:6837","nonce":813251825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.188:6837/1036630143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1036630143},{"type":"v1","addr":"172.21.15.188:6837","nonce":1036630143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.188:6839/1342671190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":1342671190},{"type":"v1","addr":"172.21.15.188:6839","nonce":1342671190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.158:6839/1127200849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6838","nonce":1127200849},{"type":"v1","addr":"172.21.15.158:6839","nonce":1127200849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-18T11:40:33.866 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 20 2024-02-18T11:40:33.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.874+0000 7fb7b50e1640 1 -- 172.21.15.12:0/2818017034 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fb79c0a2750 msgr2=0x7fb79c0a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:33.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.874+0000 7fb7b50e1640 1 --2- 172.21.15.12:0/2818017034 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fb79c0a2750 0x7fb79c0a4c10 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb79802c6a0 tx=0x7fb79802c740 comp rx=0 tx=0).stop 2024-02-18T11:40:33.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.874+0000 7fb7b50e1640 1 -- 172.21.15.12:0/2818017034 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7b01000e0 msgr2=0x7fb7b0197390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:33.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.874+0000 7fb7b50e1640 1 --2- 172.21.15.12:0/2818017034 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7b01000e0 0x7fb7b0197390 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb7a800b450 tx=0x7fb7a800b910 comp rx=0 tx=0).stop 2024-02-18T11:40:33.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.874+0000 7fb7b50e1640 1 -- 172.21.15.12:0/2818017034 shutdown_connections 2024-02-18T11:40:33.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.874+0000 7fb7b50e1640 1 --2- 172.21.15.12:0/2818017034 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fb79c0a2750 0x7fb79c0a4c10 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:33.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.874+0000 7fb7b50e1640 1 --2- 172.21.15.12:0/2818017034 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb7b0101c90 0x7fb7b019e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:33.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.874+0000 7fb7b50e1640 1 --2- 172.21.15.12:0/2818017034 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb7b01012d0 0x7fb7b01978d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:33.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.874+0000 7fb7b50e1640 1 --2- 172.21.15.12:0/2818017034 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7b01000e0 0x7fb7b0197390 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:33.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.874+0000 7fb7b50e1640 1 -- 172.21.15.12:0/2818017034 >> 172.21.15.12:0/2818017034 conn(0x7fb7b00fb650 msgr2=0x7fb7b00fd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:33.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.875+0000 7fb7b50e1640 1 -- 172.21.15.12:0/2818017034 shutdown_connections 2024-02-18T11:40:33.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.875+0000 7fb7b50e1640 1 -- 172.21.15.12:0/2818017034 wait complete. 2024-02-18T11:40:33.880 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-18T11:40:33.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.989+0000 7fafe1593640 1 -- 172.21.15.12:0/2752506552 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fafdc1011e0 msgr2=0x7fafdc101600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:33.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.989+0000 7fafe1593640 1 --2- 172.21.15.12:0/2752506552 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fafdc1011e0 0x7fafdc101600 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fafd4009900 tx=0x7fafd402bc30 comp rx=0 tx=0).stop 2024-02-18T11:40:33.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.989+0000 7fafe1593640 1 -- 172.21.15.12:0/2752506552 shutdown_connections 2024-02-18T11:40:33.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.989+0000 7fafe1593640 1 --2- 172.21.15.12:0/2752506552 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fafdc102d90 0x7fafdc109620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:33.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.989+0000 7fafe1593640 1 --2- 172.21.15.12:0/2752506552 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fafdc1023d0 0x7fafdc102850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:33.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.989+0000 7fafe1593640 1 --2- 172.21.15.12:0/2752506552 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fafdc1011e0 0x7fafdc101600 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:33.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.989+0000 7fafe1593640 1 -- 172.21.15.12:0/2752506552 >> 172.21.15.12:0/2752506552 conn(0x7fafdc0fc770 msgr2=0x7fafdc0feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:33.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.989+0000 7fafe1593640 1 -- 172.21.15.12:0/2752506552 shutdown_connections 2024-02-18T11:40:33.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.989+0000 7fafe1593640 1 -- 172.21.15.12:0/2752506552 wait complete. 2024-02-18T11:40:33.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.989+0000 7fafe1593640 1 Processor -- start 2024-02-18T11:40:33.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.989+0000 7fafe1593640 1 -- start start 2024-02-18T11:40:33.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.989+0000 7fafe1593640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fafdc1011e0 0x7fafdc197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:33.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.989+0000 7fafe1593640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fafdc1023d0 0x7fafdc197770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:33.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.989+0000 7fafe1593640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fafdc102d90 0x7fafdc19e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:33.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.990+0000 7fafe1593640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafdc19ed20 con 0x7fafdc102d90 2024-02-18T11:40:33.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.990+0000 7fafe1593640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafdc19ee90 con 0x7fafdc1023d0 2024-02-18T11:40:33.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.990+0000 7fafe1593640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafdc19f050 con 0x7fafdc1011e0 2024-02-18T11:40:33.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.990+0000 7fafdb7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fafdc102d90 0x7fafdc19e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:33.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.990+0000 7fafdb7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fafdc102d90 0x7fafdc19e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48686/0 (socket says 172.21.15.12:48686) 2024-02-18T11:40:33.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.990+0000 7fafdb7fe640 1 -- 172.21.15.12:0/3723350426 learned_addr learned my addr 172.21.15.12:0/3723350426 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:40:33.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.990+0000 7fafdb7fe640 1 -- 172.21.15.12:0/3723350426 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fafdc1011e0 msgr2=0x7fafdc197230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:33.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.990+0000 7fafdb7fe640 1 --2- 172.21.15.12:0/3723350426 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fafdc1011e0 0x7fafdc197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:33.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.990+0000 7fafdb7fe640 1 -- 172.21.15.12:0/3723350426 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fafdc1023d0 msgr2=0x7fafdc197770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:33.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.990+0000 7fafdb7fe640 1 --2- 172.21.15.12:0/3723350426 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fafdc1023d0 0x7fafdc197770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:33.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.990+0000 7fafdb7fe640 1 -- 172.21.15.12:0/3723350426 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafd4009560 con 0x7fafdc102d90 2024-02-18T11:40:33.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.990+0000 7fafdb7fe640 1 --2- 172.21.15.12:0/3723350426 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fafdc102d90 0x7fafdc19e7e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fafcc00cce0 tx=0x7fafcc00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:33.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.990+0000 7fafd3fff640 1 -- 172.21.15.12:0/3723350426 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafcc00f450 con 0x7fafdc102d90 2024-02-18T11:40:33.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.990+0000 7fafd3fff640 1 -- 172.21.15.12:0/3723350426 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fafcc00fa90 con 0x7fafdc102d90 2024-02-18T11:40:33.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.990+0000 7fafd3fff640 1 -- 172.21.15.12:0/3723350426 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafcc018b90 con 0x7fafdc102d90 2024-02-18T11:40:33.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.990+0000 7fafe1593640 1 -- 172.21.15.12:0/3723350426 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafdc19f330 con 0x7fafdc102d90 2024-02-18T11:40:33.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.990+0000 7fafe1593640 1 -- 172.21.15.12:0/3723350426 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafdc19f840 con 0x7fafdc102d90 2024-02-18T11:40:33.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.991+0000 7fafe1593640 1 -- 172.21.15.12:0/3723350426 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafdc108500 con 0x7fafdc102d90 2024-02-18T11:40:33.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.992+0000 7fafd3fff640 1 -- 172.21.15.12:0/3723350426 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) v1 ==== 137245+0+0 (secure 0 0 0) 0x7fafcc00fc00 con 0x7fafdc102d90 2024-02-18T11:40:33.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.993+0000 7fafd3fff640 1 --2- 172.21.15.12:0/3723350426 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fafc80a27d0 0x7fafc80a4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:33.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.993+0000 7fafdaffd640 1 --2- 172.21.15.12:0/3723350426 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fafc80a27d0 0x7fafc80a4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:33.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.993+0000 7fafd3fff640 1 -- 172.21.15.12:0/3723350426 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fafcc0cecf0 con 0x7fafdc102d90 2024-02-18T11:40:33.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.993+0000 7fafdaffd640 1 --2- 172.21.15.12:0/3723350426 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fafc80a27d0 0x7fafc80a4c90 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fafd402c6a0 tx=0x7fafd4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:33.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:33.997+0000 7fafd3fff640 1 -- 172.21.15.12:0/3723350426 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fafcc0d3070 con 0x7fafdc102d90 2024-02-18T11:40:34.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.180+0000 7fafe1593640 1 -- 172.21.15.12:0/3723350426 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fafdc10cbb0 con 0x7fafdc102d90 2024-02-18T11:40:34.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.182+0000 7fafd3fff640 1 -- 172.21.15.12:0/3723350426 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+22662 (secure 0 0 0) 0x7fafcc097a90 con 0x7fafdc102d90 2024-02-18T11:40:34.176 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:40:34.176 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":89,"fsid":"0c8bb0de-ce51-11ee-95ba-87774f69a715","created":"2024-02-18T11:30:49.110059+0000","modified":"2024-02-18T11:39:52.478480+0000","last_up_change":"2024-02-18T11:37:57.512337+0000","last_in_change":"2024-02-18T11:37:36.567969+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-18T11:34:15.281138+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-18T11:38:44.802134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-18T11:38:46.081188+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"35f15ea6-deb8-48a1-a7dd-ab649ec13533","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":536522426},{"type":"v1","addr":"172.21.15.12:6803","nonce":536522426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":536522426},{"type":"v1","addr":"172.21.15.12:6805","nonce":536522426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":536522426},{"type":"v1","addr":"172.21.15.12:6809","nonce":536522426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":536522426},{"type":"v1","addr":"172.21.15.12:6807","nonce":536522426}]},"public_addr":"172.21.15.12:6803/536522426","cluster_addr":"172.21.15.12:6805/536522426","heartbeat_back_addr":"172.21.15.12:6809/536522426","heartbeat_front_addr":"172.21.15.12:6807/536522426","state":["exists","up"]},{"osd":1,"uuid":"2af937ef-d827-44e6-b805-70cffbfe1126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":4249111488},{"type":"v1","addr":"172.21.15.158:6801","nonce":4249111488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":4249111488},{"type":"v1","addr":"172.21.15.158:6803","nonce":4249111488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":4249111488},{"type":"v1","addr":"172.21.15.158:6807","nonce":4249111488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":4249111488},{"type":"v1","addr":"172.21.15.158:6805","nonce":4249111488}]},"public_addr":"172.21.15.158:6801/4249111488","cluster_addr":"172.21.15.158:6803/4249111488","heartbeat_back_addr":"172.21.15.158:6807/4249111488","heartbeat_front_addr":"172.21.15.158:6805/4249111488","state":["exists","up"]},{"osd":2,"uuid":"17b804dd-31df-45ae-a637-4a5f65bbaf26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":627976150},{"type":"v1","addr":"172.21.15.188:6801","nonce":627976150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":627976150},{"type":"v1","addr":"172.21.15.188:6803","nonce":627976150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":627976150},{"type":"v1","addr":"172.21.15.188:6807","nonce":627976150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":627976150},{"type":"v1","addr":"172.21.15.188:6805","nonce":627976150}]},"public_addr":"172.21.15.188:6801/627976150","cluster_addr":"172.21.15.188:6803/627976150","heartbeat_back_addr":"172.21.15.188:6807/627976150","heartbeat_front_addr":"172.21.15.188:6805/627976150","state":["exists","up"]},{"osd":3,"uuid":"3f9cc423-cdfa-4acf-aaa2-b58b32ccb9cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":3758512503},{"type":"v1","addr":"172.21.15.12:6811","nonce":3758512503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":3758512503},{"type":"v1","addr":"172.21.15.12:6813","nonce":3758512503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3758512503},{"type":"v1","addr":"172.21.15.12:6817","nonce":3758512503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":3758512503},{"type":"v1","addr":"172.21.15.12:6815","nonce":3758512503}]},"public_addr":"172.21.15.12:6811/3758512503","cluster_addr":"172.21.15.12:6813/3758512503","heartbeat_back_addr":"172.21.15.12:6817/3758512503","heartbeat_front_addr":"172.21.15.12:6815/3758512503","state":["exists","up"]},{"osd":4,"uuid":"a92e20ec-af1c-4ef4-a0cb-caba5fcc71aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":3892820997},{"type":"v1","addr":"172.21.15.158:6809","nonce":3892820997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":3892820997},{"type":"v1","addr":"172.21.15.158:6811","nonce":3892820997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":3892820997},{"type":"v1","addr":"172.21.15.158:6815","nonce":3892820997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":3892820997},{"type":"v1","addr":"172.21.15.158:6813","nonce":3892820997}]},"public_addr":"172.21.15.158:6809/3892820997","cluster_addr":"172.21.15.158:6811/3892820997","heartbeat_back_addr":"172.21.15.158:6815/3892820997","heartbeat_front_addr":"172.21.15.158:6813/3892820997","state":["exists","up"]},{"osd":5,"uuid":"d9c2e0c8-85f5-4ae2-bf3e-125c1be3a2db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3939495896},{"type":"v1","addr":"172.21.15.188:6809","nonce":3939495896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3939495896},{"type":"v1","addr":"172.21.15.188:6811","nonce":3939495896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3939495896},{"type":"v1","addr":"172.21.15.188:6815","nonce":3939495896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3939495896},{"type":"v1","addr":"172.21.15.188:6813","nonce":3939495896}]},"public_addr":"172.21.15.188:6809/3939495896","cluster_addr":"172.21.15.188:6811/3939495896","heartbeat_back_addr":"172.21.15.188:6815/3939495896","heartbeat_front_addr":"172.21.15.188:6813/3939495896","state":["exists","up"]},{"osd":6,"uuid":"4a523512-de6b-49f0-be81-00f2f00c1c10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":1206337767},{"type":"v1","addr":"172.21.15.12:6819","nonce":1206337767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":1206337767},{"type":"v1","addr":"172.21.15.12:6821","nonce":1206337767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":1206337767},{"type":"v1","addr":"172.21.15.12:6825","nonce":1206337767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":1206337767},{"type":"v1","addr":"172.21.15.12:6823","nonce":1206337767}]},"public_addr":"172.21.15.12:6819/1206337767","cluster_addr":"172.21.15.12:6821/1206337767","heartbeat_back_addr":"172.21.15.12:6825/1206337767","heartbeat_front_addr":"172.21.15.12:6823/1206337767","state":["exists","up"]},{"osd":7,"uuid":"363271a6-df93-44c9-b0e5-3d29768aa96c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":3283400032},{"type":"v1","addr":"172.21.15.158:6817","nonce":3283400032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":3283400032},{"type":"v1","addr":"172.21.15.158:6819","nonce":3283400032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":3283400032},{"type":"v1","addr":"172.21.15.158:6823","nonce":3283400032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":3283400032},{"type":"v1","addr":"172.21.15.158:6821","nonce":3283400032}]},"public_addr":"172.21.15.158:6817/3283400032","cluster_addr":"172.21.15.158:6819/3283400032","heartbeat_back_addr":"172.21.15.158:6823/3283400032","heartbeat_front_addr":"172.21.15.158:6821/3283400032","state":["exists","up"]},{"osd":8,"uuid":"9a089528-3e8e-45d3-9279-ddb54088b96a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2110170682},{"type":"v1","addr":"172.21.15.188:6817","nonce":2110170682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2110170682},{"type":"v1","addr":"172.21.15.188:6819","nonce":2110170682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":2110170682},{"type":"v1","addr":"172.21.15.188:6823","nonce":2110170682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2110170682},{"type":"v1","addr":"172.21.15.188:6821","nonce":2110170682}]},"public_addr":"172.21.15.188:6817/2110170682","cluster_addr":"172.21.15.188:6819/2110170682","heartbeat_back_addr":"172.21.15.188:6823/2110170682","heartbeat_front_addr":"172.21.15.188:6821/2110170682","state":["exists","up"]},{"osd":9,"uuid":"b12e2727-d649-4464-90e1-fafbc1754832","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1825928986},{"type":"v1","addr":"172.21.15.12:6827","nonce":1825928986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1825928986},{"type":"v1","addr":"172.21.15.12:6829","nonce":1825928986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":1825928986},{"type":"v1","addr":"172.21.15.12:6833","nonce":1825928986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1825928986},{"type":"v1","addr":"172.21.15.12:6831","nonce":1825928986}]},"public_addr":"172.21.15.12:6827/1825928986","cluster_addr":"172.21.15.12:6829/1825928986","heartbeat_back_addr":"172.21.15.12:6833/1825928986","heartbeat_front_addr":"172.21.15.12:6831/1825928986","state":["exists","up"]},{"osd":10,"uuid":"518b4cbf-9d90-4e9a-8ac9-b5cb9b3dee4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":3239052167},{"type":"v1","addr":"172.21.15.158:6825","nonce":3239052167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":3239052167},{"type":"v1","addr":"172.21.15.158:6827","nonce":3239052167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":3239052167},{"type":"v1","addr":"172.21.15.158:6831","nonce":3239052167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":3239052167},{"type":"v1","addr":"172.21.15.158:6829","nonce":3239052167}]},"public_addr":"172.21.15.158:6825/3239052167","cluster_addr":"172.21.15.158:6827/3239052167","heartbeat_back_addr":"172.21.15.158:6831/3239052167","heartbeat_front_addr":"172.21.15.158:6829/3239052167","state":["exists","up"]},{"osd":11,"uuid":"98aa3a0d-4b2f-46c8-9341-375d13794c3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1135773350},{"type":"v1","addr":"172.21.15.188:6825","nonce":1135773350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1135773350},{"type":"v1","addr":"172.21.15.188:6827","nonce":1135773350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1135773350},{"type":"v1","addr":"172.21.15.188:6831","nonce":1135773350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1135773350},{"type":"v1","addr":"172.21.15.188:6829","nonce":1135773350}]},"public_addr":"172.21.15.188:6825/1135773350","cluster_addr":"172.21.15.188:6827/1135773350","heartbeat_back_addr":"172.21.15.188:6831/1135773350","heartbeat_front_addr":"172.21.15.188:6829/1135773350","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:33:24.280879+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:33:47.859461+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:34:11.725283+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:34:35.707629+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:35:00.024197+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:35:24.499803+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:35:48.953870+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:36:13.642456+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:36:38.936457+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:37:04.021466+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:37:29.073178+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:37:54.863919+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.37","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:6801/4135274411":"2024-02-19T11:31:50.936928+0000","172.21.15.12:6800/4135274411":"2024-02-19T11:31:50.936928+0000","172.21.15.12:0/118428584":"2024-02-19T11:31:50.936928+0000","172.21.15.12:0/4276810915":"2024-02-19T11:31:50.936928+0000","172.21.15.12:0/3777545262":"2024-02-19T11:31:24.000830+0000","172.21.15.12:0/114707743":"2024-02-19T11:31:24.000830+0000","172.21.15.12:6800/3925204723":"2024-02-19T11:31:24.000830+0000","172.21.15.12:6801/3925204723":"2024-02-19T11:31:24.000830+0000","172.21.15.12:0/74881618":"2024-02-19T11:31:24.000830+0000","172.21.15.12:0/1614805375":"2024-02-19T11:31:50.936928+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-18T11:40:34.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.184+0000 7fafe1593640 1 -- 172.21.15.12:0/3723350426 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fafc80a27d0 msgr2=0x7fafc80a4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:34.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.184+0000 7fafe1593640 1 --2- 172.21.15.12:0/3723350426 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fafc80a27d0 0x7fafc80a4c90 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fafd402c6a0 tx=0x7fafd4000bc0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.184+0000 7fafe1593640 1 -- 172.21.15.12:0/3723350426 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fafdc102d90 msgr2=0x7fafdc19e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:34.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.184+0000 7fafe1593640 1 --2- 172.21.15.12:0/3723350426 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fafdc102d90 0x7fafdc19e7e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fafcc00cce0 tx=0x7fafcc00a620 comp rx=0 tx=0).stop 2024-02-18T11:40:34.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.184+0000 7fafe1593640 1 -- 172.21.15.12:0/3723350426 shutdown_connections 2024-02-18T11:40:34.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.184+0000 7fafe1593640 1 --2- 172.21.15.12:0/3723350426 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fafc80a27d0 0x7fafc80a4c90 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.184+0000 7fafe1593640 1 --2- 172.21.15.12:0/3723350426 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fafdc102d90 0x7fafdc19e7e0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.184+0000 7fafe1593640 1 --2- 172.21.15.12:0/3723350426 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fafdc1023d0 0x7fafdc197770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.184+0000 7fafe1593640 1 --2- 172.21.15.12:0/3723350426 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fafdc1011e0 0x7fafdc197230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.184+0000 7fafe1593640 1 -- 172.21.15.12:0/3723350426 >> 172.21.15.12:0/3723350426 conn(0x7fafdc0fc770 msgr2=0x7fafdc1056d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:34.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.184+0000 7fafe1593640 1 -- 172.21.15.12:0/3723350426 shutdown_connections 2024-02-18T11:40:34.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.184+0000 7fafe1593640 1 -- 172.21.15.12:0/3723350426 wait complete. 2024-02-18T11:40:34.190 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 20, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-02-18T11:38:46.888173+0000', 'modified': '2024-02-18T11:38:51.531552+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24524, 'mds_1': 24508, 'mds_2': 24478}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24446': {'gid': 24446, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.158:6835/3673253642', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.158:6834', 'nonce': 3673253642}, {'type': 'v1', 'addr': '172.21.15.158:6835', 'nonce': 3673253642}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24466': {'gid': 24466, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.188:6835/806709644', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6834', 'nonce': 806709644}, {'type': 'v1', 'addr': '172.21.15.188:6835', 'nonce': 806709644}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24476': {'gid': 24476, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.158:6837/813251825', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.158:6836', 'nonce': 813251825}, {'type': 'v1', 'addr': '172.21.15.158:6837', 'nonce': 813251825}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24478': {'gid': 24478, 'name': 'c', 'rank': 2, 'incarnation': 19, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.188:6837/1036630143', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6836', 'nonce': 1036630143}, {'type': 'v1', 'addr': '172.21.15.188:6837', 'nonce': 1036630143}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24508': {'gid': 24508, 'name': 'i', 'rank': 1, 'incarnation': 17, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.188:6839/1342671190', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6838', 'nonce': 1342671190}, {'type': 'v1', 'addr': '172.21.15.188:6839', 'nonce': 1342671190}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24524': {'gid': 24524, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.158:6839/1127200849', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.158:6838', 'nonce': 1127200849}, {'type': 'v1', 'addr': '172.21.15.158:6839', 'nonce': 1127200849}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-02-18T11:40:34.190 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-02-18T11:40:34.190 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-02-18T11:40:34.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.300+0000 7ffbe04d4640 1 -- 172.21.15.12:0/1632353210 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffbd81012c0 msgr2=0x7ffbd8101740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:34.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.300+0000 7ffbe04d4640 1 --2- 172.21.15.12:0/1632353210 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffbd81012c0 0x7ffbd8101740 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc009900 tx=0x7ffbcc02bc30 comp rx=0 tx=0).stop 2024-02-18T11:40:34.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.300+0000 7ffbe04d4640 1 -- 172.21.15.12:0/1632353210 shutdown_connections 2024-02-18T11:40:34.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.300+0000 7ffbe04d4640 1 --2- 172.21.15.12:0/1632353210 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffbd8101c80 0x7ffbd8108510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.300+0000 7ffbe04d4640 1 --2- 172.21.15.12:0/1632353210 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffbd81012c0 0x7ffbd8101740 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.300+0000 7ffbe04d4640 1 --2- 172.21.15.12:0/1632353210 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ffbd81000d0 0x7ffbd81004f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.300+0000 7ffbe04d4640 1 -- 172.21.15.12:0/1632353210 >> 172.21.15.12:0/1632353210 conn(0x7ffbd80fb660 msgr2=0x7ffbd80fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:34.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.300+0000 7ffbe04d4640 1 -- 172.21.15.12:0/1632353210 shutdown_connections 2024-02-18T11:40:34.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.300+0000 7ffbe04d4640 1 -- 172.21.15.12:0/1632353210 wait complete. 2024-02-18T11:40:34.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.300+0000 7ffbe04d4640 1 Processor -- start 2024-02-18T11:40:34.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.300+0000 7ffbe04d4640 1 -- start start 2024-02-18T11:40:34.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.300+0000 7ffbe04d4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffbd81000d0 0x7ffbd81961b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:34.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.300+0000 7ffbe04d4640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ffbd81012c0 0x7ffbd81966f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:34.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.301+0000 7ffbe04d4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffbd8101c80 0x7ffbd819d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:34.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.301+0000 7ffbe04d4640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd819dca0 con 0x7ffbd81000d0 2024-02-18T11:40:34.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.301+0000 7ffbe04d4640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd819de10 con 0x7ffbd81012c0 2024-02-18T11:40:34.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.301+0000 7ffbe04d4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd819dfd0 con 0x7ffbd8101c80 2024-02-18T11:40:34.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.301+0000 7ffbde270640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffbd81000d0 0x7ffbd81961b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:34.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.301+0000 7ffbde270640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffbd81000d0 0x7ffbd81961b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48712/0 (socket says 172.21.15.12:48712) 2024-02-18T11:40:34.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.301+0000 7ffbde270640 1 -- 172.21.15.12:0/1287786295 learned_addr learned my addr 172.21.15.12:0/1287786295 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:40:34.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.301+0000 7ffbdda6f640 1 --2- 172.21.15.12:0/1287786295 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ffbd81012c0 0x7ffbd81966f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:34.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.301+0000 7ffbde270640 1 -- 172.21.15.12:0/1287786295 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffbd8101c80 msgr2=0x7ffbd819d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:34.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.301+0000 7ffbde270640 1 --2- 172.21.15.12:0/1287786295 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffbd8101c80 0x7ffbd819d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.301+0000 7ffbde270640 1 -- 172.21.15.12:0/1287786295 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ffbd81012c0 msgr2=0x7ffbd81966f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:34.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.301+0000 7ffbde270640 1 --2- 172.21.15.12:0/1287786295 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ffbd81012c0 0x7ffbd81966f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.301+0000 7ffbde270640 1 -- 172.21.15.12:0/1287786295 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbcc009560 con 0x7ffbd81000d0 2024-02-18T11:40:34.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.301+0000 7ffbdda6f640 1 --2- 172.21.15.12:0/1287786295 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ffbd81012c0 0x7ffbd81966f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:40:34.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.301+0000 7ffbde270640 1 --2- 172.21.15.12:0/1287786295 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffbd81000d0 0x7ffbd81961b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ffbc800ccb0 tx=0x7ffbc800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:34.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.301+0000 7ffbd77fe640 1 -- 172.21.15.12:0/1287786295 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc8003990 con 0x7ffbd81000d0 2024-02-18T11:40:34.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.301+0000 7ffbd77fe640 1 -- 172.21.15.12:0/1287786295 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffbc8004020 con 0x7ffbd81000d0 2024-02-18T11:40:34.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.301+0000 7ffbe04d4640 1 -- 172.21.15.12:0/1287786295 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbd819e2b0 con 0x7ffbd81000d0 2024-02-18T11:40:34.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.301+0000 7ffbd77fe640 1 -- 172.21.15.12:0/1287786295 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc800f6b0 con 0x7ffbd81000d0 2024-02-18T11:40:34.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.301+0000 7ffbe04d4640 1 -- 172.21.15.12:0/1287786295 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbd819e810 con 0x7ffbd81000d0 2024-02-18T11:40:34.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.303+0000 7ffbd77fe640 1 -- 172.21.15.12:0/1287786295 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) v1 ==== 137245+0+0 (secure 0 0 0) 0x7ffbc8004190 con 0x7ffbd81000d0 2024-02-18T11:40:34.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.304+0000 7ffbe317e740 1 Processor -- start 2024-02-18T11:40:34.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.304+0000 7ffbe317e740 1 -- start start 2024-02-18T11:40:34.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.304+0000 7ffbd77fe640 1 --2- 172.21.15.12:0/1287786295 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7ffbc00a27d0 0x7ffbc00a4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:34.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.304+0000 7ffbe317e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5583a6f326f0 0x5583a6f30e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:34.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.304+0000 7ffbe317e740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583a6f31380 0x5583a6f31800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:34.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.304+0000 7ffbe317e740 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5583a6f3b350 0x5583a6f3d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:34.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.304+0000 7ffbe317e740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583a6eecdc0 con 0x5583a6f31380 2024-02-18T11:40:34.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.304+0000 7ffbe317e740 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583a6e9ff20 con 0x5583a6f3b350 2024-02-18T11:40:34.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.304+0000 7ffbe317e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583a6e9d450 con 0x5583a6f326f0 2024-02-18T11:40:34.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.304+0000 7ffbdea71640 1 --2- 172.21.15.12:0/1287786295 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7ffbc00a27d0 0x7ffbc00a4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:34.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.304+0000 7ffbdda6f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583a6f31380 0x5583a6f31800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:34.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.304+0000 7ffbd77fe640 1 -- 172.21.15.12:0/1287786295 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ffbc80ce630 con 0x7ffbd81000d0 2024-02-18T11:40:34.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.305+0000 7ffbdda6f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583a6f31380 0x5583a6f31800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48722/0 (socket says 172.21.15.12:48722) 2024-02-18T11:40:34.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.305+0000 7ffbdda6f640 1 -- 172.21.15.12:0/3281380331 learned_addr learned my addr 172.21.15.12:0/3281380331 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:40:34.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.305+0000 7ffbde270640 1 --2- 172.21.15.12:0/3281380331 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5583a6f326f0 0x5583a6f30e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:34.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.305+0000 7ffbdda6f640 1 -- 172.21.15.12:0/3281380331 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5583a6f326f0 msgr2=0x5583a6f30e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:40:34.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.305+0000 7ffbdda6f640 1 --2- 172.21.15.12:0/3281380331 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5583a6f326f0 0x5583a6f30e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.305+0000 7ffbdda6f640 1 -- 172.21.15.12:0/3281380331 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5583a6f3b350 msgr2=0x5583a6f3d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:40:34.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.305+0000 7ffbdda6f640 1 --2- 172.21.15.12:0/3281380331 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5583a6f3b350 0x5583a6f3d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.305+0000 7ffbdda6f640 1 -- 172.21.15.12:0/3281380331 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5583a6e4df20 con 0x5583a6f31380 2024-02-18T11:40:34.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.305+0000 7ffbde270640 1 --2- 172.21.15.12:0/3281380331 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5583a6f326f0 0x5583a6f30e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:40:34.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.305+0000 7ffbdea71640 1 --2- 172.21.15.12:0/1287786295 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7ffbc00a27d0 0x7ffbc00a4c90 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ffbd000a8d0 tx=0x7ffbd000acd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:34.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.305+0000 7ffbdda6f640 1 --2- 172.21.15.12:0/3281380331 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583a6f31380 0x5583a6f31800 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc036040 tx=0x7ffbcc02c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bbb2301070261db server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:34.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.305+0000 7ffbd4bfe640 1 -- 172.21.15.12:0/3281380331 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc0046c0 con 0x5583a6f31380 2024-02-18T11:40:34.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.305+0000 7ffbd4bfe640 1 -- 172.21.15.12:0/3281380331 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffbcc004510 con 0x5583a6f31380 2024-02-18T11:40:34.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.305+0000 7ffbd4bfe640 1 -- 172.21.15.12:0/3281380331 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc005610 con 0x5583a6f31380 2024-02-18T11:40:34.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.306+0000 7ffbe317e740 1 -- 172.21.15.12:0/3281380331 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583a6f31380 msgr2=0x5583a6f31800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:34.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.306+0000 7ffbe317e740 1 --2- 172.21.15.12:0/3281380331 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583a6f31380 0x5583a6f31800 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc036040 tx=0x7ffbcc02c960 comp rx=0 tx=0).stop 2024-02-18T11:40:34.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.306+0000 7ffbe317e740 1 -- 172.21.15.12:0/3281380331 shutdown_connections 2024-02-18T11:40:34.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.306+0000 7ffbe317e740 1 --2- 172.21.15.12:0/3281380331 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5583a6f3b350 0x5583a6f3d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.306+0000 7ffbe317e740 1 --2- 172.21.15.12:0/3281380331 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583a6f31380 0x5583a6f31800 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.306+0000 7ffbe317e740 1 --2- 172.21.15.12:0/3281380331 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5583a6f326f0 0x5583a6f30e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.306+0000 7ffbe317e740 1 -- 172.21.15.12:0/3281380331 >> 172.21.15.12:0/3281380331 conn(0x5583a6f2c5e0 msgr2=0x5583a6f2c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:34.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.306+0000 7ffbe317e740 1 -- 172.21.15.12:0/3281380331 shutdown_connections 2024-02-18T11:40:34.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.306+0000 7ffbe317e740 1 -- 172.21.15.12:0/3281380331 wait complete. 2024-02-18T11:40:34.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.306+0000 7ffbe317e740 1 -- start start 2024-02-18T11:40:34.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.306+0000 7ffbe317e740 1 Processor -- start 2024-02-18T11:40:34.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.307+0000 7ffbe317e740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583a6f31380 0x5583a6f3f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:34.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.307+0000 7ffbe317e740 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5583a6f326f0 0x5583a6f3fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:34.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.307+0000 7ffbe317e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5583a6f3b350 0x5583a6f46b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:34.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.307+0000 7ffbe317e740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583a6e98c30 con 0x5583a6f31380 2024-02-18T11:40:34.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.307+0000 7ffbe317e740 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583a6e3dd30 con 0x5583a6f326f0 2024-02-18T11:40:34.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.307+0000 7ffbe317e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583a6b6c310 con 0x5583a6f3b350 2024-02-18T11:40:34.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.307+0000 7ffbe317e740 10 client.0 _ll_register_callbacks cb 0x5583a6e45d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-18T11:40:34.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.307+0000 7ffbde270640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583a6f31380 0x5583a6f3f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:34.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.307+0000 7ffbde270640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583a6f31380 0x5583a6f3f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48730/0 (socket says 172.21.15.12:48730) 2024-02-18T11:40:34.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.307+0000 7ffbde270640 1 -- 172.21.15.12:0/4113762509 learned_addr learned my addr 172.21.15.12:0/4113762509 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:40:34.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.307+0000 7ffbde270640 1 -- 172.21.15.12:0/4113762509 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5583a6f3b350 msgr2=0x5583a6f46b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:40:34.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.307+0000 7ffbde270640 1 --2- 172.21.15.12:0/4113762509 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5583a6f3b350 0x5583a6f46b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.307+0000 7ffbde270640 1 -- 172.21.15.12:0/4113762509 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5583a6f326f0 msgr2=0x5583a6f3fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:40:34.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.307+0000 7ffbde270640 1 --2- 172.21.15.12:0/4113762509 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5583a6f326f0 0x5583a6f3fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.307+0000 7ffbde270640 1 -- 172.21.15.12:0/4113762509 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbd8108e10 con 0x5583a6f31380 2024-02-18T11:40:34.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.307+0000 7ffbde270640 1 --2- 172.21.15.12:0/4113762509 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583a6f31380 0x5583a6f3f5a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ffbc8097aa0 tx=0x7ffbc8012a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d967a38ee0cdc41 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:34.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.307+0000 7ffbbdffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-02-18T11:40:34.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.307+0000 7ffbbdffb640 1 -- 172.21.15.12:0/4113762509 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc80a3870 con 0x5583a6f31380 2024-02-18T11:40:34.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.307+0000 7ffbbdffb640 1 -- 172.21.15.12:0/4113762509 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ffbc8004500 con 0x5583a6f31380 2024-02-18T11:40:34.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.307+0000 7ffbe317e740 1 -- 172.21.15.12:0/4113762509 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5583a6b986c0 con 0x5583a6f31380 2024-02-18T11:40:34.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.307+0000 7ffbbdffb640 1 -- 172.21.15.12:0/4113762509 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffbc80046c0 con 0x5583a6f31380 2024-02-18T11:40:34.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.307+0000 7ffbe317e740 10 client.15198 fetch_fsmap learned FSMap version 20 2024-02-18T11:40:34.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.307+0000 7ffbe317e740 1 -- 172.21.15.12:0/4113762509 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x5583a6e972b0 con 0x5583a6f31380 2024-02-18T11:40:34.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.308+0000 7ffbbdffb640 1 -- 172.21.15.12:0/4113762509 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffbc807d020 con 0x5583a6f31380 2024-02-18T11:40:34.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.308+0000 7ffbbdffb640 1 -- 172.21.15.12:0/4113762509 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ffbc8016030 con 0x5583a6f31380 2024-02-18T11:40:34.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.308+0000 7ffbe317e740 10 client.15198 fetch_fsmap finished waiting for FSMap version 20 2024-02-18T11:40:34.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.308+0000 7ffbe317e740 10 client.15198 resolve_mds: resolved name 'c' to daemon mds.c 2024-02-18T11:40:34.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.308+0000 7ffbe317e740 20 client.15198 populate_metadata read hostname 'smithi012' 2024-02-18T11:40:34.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.308+0000 7ffbe317e740 1 --2- 172.21.15.12:0/4113762509 >> [v2:172.21.15.188:6836/1036630143,v1:172.21.15.188:6837/1036630143] conn(0x5583a6f4c030 0x5583a6f4b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:34.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.308+0000 7ffbe317e740 4 client.15198 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-18T11:40:34.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.308+0000 7ffbe317e740 1 -- 172.21.15.12:0/4113762509 --> [v2:172.21.15.188:6836/1036630143,v1:172.21.15.188:6837/1036630143] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5583a6be9270 con 0x5583a6f4c030 2024-02-18T11:40:34.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.308+0000 7ffbdda6f640 1 --2- 172.21.15.12:0/4113762509 >> [v2:172.21.15.188:6836/1036630143,v1:172.21.15.188:6837/1036630143] conn(0x5583a6f4c030 0x5583a6f4b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:34.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.309+0000 7ffbbdffb640 10 client.15198 ms_handle_connect on v2:172.21.15.188:6836/1036630143 2024-02-18T11:40:34.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.309+0000 7ffbdda6f640 1 --2- 172.21.15.12:0/4113762509 >> [v2:172.21.15.188:6836/1036630143,v1:172.21.15.188:6837/1036630143] conn(0x5583a6f4c030 0x5583a6f4b200 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=10b615280175e85b server_cookie=66f19005de8823ed in_seq=0 out_seq=0 2024-02-18T11:40:34.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.309+0000 7ffbbdffb640 1 -- 172.21.15.12:0/4113762509 <== mds.2 v2:172.21.15.188:6836/1036630143 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5583a6be9270 con 0x5583a6f4c030 2024-02-18T11:40:34.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.309+0000 7ffbbdffb640 10 client.15198 handle_command_reply: tid=0 2024-02-18T11:40:34.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.309+0000 7ffbe317e740 1 client.15198 shutdown 2024-02-18T11:40:34.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.309+0000 7ffbe317e740 1 -- 172.21.15.12:0/4113762509 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583a6f31380 msgr2=0x5583a6f3f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:34.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.309+0000 7ffbe317e740 1 --2- 172.21.15.12:0/4113762509 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583a6f31380 0x5583a6f3f5a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ffbc8097aa0 tx=0x7ffbc8012a00 comp rx=0 tx=0).stop 2024-02-18T11:40:34.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.309+0000 7ffbe317e740 1 -- 172.21.15.12:0/4113762509 shutdown_connections 2024-02-18T11:40:34.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.309+0000 7ffbe317e740 1 --2- 172.21.15.12:0/4113762509 >> [v2:172.21.15.188:6836/1036630143,v1:172.21.15.188:6837/1036630143] conn(0x5583a6f4c030 0x5583a6f4b200 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.309+0000 7ffbe317e740 1 --2- 172.21.15.12:0/4113762509 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5583a6f3b350 0x5583a6f46b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.309+0000 7ffbe317e740 1 --2- 172.21.15.12:0/4113762509 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5583a6f326f0 0x5583a6f3fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.309+0000 7ffbbdffb640 0 client.15198 ms_handle_reset on v2:172.21.15.188:6836/1036630143 2024-02-18T11:40:34.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.309+0000 7ffbe317e740 1 --2- 172.21.15.12:0/4113762509 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583a6f31380 0x5583a6f3f5a0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.309+0000 7ffbe317e740 1 -- 172.21.15.12:0/4113762509 >> 172.21.15.12:0/4113762509 conn(0x5583a6f2c5e0 msgr2=0x5583a6f2b8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:34.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.309+0000 7ffbe317e740 1 -- 172.21.15.12:0/4113762509 shutdown_connections 2024-02-18T11:40:34.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.309+0000 7ffbe317e740 1 -- 172.21.15.12:0/4113762509 wait complete. 2024-02-18T11:40:34.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.310+0000 7ffbe317e740 20 client.15198 trim_cache size 0 max 16384 2024-02-18T11:40:34.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.325+0000 7ffbe317e740 1 Processor -- start 2024-02-18T11:40:34.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.325+0000 7ffbe317e740 1 -- start start 2024-02-18T11:40:34.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.325+0000 7ffbe317e740 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5583a6f2c5e0 0x5583a6f3ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:34.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.325+0000 7ffbe317e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5583a6f31380 0x5583a6f46ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:34.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.325+0000 7ffbe317e740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583a6f326f0 0x5583a6f37750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:34.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.325+0000 7ffbe317e740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583a6e1e490 con 0x5583a6f326f0 2024-02-18T11:40:34.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.325+0000 7ffbe317e740 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583a6f4e010 con 0x5583a6f2c5e0 2024-02-18T11:40:34.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.325+0000 7ffbe317e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583a6f3b1f0 con 0x5583a6f31380 2024-02-18T11:40:34.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.325+0000 7ffbdea71640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583a6f326f0 0x5583a6f37750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:34.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.325+0000 7ffbdea71640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583a6f326f0 0x5583a6f37750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48738/0 (socket says 172.21.15.12:48738) 2024-02-18T11:40:34.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.325+0000 7ffbdea71640 1 -- 172.21.15.12:0/52338454 learned_addr learned my addr 172.21.15.12:0/52338454 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:40:34.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.326+0000 7ffbde270640 1 --2- 172.21.15.12:0/52338454 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5583a6f2c5e0 0x5583a6f3ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:34.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.326+0000 7ffbdea71640 1 -- 172.21.15.12:0/52338454 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5583a6f31380 msgr2=0x5583a6f46ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:40:34.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.326+0000 7ffbdea71640 1 --2- 172.21.15.12:0/52338454 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5583a6f31380 0x5583a6f46ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.326+0000 7ffbdea71640 1 -- 172.21.15.12:0/52338454 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5583a6f2c5e0 msgr2=0x5583a6f3ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:40:34.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.326+0000 7ffbdea71640 1 --2- 172.21.15.12:0/52338454 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5583a6f2c5e0 0x5583a6f3ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.326+0000 7ffbdea71640 1 -- 172.21.15.12:0/52338454 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5583a6e33a20 con 0x5583a6f326f0 2024-02-18T11:40:34.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.326+0000 7ffbde270640 1 --2- 172.21.15.12:0/52338454 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5583a6f2c5e0 0x5583a6f3ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:40:34.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.326+0000 7ffbdea71640 1 --2- 172.21.15.12:0/52338454 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583a6f326f0 0x5583a6f37750 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ffbd000cc80 tx=0x7ffbd00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc7f1e64094fca0a server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:34.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.326+0000 7ffbbffff640 1 -- 172.21.15.12:0/52338454 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbd0022070 con 0x5583a6f326f0 2024-02-18T11:40:34.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.326+0000 7ffbbffff640 1 -- 172.21.15.12:0/52338454 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffbd0005d70 con 0x5583a6f326f0 2024-02-18T11:40:34.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.326+0000 7ffbbffff640 1 -- 172.21.15.12:0/52338454 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbd0003b20 con 0x5583a6f326f0 2024-02-18T11:40:34.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.327+0000 7ffbe317e740 1 -- 172.21.15.12:0/52338454 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583a6f326f0 msgr2=0x5583a6f37750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:34.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.327+0000 7ffbe317e740 1 --2- 172.21.15.12:0/52338454 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583a6f326f0 0x5583a6f37750 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ffbd000cc80 tx=0x7ffbd00046c0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.327+0000 7ffbe317e740 1 -- 172.21.15.12:0/52338454 shutdown_connections 2024-02-18T11:40:34.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.327+0000 7ffbe317e740 1 --2- 172.21.15.12:0/52338454 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5583a6f31380 0x5583a6f46ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.327+0000 7ffbe317e740 1 --2- 172.21.15.12:0/52338454 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583a6f326f0 0x5583a6f37750 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.327+0000 7ffbe317e740 1 --2- 172.21.15.12:0/52338454 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5583a6f2c5e0 0x5583a6f3ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.327+0000 7ffbe317e740 1 -- 172.21.15.12:0/52338454 >> 172.21.15.12:0/52338454 conn(0x5583a6f30fa0 msgr2=0x5583a6f544d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:34.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.327+0000 7ffbe317e740 1 -- 172.21.15.12:0/52338454 shutdown_connections 2024-02-18T11:40:34.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.327+0000 7ffbe317e740 1 -- 172.21.15.12:0/52338454 wait complete. 2024-02-18T11:40:34.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.327+0000 7ffbe317e740 1 -- start start 2024-02-18T11:40:34.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.327+0000 7ffbe317e740 1 Processor -- start 2024-02-18T11:40:34.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.327+0000 7ffbe317e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5583a6f2c5e0 0x5583a6f43b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:34.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.327+0000 7ffbe317e740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583a6f31380 0x5583a6f440d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:34.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.327+0000 7ffbe317e740 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5583a6f326f0 0x5583a6f450e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:34.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.327+0000 7ffbe317e740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583a6f45620 con 0x5583a6f31380 2024-02-18T11:40:34.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.327+0000 7ffbe317e740 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583a6f45790 con 0x5583a6f326f0 2024-02-18T11:40:34.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.327+0000 7ffbe317e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583a6f45950 con 0x5583a6f2c5e0 2024-02-18T11:40:34.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.327+0000 7ffbe317e740 10 client.0 _ll_register_callbacks cb 0x7ffbcc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-18T11:40:34.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.327+0000 7ffbdda6f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583a6f31380 0x5583a6f440d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:34.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.327+0000 7ffbdda6f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583a6f31380 0x5583a6f440d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48740/0 (socket says 172.21.15.12:48740) 2024-02-18T11:40:34.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.327+0000 7ffbdda6f640 1 -- 172.21.15.12:0/3955249780 learned_addr learned my addr 172.21.15.12:0/3955249780 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:40:34.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.327+0000 7ffbde270640 1 --2- 172.21.15.12:0/3955249780 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5583a6f2c5e0 0x5583a6f43b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:34.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.327+0000 7ffbdda6f640 1 -- 172.21.15.12:0/3955249780 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5583a6f2c5e0 msgr2=0x5583a6f43b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:40:34.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.327+0000 7ffbdda6f640 1 --2- 172.21.15.12:0/3955249780 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5583a6f2c5e0 0x5583a6f43b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.328+0000 7ffbdda6f640 1 -- 172.21.15.12:0/3955249780 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5583a6f326f0 msgr2=0x5583a6f450e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:40:34.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.328+0000 7ffbdea71640 1 --2- 172.21.15.12:0/3955249780 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5583a6f326f0 0x5583a6f450e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:34.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.328+0000 7ffbdda6f640 1 --2- 172.21.15.12:0/3955249780 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5583a6f326f0 0x5583a6f450e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.328+0000 7ffbdda6f640 1 -- 172.21.15.12:0/3955249780 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbd000c950 con 0x5583a6f31380 2024-02-18T11:40:34.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.328+0000 7ffbde270640 1 --2- 172.21.15.12:0/3955249780 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5583a6f2c5e0 0x5583a6f43b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:40:34.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.328+0000 7ffbdda6f640 1 --2- 172.21.15.12:0/3955249780 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583a6f31380 0x5583a6f440d0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x5583a6f2d550 tx=0x7ffbcc0058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1029b0b48179c3e8 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:34.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.328+0000 7ffbbdffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-02-18T11:40:34.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.328+0000 7ffbbdffb640 1 -- 172.21.15.12:0/3955249780 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc0046c0 con 0x5583a6f31380 2024-02-18T11:40:34.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.328+0000 7ffbbdffb640 1 -- 172.21.15.12:0/3955249780 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ffbcc04b040 con 0x5583a6f31380 2024-02-18T11:40:34.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.328+0000 7ffbe317e740 1 -- 172.21.15.12:0/3955249780 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffbcc004510 con 0x5583a6f31380 2024-02-18T11:40:34.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.328+0000 7ffbbdffb640 1 -- 172.21.15.12:0/3955249780 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffbcc004820 con 0x5583a6f31380 2024-02-18T11:40:34.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.328+0000 7ffbe317e740 10 client.15204 fetch_fsmap learned FSMap version 20 2024-02-18T11:40:34.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.328+0000 7ffbe317e740 1 -- 172.21.15.12:0/3955249780 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x5583a6e59780 con 0x5583a6f31380 2024-02-18T11:40:34.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.328+0000 7ffbbdffb640 1 -- 172.21.15.12:0/3955249780 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffbcc004510 con 0x5583a6f31380 2024-02-18T11:40:34.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.328+0000 7ffbbdffb640 1 -- 172.21.15.12:0/3955249780 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ffbcc03c040 con 0x5583a6f31380 2024-02-18T11:40:34.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.328+0000 7ffbe317e740 10 client.15204 fetch_fsmap finished waiting for FSMap version 20 2024-02-18T11:40:34.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.328+0000 7ffbe317e740 10 client.15204 resolve_mds: resolved name 'c' to daemon mds.c 2024-02-18T11:40:34.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.328+0000 7ffbe317e740 20 client.15204 populate_metadata read hostname 'smithi012' 2024-02-18T11:40:34.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.328+0000 7ffbe317e740 1 --2- 172.21.15.12:0/3955249780 >> [v2:172.21.15.188:6836/1036630143,v1:172.21.15.188:6837/1036630143] conn(0x5583a6f3b350 0x5583a6fa3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:34.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.328+0000 7ffbe317e740 4 client.15204 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "status"}] 2024-02-18T11:40:34.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.328+0000 7ffbe317e740 1 -- 172.21.15.12:0/3955249780 --> [v2:172.21.15.188:6836/1036630143,v1:172.21.15.188:6837/1036630143] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5583a6bd5610 con 0x5583a6f3b350 2024-02-18T11:40:34.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.329+0000 7ffbde270640 1 --2- 172.21.15.12:0/3955249780 >> [v2:172.21.15.188:6836/1036630143,v1:172.21.15.188:6837/1036630143] conn(0x5583a6f3b350 0x5583a6fa3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:34.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.329+0000 7ffbbdffb640 10 client.15204 ms_handle_connect on v2:172.21.15.188:6836/1036630143 2024-02-18T11:40:34.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.329+0000 7ffbde270640 1 --2- 172.21.15.12:0/3955249780 >> [v2:172.21.15.188:6836/1036630143,v1:172.21.15.188:6837/1036630143] conn(0x5583a6f3b350 0x5583a6fa3c40 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dae9cb44ea9c81fc server_cookie=53095b2aca5cb734 in_seq=0 out_seq=0 2024-02-18T11:40:34.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.329+0000 7ffbbdffb640 1 -- 172.21.15.12:0/3955249780 <== mds.2 v2:172.21.15.188:6836/1036630143 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x5583a6bd5610 con 0x5583a6f3b350 2024-02-18T11:40:34.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.329+0000 7ffbbdffb640 10 client.15204 handle_command_reply: tid=0 2024-02-18T11:40:34.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.329+0000 7ffbe317e740 1 client.15204 shutdown 2024-02-18T11:40:34.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.329+0000 7ffbe317e740 1 -- 172.21.15.12:0/3955249780 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583a6f31380 msgr2=0x5583a6f440d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:34.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.329+0000 7ffbe317e740 1 --2- 172.21.15.12:0/3955249780 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583a6f31380 0x5583a6f440d0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x5583a6f2d550 tx=0x7ffbcc0058d0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.330+0000 7ffbe317e740 1 -- 172.21.15.12:0/3955249780 shutdown_connections 2024-02-18T11:40:34.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.330+0000 7ffbe317e740 1 --2- 172.21.15.12:0/3955249780 >> [v2:172.21.15.188:6836/1036630143,v1:172.21.15.188:6837/1036630143] conn(0x5583a6f3b350 0x5583a6fa3c40 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.330+0000 7ffbe317e740 1 --2- 172.21.15.12:0/3955249780 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5583a6f326f0 0x5583a6f450e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.330+0000 7ffbe317e740 1 --2- 172.21.15.12:0/3955249780 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583a6f31380 0x5583a6f440d0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.330+0000 7ffbbdffb640 0 client.15204 ms_handle_reset on v2:172.21.15.188:6836/1036630143 2024-02-18T11:40:34.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.330+0000 7ffbe317e740 1 --2- 172.21.15.12:0/3955249780 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5583a6f2c5e0 0x5583a6f43b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.330+0000 7ffbe317e740 1 -- 172.21.15.12:0/3955249780 >> 172.21.15.12:0/3955249780 conn(0x5583a6f30fa0 msgr2=0x5583a6f37750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:34.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.330+0000 7ffbe317e740 1 -- 172.21.15.12:0/3955249780 shutdown_connections 2024-02-18T11:40:34.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.330+0000 7ffbe317e740 1 -- 172.21.15.12:0/3955249780 wait complete. 2024-02-18T11:40:34.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.330+0000 7ffbe317e740 20 client.15204 trim_cache size 0 max 16384 2024-02-18T11:40:34.331 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-02-18T11:40:34.331 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_fsid": "0c8bb0de-ce51-11ee-95ba-87774f69a715", 2024-02-18T11:40:34.331 INFO:teuthology.orchestra.run.smithi012.stdout: "whoami": 2, 2024-02-18T11:40:34.331 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 24478, 2024-02-18T11:40:34.331 INFO:teuthology.orchestra.run.smithi012.stdout: "want_state": "up:active", 2024-02-18T11:40:34.331 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "up:active", 2024-02-18T11:40:34.331 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs", 2024-02-18T11:40:34.331 INFO:teuthology.orchestra.run.smithi012.stdout: "rank_uptime": 103.47782814, 2024-02-18T11:40:34.332 INFO:teuthology.orchestra.run.smithi012.stdout: "mdsmap_epoch": 20, 2024-02-18T11:40:34.332 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch": 89, 2024-02-18T11:40:34.332 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch_barrier": 87, 2024-02-18T11:40:34.332 INFO:teuthology.orchestra.run.smithi012.stdout: "uptime": 126.39004110099999 2024-02-18T11:40:34.332 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-02-18T11:40:34.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.330+0000 7ffbe04d4640 1 -- 172.21.15.12:0/1287786295 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7ffbc00a27d0 msgr2=0x7ffbc00a4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:34.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.330+0000 7ffbe04d4640 1 --2- 172.21.15.12:0/1287786295 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7ffbc00a27d0 0x7ffbc00a4c90 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ffbd000a8d0 tx=0x7ffbd000acd0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.330+0000 7ffbe04d4640 1 -- 172.21.15.12:0/1287786295 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffbd81000d0 msgr2=0x7ffbd81961b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:34.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.330+0000 7ffbe04d4640 1 --2- 172.21.15.12:0/1287786295 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffbd81000d0 0x7ffbd81961b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ffbc800ccb0 tx=0x7ffbc800a620 comp rx=0 tx=0).stop 2024-02-18T11:40:34.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.330+0000 7ffbe04d4640 1 -- 172.21.15.12:0/1287786295 shutdown_connections 2024-02-18T11:40:34.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.330+0000 7ffbe04d4640 1 --2- 172.21.15.12:0/1287786295 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7ffbc00a27d0 0x7ffbc00a4c90 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.330+0000 7ffbe04d4640 1 --2- 172.21.15.12:0/1287786295 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffbd8101c80 0x7ffbd819d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.330+0000 7ffbe04d4640 1 --2- 172.21.15.12:0/1287786295 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ffbd81012c0 0x7ffbd81966f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.330+0000 7ffbe04d4640 1 --2- 172.21.15.12:0/1287786295 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffbd81000d0 0x7ffbd81961b0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.330+0000 7ffbe04d4640 1 -- 172.21.15.12:0/1287786295 >> 172.21.15.12:0/1287786295 conn(0x7ffbd80fb660 msgr2=0x7ffbd8104690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:34.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.330+0000 7ffbe04d4640 1 -- 172.21.15.12:0/1287786295 shutdown_connections 2024-02-18T11:40:34.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.330+0000 7ffbe04d4640 1 -- 172.21.15.12:0/1287786295 wait complete. 2024-02-18T11:40:34.335 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-02-18T11:40:34.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.444+0000 7f77d4956640 1 -- 172.21.15.12:0/849329649 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f77d0101c90 msgr2=0x7f77d0108520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:34.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.444+0000 7f77d4956640 1 --2- 172.21.15.12:0/849329649 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f77d0101c90 0x7f77d0108520 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f77c8009ca0 tx=0x7f77c802c0f0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.444+0000 7f77d4956640 1 -- 172.21.15.12:0/849329649 shutdown_connections 2024-02-18T11:40:34.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.444+0000 7f77d4956640 1 --2- 172.21.15.12:0/849329649 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f77d0101c90 0x7f77d0108520 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.444+0000 7f77d4956640 1 --2- 172.21.15.12:0/849329649 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f77d01012d0 0x7f77d0101750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.444+0000 7f77d4956640 1 --2- 172.21.15.12:0/849329649 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f77d01000e0 0x7f77d0100500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.444+0000 7f77d4956640 1 -- 172.21.15.12:0/849329649 >> 172.21.15.12:0/849329649 conn(0x7f77d00fb670 msgr2=0x7f77d00fda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:34.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.444+0000 7f77d4956640 1 -- 172.21.15.12:0/849329649 shutdown_connections 2024-02-18T11:40:34.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.444+0000 7f77d4956640 1 -- 172.21.15.12:0/849329649 wait complete. 2024-02-18T11:40:34.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.445+0000 7f77d4956640 1 Processor -- start 2024-02-18T11:40:34.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.445+0000 7f77d4956640 1 -- start start 2024-02-18T11:40:34.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.445+0000 7f77d4956640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f77d01000e0 0x7f77d006a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:34.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.445+0000 7f77d4956640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f77d01012d0 0x7f77d019bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:34.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.445+0000 7f77d4956640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f77d0101c90 0x7f77d019e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:34.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.445+0000 7f77d4956640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d019ed90 con 0x7f77d01012d0 2024-02-18T11:40:34.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.445+0000 7f77d4956640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d019ef00 con 0x7f77d01000e0 2024-02-18T11:40:34.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.445+0000 7f77d4956640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d019f0c0 con 0x7f77d0101c90 2024-02-18T11:40:34.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.445+0000 7f77c7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f77d01012d0 0x7f77d019bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:34.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.445+0000 7f77c7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f77d01012d0 0x7f77d019bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48758/0 (socket says 172.21.15.12:48758) 2024-02-18T11:40:34.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.445+0000 7f77c7fff640 1 -- 172.21.15.12:0/3229740185 learned_addr learned my addr 172.21.15.12:0/3229740185 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:40:34.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.445+0000 7f77ced9d640 1 --2- 172.21.15.12:0/3229740185 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f77d0101c90 0x7f77d019e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:34.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.445+0000 7f77ce59c640 1 --2- 172.21.15.12:0/3229740185 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f77d01000e0 0x7f77d006a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:34.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.445+0000 7f77c7fff640 1 -- 172.21.15.12:0/3229740185 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f77d0101c90 msgr2=0x7f77d019e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:34.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.445+0000 7f77c7fff640 1 --2- 172.21.15.12:0/3229740185 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f77d0101c90 0x7f77d019e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.445+0000 7f77c7fff640 1 -- 172.21.15.12:0/3229740185 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f77d01000e0 msgr2=0x7f77d006a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:34.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.445+0000 7f77c7fff640 1 --2- 172.21.15.12:0/3229740185 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f77d01000e0 0x7f77d006a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.445+0000 7f77c7fff640 1 -- 172.21.15.12:0/3229740185 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77c8009900 con 0x7f77d01012d0 2024-02-18T11:40:34.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.445+0000 7f77ce59c640 1 --2- 172.21.15.12:0/3229740185 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f77d01000e0 0x7f77d006a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:40:34.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.446+0000 7f77c7fff640 1 --2- 172.21.15.12:0/3229740185 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f77d01012d0 0x7f77d019bf90 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f77b800d950 tx=0x7f77b800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:34.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.446+0000 7f77c77fe640 1 -- 172.21.15.12:0/3229740185 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b8009c30 con 0x7f77d01012d0 2024-02-18T11:40:34.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.446+0000 7f77d4956640 1 -- 172.21.15.12:0/3229740185 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77d019f340 con 0x7f77d01012d0 2024-02-18T11:40:34.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.446+0000 7f77c77fe640 1 -- 172.21.15.12:0/3229740185 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f77b8009d90 con 0x7f77d01012d0 2024-02-18T11:40:34.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.446+0000 7f77c77fe640 1 -- 172.21.15.12:0/3229740185 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b800f7e0 con 0x7f77d01012d0 2024-02-18T11:40:34.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.446+0000 7f77d4956640 1 -- 172.21.15.12:0/3229740185 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77d019f8a0 con 0x7f77d01012d0 2024-02-18T11:40:34.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.448+0000 7f77c77fe640 1 -- 172.21.15.12:0/3229740185 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) v1 ==== 137245+0+0 (secure 0 0 0) 0x7f77b8010ad0 con 0x7f77d01012d0 2024-02-18T11:40:34.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.448+0000 7f77d7608740 1 Processor -- start 2024-02-18T11:40:34.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.448+0000 7f77d7608740 1 -- start start 2024-02-18T11:40:34.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.448+0000 7f77d7608740 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x559081332070 0x559081332490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:34.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.448+0000 7f77d7608740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559081332a60 0x559081337560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:34.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.448+0000 7f77d7608740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559081337bd0 0x55908133a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:34.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.448+0000 7f77d7608740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590812eddc0 con 0x559081332a60 2024-02-18T11:40:34.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.448+0000 7f77d7608740 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590812a0f20 con 0x559081332070 2024-02-18T11:40:34.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.448+0000 7f77d7608740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55908129e450 con 0x559081337bd0 2024-02-18T11:40:34.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.448+0000 7f77c77fe640 1 --2- 172.21.15.12:0/3229740185 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f77bc0a2750 0x7f77bc0a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:34.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.448+0000 7f77ce59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559081332a60 0x559081337560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:34.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.449+0000 7f77ce59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559081332a60 0x559081337560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48768/0 (socket says 172.21.15.12:48768) 2024-02-18T11:40:34.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.449+0000 7f77ce59c640 1 -- 172.21.15.12:0/2537223902 learned_addr learned my addr 172.21.15.12:0/2537223902 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:40:34.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.449+0000 7f77ced9d640 1 --2- 172.21.15.12:0/3229740185 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f77bc0a2750 0x7f77bc0a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:34.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.449+0000 7f77c7fff640 1 --2- 172.21.15.12:0/2537223902 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559081337bd0 0x55908133a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:34.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.449+0000 7f77ced9d640 1 --2- 172.21.15.12:0/2537223902 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x559081332070 0x559081332490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:34.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.449+0000 7f77c77fe640 1 -- 172.21.15.12:0/3229740185 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f77b80ce1d0 con 0x7f77d01012d0 2024-02-18T11:40:34.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.449+0000 7f77ce59c640 1 -- 172.21.15.12:0/2537223902 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559081337bd0 msgr2=0x55908133a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:40:34.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.449+0000 7f77ce59c640 1 --2- 172.21.15.12:0/2537223902 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559081337bd0 0x55908133a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.449+0000 7f77ce59c640 1 -- 172.21.15.12:0/2537223902 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x559081332070 msgr2=0x559081332490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:40:34.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.449+0000 7f77ce59c640 1 --2- 172.21.15.12:0/2537223902 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x559081332070 0x559081332490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.449+0000 7f77ce59c640 1 -- 172.21.15.12:0/2537223902 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55908124ef20 con 0x559081332a60 2024-02-18T11:40:34.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.449+0000 7f77ced9d640 1 --2- 172.21.15.12:0/2537223902 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x559081332070 0x559081332490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:40:34.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.449+0000 7f77ce59c640 1 --2- 172.21.15.12:0/2537223902 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559081332a60 0x559081337560 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f77c000bd30 tx=0x7f77c000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f23b254a4931b1a4 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:34.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.449+0000 7f77ced9d640 1 --2- 172.21.15.12:0/3229740185 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f77bc0a2750 0x7f77bc0a4c10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f77c8005e30 tx=0x7f77c8005ef0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:34.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.449+0000 7f77c4ff9640 1 -- 172.21.15.12:0/2537223902 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c001a660 con 0x559081332a60 2024-02-18T11:40:34.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.449+0000 7f77c4ff9640 1 -- 172.21.15.12:0/2537223902 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f77c000be20 con 0x559081332a60 2024-02-18T11:40:34.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.449+0000 7f77c4ff9640 1 -- 172.21.15.12:0/2537223902 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c0011b50 con 0x559081332a60 2024-02-18T11:40:34.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.450+0000 7f77d7608740 1 -- 172.21.15.12:0/2537223902 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559081332a60 msgr2=0x559081337560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:34.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.450+0000 7f77d7608740 1 --2- 172.21.15.12:0/2537223902 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559081332a60 0x559081337560 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f77c000bd30 tx=0x7f77c000f730 comp rx=0 tx=0).stop 2024-02-18T11:40:34.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.450+0000 7f77d7608740 1 -- 172.21.15.12:0/2537223902 shutdown_connections 2024-02-18T11:40:34.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.450+0000 7f77d7608740 1 --2- 172.21.15.12:0/2537223902 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559081337bd0 0x55908133a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.450+0000 7f77d7608740 1 --2- 172.21.15.12:0/2537223902 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559081332a60 0x559081337560 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.450+0000 7f77d7608740 1 --2- 172.21.15.12:0/2537223902 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x559081332070 0x559081332490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.450+0000 7f77d7608740 1 -- 172.21.15.12:0/2537223902 >> 172.21.15.12:0/2537223902 conn(0x55908132d5e0 msgr2=0x55908132d9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:34.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.450+0000 7f77d7608740 1 -- 172.21.15.12:0/2537223902 shutdown_connections 2024-02-18T11:40:34.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.450+0000 7f77d7608740 1 -- 172.21.15.12:0/2537223902 wait complete. 2024-02-18T11:40:34.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.450+0000 7f77d7608740 1 -- start start 2024-02-18T11:40:34.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.450+0000 7f77d7608740 1 Processor -- start 2024-02-18T11:40:34.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.451+0000 7f77d7608740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559081332070 0x55908133c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:34.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.451+0000 7f77d7608740 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x559081332a60 0x55908133c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:34.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.451+0000 7f77d7608740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559081337bd0 0x5590813437a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:34.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.451+0000 7f77d7608740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x559081299c30 con 0x559081332070 2024-02-18T11:40:34.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.451+0000 7f77d7608740 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x55908123ed30 con 0x559081332a60 2024-02-18T11:40:34.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.451+0000 7f77d7608740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x559080f6d310 con 0x559081337bd0 2024-02-18T11:40:34.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.451+0000 7f77ce59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559081332070 0x55908133c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:34.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.451+0000 7f77d7608740 10 client.0 _ll_register_callbacks cb 0x559081246d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-18T11:40:34.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.451+0000 7f77ce59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559081332070 0x55908133c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48780/0 (socket says 172.21.15.12:48780) 2024-02-18T11:40:34.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.451+0000 7f77ce59c640 1 -- 172.21.15.12:0/1367929913 learned_addr learned my addr 172.21.15.12:0/1367929913 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:40:34.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.451+0000 7f77c7fff640 1 --2- 172.21.15.12:0/1367929913 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x559081332a60 0x55908133c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:34.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.451+0000 7f77ce59c640 1 -- 172.21.15.12:0/1367929913 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559081337bd0 msgr2=0x5590813437a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:40:34.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.451+0000 7f77ce59c640 1 --2- 172.21.15.12:0/1367929913 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559081337bd0 0x5590813437a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.451+0000 7f77ce59c640 1 -- 172.21.15.12:0/1367929913 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x559081332a60 msgr2=0x55908133c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:40:34.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.451+0000 7f77ce59c640 1 --2- 172.21.15.12:0/1367929913 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x559081332a60 0x55908133c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.451+0000 7f77ce59c640 1 -- 172.21.15.12:0/1367929913 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77c000b960 con 0x559081332070 2024-02-18T11:40:34.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.451+0000 7f77ce59c640 1 --2- 172.21.15.12:0/1367929913 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559081332070 0x55908133c220 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f77c0019000 tx=0x7f77c0012bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0e52f6f69898df7 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:34.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.451+0000 7f77b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-02-18T11:40:34.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.451+0000 7f77b5ffb640 1 -- 172.21.15.12:0/1367929913 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c0004230 con 0x559081332070 2024-02-18T11:40:34.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.451+0000 7f77b5ffb640 1 -- 172.21.15.12:0/1367929913 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f77c0004390 con 0x559081332070 2024-02-18T11:40:34.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.451+0000 7f77d7608740 1 -- 172.21.15.12:0/1367929913 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559080f996c0 con 0x559081332070 2024-02-18T11:40:34.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.451+0000 7f77b5ffb640 1 -- 172.21.15.12:0/1367929913 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f77c0011550 con 0x559081332070 2024-02-18T11:40:34.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.451+0000 7f77d7608740 10 client.15216 fetch_fsmap learned FSMap version 20 2024-02-18T11:40:34.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.451+0000 7f77d7608740 1 -- 172.21.15.12:0/1367929913 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x5590812982b0 con 0x559081332070 2024-02-18T11:40:34.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.452+0000 7f77b5ffb640 1 -- 172.21.15.12:0/1367929913 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x559080f996c0 con 0x559081332070 2024-02-18T11:40:34.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.452+0000 7f77b5ffb640 1 -- 172.21.15.12:0/1367929913 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f77c0023030 con 0x559081332070 2024-02-18T11:40:34.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.452+0000 7f77d7608740 10 client.15216 fetch_fsmap finished waiting for FSMap version 20 2024-02-18T11:40:34.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.452+0000 7f77d7608740 10 client.15216 resolve_mds: resolved name 'i' to daemon mds.i 2024-02-18T11:40:34.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.452+0000 7f77d7608740 20 client.15216 populate_metadata read hostname 'smithi012' 2024-02-18T11:40:34.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.452+0000 7f77d7608740 1 --2- 172.21.15.12:0/1367929913 >> [v2:172.21.15.188:6838/1342671190,v1:172.21.15.188:6839/1342671190] conn(0x5590813489b0 0x559081347c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:34.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.452+0000 7f77d7608740 4 client.15216 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-18T11:40:34.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.452+0000 7f77d7608740 1 -- 172.21.15.12:0/1367929913 --> [v2:172.21.15.188:6838/1342671190,v1:172.21.15.188:6839/1342671190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559080fea270 con 0x5590813489b0 2024-02-18T11:40:34.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.452+0000 7f77c7fff640 1 --2- 172.21.15.12:0/1367929913 >> [v2:172.21.15.188:6838/1342671190,v1:172.21.15.188:6839/1342671190] conn(0x5590813489b0 0x559081347c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:34.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.453+0000 7f77b5ffb640 10 client.15216 ms_handle_connect on v2:172.21.15.188:6838/1342671190 2024-02-18T11:40:34.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.453+0000 7f77c7fff640 1 --2- 172.21.15.12:0/1367929913 >> [v2:172.21.15.188:6838/1342671190,v1:172.21.15.188:6839/1342671190] conn(0x5590813489b0 0x559081347c50 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=310f0db3625d8602 server_cookie=255aee3b5431b7ab in_seq=0 out_seq=0 2024-02-18T11:40:34.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.453+0000 7f77b5ffb640 1 -- 172.21.15.12:0/1367929913 <== mds.1 v2:172.21.15.188:6838/1342671190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559080fea270 con 0x5590813489b0 2024-02-18T11:40:34.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.453+0000 7f77b5ffb640 10 client.15216 handle_command_reply: tid=0 2024-02-18T11:40:34.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.453+0000 7f77d7608740 1 client.15216 shutdown 2024-02-18T11:40:34.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.453+0000 7f77d7608740 1 -- 172.21.15.12:0/1367929913 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559081332070 msgr2=0x55908133c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:34.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.453+0000 7f77d7608740 1 --2- 172.21.15.12:0/1367929913 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559081332070 0x55908133c220 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f77c0019000 tx=0x7f77c0012bf0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.453+0000 7f77d7608740 1 -- 172.21.15.12:0/1367929913 shutdown_connections 2024-02-18T11:40:34.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.453+0000 7f77d7608740 1 --2- 172.21.15.12:0/1367929913 >> [v2:172.21.15.188:6838/1342671190,v1:172.21.15.188:6839/1342671190] conn(0x5590813489b0 0x559081347c50 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.453+0000 7f77d7608740 1 --2- 172.21.15.12:0/1367929913 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559081337bd0 0x5590813437a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.453+0000 7f77d7608740 1 --2- 172.21.15.12:0/1367929913 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x559081332a60 0x55908133c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.453+0000 7f77d7608740 1 --2- 172.21.15.12:0/1367929913 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559081332070 0x55908133c220 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.453+0000 7f77b5ffb640 0 client.15216 ms_handle_reset on v2:172.21.15.188:6838/1342671190 2024-02-18T11:40:34.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.453+0000 7f77d7608740 1 -- 172.21.15.12:0/1367929913 >> 172.21.15.12:0/1367929913 conn(0x55908132d5e0 msgr2=0x55908132c8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:34.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.453+0000 7f77d7608740 1 -- 172.21.15.12:0/1367929913 shutdown_connections 2024-02-18T11:40:34.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.453+0000 7f77d7608740 1 -- 172.21.15.12:0/1367929913 wait complete. 2024-02-18T11:40:34.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.454+0000 7f77d7608740 20 client.15216 trim_cache size 0 max 16384 2024-02-18T11:40:34.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.469+0000 7f77d7608740 1 Processor -- start 2024-02-18T11:40:34.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.469+0000 7f77d7608740 1 -- start start 2024-02-18T11:40:34.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.469+0000 7f77d7608740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559081332070 0x55908132c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:34.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.469+0000 7f77d7608740 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x559081332a60 0x559081343760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:34.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.469+0000 7f77d7608740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559081337bd0 0x55908132c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:34.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.469+0000 7f77d7608740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55908121f490 con 0x559081337bd0 2024-02-18T11:40:34.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.469+0000 7f77d7608740 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x559081347e80 con 0x559081332a60 2024-02-18T11:40:34.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.469+0000 7f77d7608740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x559080fea270 con 0x559081332070 2024-02-18T11:40:34.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.469+0000 7f77ced9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559081337bd0 0x55908132c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:34.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.469+0000 7f77ced9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559081337bd0 0x55908132c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48792/0 (socket says 172.21.15.12:48792) 2024-02-18T11:40:34.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.469+0000 7f77ced9d640 1 -- 172.21.15.12:0/159159165 learned_addr learned my addr 172.21.15.12:0/159159165 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:40:34.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.470+0000 7f77ce59c640 1 --2- 172.21.15.12:0/159159165 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559081332070 0x55908132c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:34.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.470+0000 7f77ced9d640 1 -- 172.21.15.12:0/159159165 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559081332070 msgr2=0x55908132c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:40:34.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.470+0000 7f77ced9d640 1 --2- 172.21.15.12:0/159159165 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559081332070 0x55908132c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.470+0000 7f77ced9d640 1 -- 172.21.15.12:0/159159165 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x559081332a60 msgr2=0x559081343760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:40:34.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.470+0000 7f77ced9d640 1 --2- 172.21.15.12:0/159159165 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x559081332a60 0x559081343760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.470+0000 7f77ced9d640 1 -- 172.21.15.12:0/159159165 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559081234a20 con 0x559081337bd0 2024-02-18T11:40:34.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.470+0000 7f77ced9d640 1 --2- 172.21.15.12:0/159159165 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559081337bd0 0x55908132c570 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f77c8009ca0 tx=0x7f77c80095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7b63511b7d604c7 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:34.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.470+0000 7f77b7fff640 1 -- 172.21.15.12:0/159159165 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c80327b0 con 0x559081337bd0 2024-02-18T11:40:34.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.470+0000 7f77b7fff640 1 -- 172.21.15.12:0/159159165 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f77c8032910 con 0x559081337bd0 2024-02-18T11:40:34.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.470+0000 7f77b7fff640 1 -- 172.21.15.12:0/159159165 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c80042a0 con 0x559081337bd0 2024-02-18T11:40:34.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.471+0000 7f77d7608740 1 -- 172.21.15.12:0/159159165 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559081337bd0 msgr2=0x55908132c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:34.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.471+0000 7f77d7608740 1 --2- 172.21.15.12:0/159159165 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559081337bd0 0x55908132c570 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f77c8009ca0 tx=0x7f77c80095c0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.471+0000 7f77d7608740 1 -- 172.21.15.12:0/159159165 shutdown_connections 2024-02-18T11:40:34.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.471+0000 7f77d7608740 1 --2- 172.21.15.12:0/159159165 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559081337bd0 0x55908132c570 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.471+0000 7f77d7608740 1 --2- 172.21.15.12:0/159159165 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x559081332a60 0x559081343760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.471+0000 7f77d7608740 1 --2- 172.21.15.12:0/159159165 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559081332070 0x55908132c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.471+0000 7f77d7608740 1 -- 172.21.15.12:0/159159165 >> 172.21.15.12:0/159159165 conn(0x55908132d5e0 msgr2=0x559081333ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:34.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.471+0000 7f77d7608740 1 -- 172.21.15.12:0/159159165 shutdown_connections 2024-02-18T11:40:34.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.471+0000 7f77d7608740 1 -- 172.21.15.12:0/159159165 wait complete. 2024-02-18T11:40:34.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.471+0000 7f77d7608740 1 -- start start 2024-02-18T11:40:34.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.471+0000 7f77d7608740 1 Processor -- start 2024-02-18T11:40:34.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.471+0000 7f77d7608740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559081332070 0x559081340ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:34.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.471+0000 7f77d7608740 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x559081332a60 0x559081340ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:34.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.471+0000 7f77d7608740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559081337bd0 0x559081342000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:34.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.471+0000 7f77d7608740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x559081342540 con 0x559081337bd0 2024-02-18T11:40:34.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.471+0000 7f77d7608740 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590813426b0 con 0x559081332a60 2024-02-18T11:40:34.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.471+0000 7f77d7608740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x559081342870 con 0x559081332070 2024-02-18T11:40:34.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.471+0000 7f77d7608740 10 client.0 _ll_register_callbacks cb 0x7f77c000b4c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-18T11:40:34.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.471+0000 7f77ced9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559081337bd0 0x559081342000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:34.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.471+0000 7f77ced9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559081337bd0 0x559081342000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48798/0 (socket says 172.21.15.12:48798) 2024-02-18T11:40:34.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.471+0000 7f77ced9d640 1 -- 172.21.15.12:0/1246277415 learned_addr learned my addr 172.21.15.12:0/1246277415 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:40:34.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.471+0000 7f77ce59c640 1 --2- 172.21.15.12:0/1246277415 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559081332070 0x559081340ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:34.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.471+0000 7f77c7fff640 1 --2- 172.21.15.12:0/1246277415 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x559081332a60 0x559081340ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:34.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.472+0000 7f77ced9d640 1 -- 172.21.15.12:0/1246277415 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559081332070 msgr2=0x559081340ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:40:34.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.472+0000 7f77ced9d640 1 --2- 172.21.15.12:0/1246277415 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559081332070 0x559081340ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.472+0000 7f77ced9d640 1 -- 172.21.15.12:0/1246277415 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x559081332a60 msgr2=0x559081340ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:40:34.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.472+0000 7f77ced9d640 1 --2- 172.21.15.12:0/1246277415 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x559081332a60 0x559081340ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.472+0000 7f77ced9d640 1 -- 172.21.15.12:0/1246277415 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77d0108e20 con 0x559081337bd0 2024-02-18T11:40:34.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.472+0000 7f77c7fff640 1 --2- 172.21.15.12:0/1246277415 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x559081332a60 0x559081340ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:40:34.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.472+0000 7f77ce59c640 1 --2- 172.21.15.12:0/1246277415 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559081332070 0x559081340ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:40:34.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.472+0000 7f77ced9d640 1 --2- 172.21.15.12:0/1246277415 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559081337bd0 0x559081342000 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f77c80325e0 tx=0x7f77c8032460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5bc718d76970302 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:34.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.472+0000 7f77b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-02-18T11:40:34.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.472+0000 7f77b5ffb640 1 -- 172.21.15.12:0/1246277415 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c80327b0 con 0x559081337bd0 2024-02-18T11:40:34.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.472+0000 7f77b5ffb640 1 -- 172.21.15.12:0/1246277415 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f77c8004130 con 0x559081337bd0 2024-02-18T11:40:34.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.472+0000 7f77d7608740 1 -- 172.21.15.12:0/1246277415 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f77c000be20 con 0x559081337bd0 2024-02-18T11:40:34.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.472+0000 7f77b5ffb640 1 -- 172.21.15.12:0/1246277415 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f77c80042f0 con 0x559081337bd0 2024-02-18T11:40:34.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.472+0000 7f77d7608740 10 client.15222 fetch_fsmap learned FSMap version 20 2024-02-18T11:40:34.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.472+0000 7f77d7608740 1 -- 172.21.15.12:0/1246277415 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55908125a780 con 0x559081337bd0 2024-02-18T11:40:34.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.472+0000 7f77b5ffb640 1 -- 172.21.15.12:0/1246277415 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f77c000be20 con 0x559081337bd0 2024-02-18T11:40:34.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.472+0000 7f77b5ffb640 1 -- 172.21.15.12:0/1246277415 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f77c8055d00 con 0x559081337bd0 2024-02-18T11:40:34.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.472+0000 7f77d7608740 10 client.15222 fetch_fsmap finished waiting for FSMap version 20 2024-02-18T11:40:34.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.472+0000 7f77d7608740 10 client.15222 resolve_mds: resolved name 'i' to daemon mds.i 2024-02-18T11:40:34.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.472+0000 7f77d7608740 20 client.15222 populate_metadata read hostname 'smithi012' 2024-02-18T11:40:34.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.472+0000 7f77d7608740 1 --2- 172.21.15.12:0/1246277415 >> [v2:172.21.15.188:6838/1342671190,v1:172.21.15.188:6839/1342671190] conn(0x5590813a40f0 0x5590813a0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:34.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.472+0000 7f77d7608740 4 client.15222 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "status"}] 2024-02-18T11:40:34.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.472+0000 7f77d7608740 1 -- 172.21.15.12:0/1246277415 --> [v2:172.21.15.188:6838/1342671190,v1:172.21.15.188:6839/1342671190] -- command(tid 0: {"prefix": "status"}) v1 -- 0x559080fd6610 con 0x5590813a40f0 2024-02-18T11:40:34.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.473+0000 7f77ce59c640 1 --2- 172.21.15.12:0/1246277415 >> [v2:172.21.15.188:6838/1342671190,v1:172.21.15.188:6839/1342671190] conn(0x5590813a40f0 0x5590813a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:34.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.473+0000 7f77b5ffb640 10 client.15222 ms_handle_connect on v2:172.21.15.188:6838/1342671190 2024-02-18T11:40:34.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.473+0000 7f77ce59c640 1 --2- 172.21.15.12:0/1246277415 >> [v2:172.21.15.188:6838/1342671190,v1:172.21.15.188:6839/1342671190] conn(0x5590813a40f0 0x5590813a0610 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=70d066b8c0d01871 server_cookie=eb420df44e3fa7fb in_seq=0 out_seq=0 2024-02-18T11:40:34.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.473+0000 7f77b5ffb640 1 -- 172.21.15.12:0/1246277415 <== mds.1 v2:172.21.15.188:6838/1342671190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x559080fd6610 con 0x5590813a40f0 2024-02-18T11:40:34.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.473+0000 7f77b5ffb640 10 client.15222 handle_command_reply: tid=0 2024-02-18T11:40:34.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.473+0000 7f77d7608740 1 client.15222 shutdown 2024-02-18T11:40:34.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.474+0000 7f77d7608740 1 -- 172.21.15.12:0/1246277415 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559081337bd0 msgr2=0x559081342000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:34.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.474+0000 7f77d7608740 1 --2- 172.21.15.12:0/1246277415 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559081337bd0 0x559081342000 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f77c80325e0 tx=0x7f77c8032460 comp rx=0 tx=0).stop 2024-02-18T11:40:34.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.474+0000 7f77d7608740 1 -- 172.21.15.12:0/1246277415 shutdown_connections 2024-02-18T11:40:34.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.474+0000 7f77d7608740 1 --2- 172.21.15.12:0/1246277415 >> [v2:172.21.15.188:6838/1342671190,v1:172.21.15.188:6839/1342671190] conn(0x5590813a40f0 0x5590813a0610 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.474+0000 7f77d7608740 1 --2- 172.21.15.12:0/1246277415 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559081337bd0 0x559081342000 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.474+0000 7f77b5ffb640 0 client.15222 ms_handle_reset on v2:172.21.15.188:6838/1342671190 2024-02-18T11:40:34.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.474+0000 7f77d7608740 1 --2- 172.21.15.12:0/1246277415 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x559081332a60 0x559081340ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.474+0000 7f77d7608740 1 --2- 172.21.15.12:0/1246277415 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559081332070 0x559081340ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.474+0000 7f77d7608740 1 -- 172.21.15.12:0/1246277415 >> 172.21.15.12:0/1246277415 conn(0x55908132d5e0 msgr2=0x559081330840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:34.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.474+0000 7f77d7608740 1 -- 172.21.15.12:0/1246277415 shutdown_connections 2024-02-18T11:40:34.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.474+0000 7f77d7608740 1 -- 172.21.15.12:0/1246277415 wait complete. 2024-02-18T11:40:34.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.474+0000 7f77d7608740 20 client.15222 trim_cache size 0 max 16384 2024-02-18T11:40:34.475 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-02-18T11:40:34.475 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_fsid": "0c8bb0de-ce51-11ee-95ba-87774f69a715", 2024-02-18T11:40:34.475 INFO:teuthology.orchestra.run.smithi012.stdout: "whoami": 1, 2024-02-18T11:40:34.475 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 24508, 2024-02-18T11:40:34.475 INFO:teuthology.orchestra.run.smithi012.stdout: "want_state": "up:active", 2024-02-18T11:40:34.475 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "up:active", 2024-02-18T11:40:34.475 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs", 2024-02-18T11:40:34.475 INFO:teuthology.orchestra.run.smithi012.stdout: "rank_uptime": 104.627489892, 2024-02-18T11:40:34.475 INFO:teuthology.orchestra.run.smithi012.stdout: "mdsmap_epoch": 20, 2024-02-18T11:40:34.475 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch": 89, 2024-02-18T11:40:34.475 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch_barrier": 87, 2024-02-18T11:40:34.476 INFO:teuthology.orchestra.run.smithi012.stdout: "uptime": 120.878666482 2024-02-18T11:40:34.476 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-02-18T11:40:34.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.474+0000 7f77d4956640 1 -- 172.21.15.12:0/3229740185 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f77bc0a2750 msgr2=0x7f77bc0a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:34.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.474+0000 7f77d4956640 1 --2- 172.21.15.12:0/3229740185 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f77bc0a2750 0x7f77bc0a4c10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f77c8005e30 tx=0x7f77c8005ef0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.474+0000 7f77d4956640 1 -- 172.21.15.12:0/3229740185 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f77d01012d0 msgr2=0x7f77d019bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:34.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.474+0000 7f77d4956640 1 --2- 172.21.15.12:0/3229740185 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f77d01012d0 0x7f77d019bf90 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f77b800d950 tx=0x7f77b800de10 comp rx=0 tx=0).stop 2024-02-18T11:40:34.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.474+0000 7f77d4956640 1 -- 172.21.15.12:0/3229740185 shutdown_connections 2024-02-18T11:40:34.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.474+0000 7f77d4956640 1 --2- 172.21.15.12:0/3229740185 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f77bc0a2750 0x7f77bc0a4c10 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.474+0000 7f77d4956640 1 --2- 172.21.15.12:0/3229740185 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f77d0101c90 0x7f77d019e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.474+0000 7f77d4956640 1 --2- 172.21.15.12:0/3229740185 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f77d01012d0 0x7f77d019bf90 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.474+0000 7f77d4956640 1 --2- 172.21.15.12:0/3229740185 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f77d01000e0 0x7f77d006a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.474+0000 7f77d4956640 1 -- 172.21.15.12:0/3229740185 >> 172.21.15.12:0/3229740185 conn(0x7f77d00fb670 msgr2=0x7f77d0104570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:34.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.474+0000 7f77d4956640 1 -- 172.21.15.12:0/3229740185 shutdown_connections 2024-02-18T11:40:34.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.474+0000 7f77d4956640 1 -- 172.21.15.12:0/3229740185 wait complete. 2024-02-18T11:40:34.478 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:40:34 smithi012 ceph-mon[32460]: pgmap v340: 97 pgs: 97 active+clean; 640 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-02-18T11:40:34.478 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:40:34 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/1059862397' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-18T11:40:34.478 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:40:34 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/876170320' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-18T11:40:34.478 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:40:34 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/2818017034' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-18T11:40:34.478 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:40:34 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/3723350426' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-18T11:40:34.479 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-02-18T11:40:34.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:40:34 smithi158 ceph-mon[36626]: pgmap v340: 97 pgs: 97 active+clean; 640 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-02-18T11:40:34.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:40:34 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/1059862397' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-18T11:40:34.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:40:34 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/876170320' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-18T11:40:34.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:40:34 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/2818017034' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-18T11:40:34.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:40:34 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/3723350426' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-18T11:40:34.505 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:40:34 smithi188 ceph-mon[36925]: pgmap v340: 97 pgs: 97 active+clean; 640 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-02-18T11:40:34.505 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:40:34 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/1059862397' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-18T11:40:34.505 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:40:34 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/876170320' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-18T11:40:34.505 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:40:34 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/2818017034' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-18T11:40:34.505 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:40:34 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/3723350426' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-18T11:40:34.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.588+0000 7f99952d4640 1 -- 172.21.15.12:0/2579252288 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9990101c70 msgr2=0x7f9990108500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:34.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.588+0000 7f99952d4640 1 --2- 172.21.15.12:0/2579252288 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9990101c70 0x7f9990108500 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f9988009ca0 tx=0x7f998802c0f0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.588+0000 7f99952d4640 1 -- 172.21.15.12:0/2579252288 shutdown_connections 2024-02-18T11:40:34.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.588+0000 7f99952d4640 1 --2- 172.21.15.12:0/2579252288 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9990101c70 0x7f9990108500 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.588+0000 7f99952d4640 1 --2- 172.21.15.12:0/2579252288 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f99901012b0 0x7f9990101730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.588+0000 7f99952d4640 1 --2- 172.21.15.12:0/2579252288 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99901000c0 0x7f99901004e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.588+0000 7f99952d4640 1 -- 172.21.15.12:0/2579252288 >> 172.21.15.12:0/2579252288 conn(0x7f99900fb650 msgr2=0x7f99900fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:34.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.588+0000 7f99952d4640 1 -- 172.21.15.12:0/2579252288 shutdown_connections 2024-02-18T11:40:34.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.588+0000 7f99952d4640 1 -- 172.21.15.12:0/2579252288 wait complete. 2024-02-18T11:40:34.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.589+0000 7f99952d4640 1 Processor -- start 2024-02-18T11:40:34.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.589+0000 7f99952d4640 1 -- start start 2024-02-18T11:40:34.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.589+0000 7f99952d4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99901000c0 0x7f99901972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:34.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.589+0000 7f99952d4640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f99901012b0 0x7f9990197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:34.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.589+0000 7f99952d4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9990101c70 0x7f999019e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:34.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.589+0000 7f99952d4640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f999019edc0 con 0x7f9990101c70 2024-02-18T11:40:34.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.589+0000 7f99952d4640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f999019ef30 con 0x7f99901012b0 2024-02-18T11:40:34.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.589+0000 7f99952d4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f999019f0f0 con 0x7f99901000c0 2024-02-18T11:40:34.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.589+0000 7f998f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9990101c70 0x7f999019e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:34.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.589+0000 7f998f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9990101c70 0x7f999019e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48816/0 (socket says 172.21.15.12:48816) 2024-02-18T11:40:34.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.589+0000 7f998f7fe640 1 -- 172.21.15.12:0/3929252273 learned_addr learned my addr 172.21.15.12:0/3929252273 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:40:34.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.589+0000 7f9987fff640 1 --2- 172.21.15.12:0/3929252273 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f99901012b0 0x7f9990197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:34.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.589+0000 7f998effd640 1 --2- 172.21.15.12:0/3929252273 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99901000c0 0x7f99901972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:34.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.589+0000 7f998f7fe640 1 -- 172.21.15.12:0/3929252273 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99901000c0 msgr2=0x7f99901972d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:34.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.589+0000 7f998f7fe640 1 --2- 172.21.15.12:0/3929252273 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99901000c0 0x7f99901972d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.589+0000 7f998f7fe640 1 -- 172.21.15.12:0/3929252273 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f99901012b0 msgr2=0x7f9990197810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:34.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.589+0000 7f998f7fe640 1 --2- 172.21.15.12:0/3929252273 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f99901012b0 0x7f9990197810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.589+0000 7f998f7fe640 1 -- 172.21.15.12:0/3929252273 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9988009900 con 0x7f9990101c70 2024-02-18T11:40:34.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.589+0000 7f9987fff640 1 --2- 172.21.15.12:0/3929252273 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f99901012b0 0x7f9990197810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:40:34.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.590+0000 7f998f7fe640 1 --2- 172.21.15.12:0/3929252273 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9990101c70 0x7f999019e880 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f9988037040 tx=0x7f9988034a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:34.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.590+0000 7f998cff9640 1 -- 172.21.15.12:0/3929252273 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99880329b0 con 0x7f9990101c70 2024-02-18T11:40:34.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.590+0000 7f99952d4640 1 -- 172.21.15.12:0/3929252273 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f999019f370 con 0x7f9990101c70 2024-02-18T11:40:34.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.590+0000 7f998cff9640 1 -- 172.21.15.12:0/3929252273 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9988003e50 con 0x7f9990101c70 2024-02-18T11:40:34.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.590+0000 7f998cff9640 1 -- 172.21.15.12:0/3929252273 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9988040450 con 0x7f9990101c70 2024-02-18T11:40:34.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.590+0000 7f99952d4640 1 -- 172.21.15.12:0/3929252273 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f999019f870 con 0x7f9990101c70 2024-02-18T11:40:34.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.592+0000 7f998cff9640 1 -- 172.21.15.12:0/3929252273 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) v1 ==== 137245+0+0 (secure 0 0 0) 0x7f998800a040 con 0x7f9990101c70 2024-02-18T11:40:34.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.592+0000 7f9997f7e740 1 Processor -- start 2024-02-18T11:40:34.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.592+0000 7f9997f7e740 1 -- start start 2024-02-18T11:40:34.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.592+0000 7f998cff9640 1 --2- 172.21.15.12:0/3929252273 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f997c0a2750 0x7f997c0a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:34.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.593+0000 7f9987fff640 1 --2- 172.21.15.12:0/3929252273 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f997c0a2750 0x7f997c0a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:34.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.593+0000 7f998cff9640 1 -- 172.21.15.12:0/3929252273 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f99880f1510 con 0x7f9990101c70 2024-02-18T11:40:34.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.593+0000 7f9997f7e740 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55882a0ac480 0x55882a0ac8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:34.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.593+0000 7f9997f7e740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55882a0ace70 0x55882a0b1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:34.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.593+0000 7f9997f7e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55882a0b1fe0 0x55882a0b4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:34.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.593+0000 7f9997f7e740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55882a063dc0 con 0x55882a0ace70 2024-02-18T11:40:34.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.593+0000 7f9997f7e740 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x55882a016f20 con 0x55882a0ac480 2024-02-18T11:40:34.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.593+0000 7f9997f7e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55882a014450 con 0x55882a0b1fe0 2024-02-18T11:40:34.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.593+0000 7f9987fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55882a0ace70 0x55882a0b1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:34.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.593+0000 7f9987fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55882a0ace70 0x55882a0b1970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48832/0 (socket says 172.21.15.12:48832) 2024-02-18T11:40:34.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.593+0000 7f9987fff640 1 -- 172.21.15.12:0/596205648 learned_addr learned my addr 172.21.15.12:0/596205648 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:40:34.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.593+0000 7f998effd640 1 --2- 172.21.15.12:0/596205648 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55882a0ac480 0x55882a0ac8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:34.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.593+0000 7f998f7fe640 1 --2- 172.21.15.12:0/596205648 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55882a0b1fe0 0x55882a0b4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:34.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.593+0000 7f9987fff640 1 --2- 172.21.15.12:0/3929252273 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f997c0a2750 0x7f997c0a4c10 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9978009e70 tx=0x7f9978014660 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:34.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.593+0000 7f9987fff640 1 -- 172.21.15.12:0/596205648 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55882a0b1fe0 msgr2=0x55882a0b4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:40:34.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.593+0000 7f9987fff640 1 --2- 172.21.15.12:0/596205648 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55882a0b1fe0 0x55882a0b4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.593+0000 7f9987fff640 1 -- 172.21.15.12:0/596205648 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55882a0ac480 msgr2=0x55882a0ac8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:40:34.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.593+0000 7f9987fff640 1 --2- 172.21.15.12:0/596205648 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55882a0ac480 0x55882a0ac8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.593+0000 7f9987fff640 1 -- 172.21.15.12:0/596205648 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558829fc4f20 con 0x55882a0ace70 2024-02-18T11:40:34.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.593+0000 7f9987fff640 1 --2- 172.21.15.12:0/596205648 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55882a0ace70 0x55882a0b1970 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f9978014ee0 tx=0x7f997801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8df2bf8bc0321df8 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:34.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.594+0000 7f99857fa640 1 -- 172.21.15.12:0/596205648 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99780115c0 con 0x55882a0ace70 2024-02-18T11:40:34.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.594+0000 7f99857fa640 1 -- 172.21.15.12:0/596205648 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9978011c00 con 0x55882a0ace70 2024-02-18T11:40:34.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.594+0000 7f99857fa640 1 -- 172.21.15.12:0/596205648 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9978021d40 con 0x55882a0ace70 2024-02-18T11:40:34.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.594+0000 7f9997f7e740 1 -- 172.21.15.12:0/596205648 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55882a0ace70 msgr2=0x55882a0b1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:34.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.594+0000 7f9997f7e740 1 --2- 172.21.15.12:0/596205648 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55882a0ace70 0x55882a0b1970 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f9978014ee0 tx=0x7f997801b000 comp rx=0 tx=0).stop 2024-02-18T11:40:34.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.594+0000 7f9997f7e740 1 -- 172.21.15.12:0/596205648 shutdown_connections 2024-02-18T11:40:34.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.594+0000 7f9997f7e740 1 --2- 172.21.15.12:0/596205648 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55882a0b1fe0 0x55882a0b4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.594+0000 7f9997f7e740 1 --2- 172.21.15.12:0/596205648 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55882a0ace70 0x55882a0b1970 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.594+0000 7f9997f7e740 1 --2- 172.21.15.12:0/596205648 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55882a0ac480 0x55882a0ac8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.594+0000 7f9997f7e740 1 -- 172.21.15.12:0/596205648 >> 172.21.15.12:0/596205648 conn(0x55882a0a35e0 msgr2=0x55882a0a39e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:34.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.594+0000 7f9997f7e740 1 -- 172.21.15.12:0/596205648 shutdown_connections 2024-02-18T11:40:34.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.594+0000 7f9997f7e740 1 -- 172.21.15.12:0/596205648 wait complete. 2024-02-18T11:40:34.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.595+0000 7f9997f7e740 1 -- start start 2024-02-18T11:40:34.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.595+0000 7f9997f7e740 1 Processor -- start 2024-02-18T11:40:34.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.595+0000 7f9997f7e740 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55882a0ac480 0x55882a0b64e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:34.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.595+0000 7f9997f7e740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55882a0ace70 0x55882a0b6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:34.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.595+0000 7f9997f7e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55882a0b1fe0 0x55882a0bda60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:34.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.595+0000 7f9997f7e740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55882a00fc30 con 0x55882a0ace70 2024-02-18T11:40:34.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.595+0000 7f9997f7e740 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x558829fb4d30 con 0x55882a0ac480 2024-02-18T11:40:34.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.595+0000 7f9997f7e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x558829ce3310 con 0x55882a0b1fe0 2024-02-18T11:40:34.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.595+0000 7f9997f7e740 10 client.0 _ll_register_callbacks cb 0x558829fbcd40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-18T11:40:34.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.595+0000 7f9987fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55882a0ace70 0x55882a0b6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:34.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.595+0000 7f9987fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55882a0ace70 0x55882a0b6a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48846/0 (socket says 172.21.15.12:48846) 2024-02-18T11:40:34.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.595+0000 7f9987fff640 1 -- 172.21.15.12:0/1193465919 learned_addr learned my addr 172.21.15.12:0/1193465919 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:40:34.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.595+0000 7f998effd640 1 --2- 172.21.15.12:0/1193465919 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55882a0ac480 0x55882a0b64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:34.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.595+0000 7f998f7fe640 1 --2- 172.21.15.12:0/1193465919 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55882a0b1fe0 0x55882a0bda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:34.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.595+0000 7f9987fff640 1 -- 172.21.15.12:0/1193465919 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55882a0b1fe0 msgr2=0x55882a0bda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:40:34.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.595+0000 7f9987fff640 1 --2- 172.21.15.12:0/1193465919 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55882a0b1fe0 0x55882a0bda60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.595+0000 7f9987fff640 1 -- 172.21.15.12:0/1193465919 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55882a0ac480 msgr2=0x55882a0b64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:40:34.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.595+0000 7f9987fff640 1 --2- 172.21.15.12:0/1193465919 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55882a0ac480 0x55882a0b64e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.595+0000 7f9987fff640 1 -- 172.21.15.12:0/1193465919 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f997800e040 con 0x55882a0ace70 2024-02-18T11:40:34.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.596+0000 7f9987fff640 1 --2- 172.21.15.12:0/1193465919 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55882a0ace70 0x55882a0b6a20 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f9978014b60 tx=0x7f9978022fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d34219c9e467496 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:34.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.596+0000 7f99767fc640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-02-18T11:40:34.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.596+0000 7f99767fc640 1 -- 172.21.15.12:0/1193465919 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99780115c0 con 0x55882a0ace70 2024-02-18T11:40:34.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.596+0000 7f99767fc640 1 -- 172.21.15.12:0/1193465919 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9978011720 con 0x55882a0ace70 2024-02-18T11:40:34.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.596+0000 7f9997f7e740 1 -- 172.21.15.12:0/1193465919 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558829d0f6c0 con 0x55882a0ace70 2024-02-18T11:40:34.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.596+0000 7f99767fc640 1 -- 172.21.15.12:0/1193465919 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99780118e0 con 0x55882a0ace70 2024-02-18T11:40:34.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.596+0000 7f9997f7e740 10 client.15234 fetch_fsmap learned FSMap version 20 2024-02-18T11:40:34.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.596+0000 7f9997f7e740 1 -- 172.21.15.12:0/1193465919 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55882a00e2b0 con 0x55882a0ace70 2024-02-18T11:40:34.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.596+0000 7f99767fc640 1 -- 172.21.15.12:0/1193465919 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x558829d0f6c0 con 0x55882a0ace70 2024-02-18T11:40:34.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.596+0000 7f99767fc640 1 -- 172.21.15.12:0/1193465919 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9978017d20 con 0x55882a0ace70 2024-02-18T11:40:34.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.596+0000 7f9997f7e740 10 client.15234 fetch_fsmap finished waiting for FSMap version 20 2024-02-18T11:40:34.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.596+0000 7f9997f7e740 10 client.15234 resolve_mds: resolved name 'b' to daemon mds.b 2024-02-18T11:40:34.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.596+0000 7f9997f7e740 20 client.15234 populate_metadata read hostname 'smithi012' 2024-02-18T11:40:34.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.596+0000 7f9997f7e740 1 --2- 172.21.15.12:0/1193465919 >> [v2:172.21.15.158:6838/1127200849,v1:172.21.15.158:6839/1127200849] conn(0x55882a0c27e0 0x55882a0c20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:34.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.596+0000 7f9997f7e740 4 client.15234 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-18T11:40:34.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.596+0000 7f9997f7e740 1 -- 172.21.15.12:0/1193465919 --> [v2:172.21.15.158:6838/1127200849,v1:172.21.15.158:6839/1127200849] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558829d60270 con 0x55882a0c27e0 2024-02-18T11:40:34.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.597+0000 7f998effd640 1 --2- 172.21.15.12:0/1193465919 >> [v2:172.21.15.158:6838/1127200849,v1:172.21.15.158:6839/1127200849] conn(0x55882a0c27e0 0x55882a0c20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:34.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.597+0000 7f998effd640 1 --2- 172.21.15.12:0/1193465919 >> [v2:172.21.15.158:6838/1127200849,v1:172.21.15.158:6839/1127200849] conn(0x55882a0c27e0 0x55882a0c20c0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=760035c1b793915d server_cookie=bae565f5697ee65d in_seq=0 out_seq=0 2024-02-18T11:40:34.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.597+0000 7f99767fc640 10 client.15234 ms_handle_connect on v2:172.21.15.158:6838/1127200849 2024-02-18T11:40:34.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.598+0000 7f99767fc640 1 -- 172.21.15.12:0/1193465919 <== mds.0 v2:172.21.15.158:6838/1127200849 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558829d60270 con 0x55882a0c27e0 2024-02-18T11:40:34.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.598+0000 7f99767fc640 10 client.15234 handle_command_reply: tid=0 2024-02-18T11:40:34.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.598+0000 7f9997f7e740 1 client.15234 shutdown 2024-02-18T11:40:34.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.598+0000 7f9997f7e740 1 -- 172.21.15.12:0/1193465919 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55882a0ace70 msgr2=0x55882a0b6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:34.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.599+0000 7f9997f7e740 1 --2- 172.21.15.12:0/1193465919 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55882a0ace70 0x55882a0b6a20 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f9978014b60 tx=0x7f9978022fb0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.599+0000 7f9997f7e740 1 -- 172.21.15.12:0/1193465919 shutdown_connections 2024-02-18T11:40:34.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.599+0000 7f9997f7e740 1 --2- 172.21.15.12:0/1193465919 >> [v2:172.21.15.158:6838/1127200849,v1:172.21.15.158:6839/1127200849] conn(0x55882a0c27e0 0x55882a0c20c0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.599+0000 7f9997f7e740 1 --2- 172.21.15.12:0/1193465919 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55882a0b1fe0 0x55882a0bda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.599+0000 7f99767fc640 0 client.15234 ms_handle_reset on v2:172.21.15.158:6838/1127200849 2024-02-18T11:40:34.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.599+0000 7f9997f7e740 1 --2- 172.21.15.12:0/1193465919 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55882a0ace70 0x55882a0b6a20 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.599+0000 7f9997f7e740 1 --2- 172.21.15.12:0/1193465919 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55882a0ac480 0x55882a0b64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.599+0000 7f9997f7e740 1 -- 172.21.15.12:0/1193465919 >> 172.21.15.12:0/1193465919 conn(0x55882a0a35e0 msgr2=0x55882a0a28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:34.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.599+0000 7f9997f7e740 1 -- 172.21.15.12:0/1193465919 shutdown_connections 2024-02-18T11:40:34.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.599+0000 7f9997f7e740 1 -- 172.21.15.12:0/1193465919 wait complete. 2024-02-18T11:40:34.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.599+0000 7f9997f7e740 20 client.15234 trim_cache size 0 max 16384 2024-02-18T11:40:34.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.614+0000 7f9997f7e740 1 Processor -- start 2024-02-18T11:40:34.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.614+0000 7f9997f7e740 1 -- start start 2024-02-18T11:40:34.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.615+0000 7f9997f7e740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55882a0ac480 0x55882a0b50f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:34.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.615+0000 7f9997f7e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55882a0ace70 0x55882a0bda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:34.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.615+0000 7f9997f7e740 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55882a0b1fe0 0x55882a0cb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:34.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.615+0000 7f9997f7e740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x558829f95490 con 0x55882a0ac480 2024-02-18T11:40:34.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.615+0000 7f9997f7e740 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x55882a0ac140 con 0x55882a0b1fe0 2024-02-18T11:40:34.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.615+0000 7f9997f7e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55882a0a40d0 con 0x55882a0ace70 2024-02-18T11:40:34.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.615+0000 7f998effd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55882a0ac480 0x55882a0b50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:34.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.615+0000 7f998effd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55882a0ac480 0x55882a0b50f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48850/0 (socket says 172.21.15.12:48850) 2024-02-18T11:40:34.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.615+0000 7f998effd640 1 -- 172.21.15.12:0/1977202366 learned_addr learned my addr 172.21.15.12:0/1977202366 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:40:34.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.615+0000 7f998effd640 1 -- 172.21.15.12:0/1977202366 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55882a0ace70 msgr2=0x55882a0bda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:40:34.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.615+0000 7f998effd640 1 --2- 172.21.15.12:0/1977202366 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55882a0ace70 0x55882a0bda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.615+0000 7f998effd640 1 -- 172.21.15.12:0/1977202366 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55882a0b1fe0 msgr2=0x55882a0cb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:40:34.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.615+0000 7f998effd640 1 --2- 172.21.15.12:0/1977202366 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55882a0b1fe0 0x55882a0cb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.615+0000 7f998effd640 1 -- 172.21.15.12:0/1977202366 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558829faaa20 con 0x55882a0ac480 2024-02-18T11:40:34.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.615+0000 7f998effd640 1 --2- 172.21.15.12:0/1977202366 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55882a0ac480 0x55882a0b50f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x55882a0ad6d0 tx=0x7f9980005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7864b3f6d5352e23 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:34.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.615+0000 7f9984ff9640 1 -- 172.21.15.12:0/1977202366 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9980017040 con 0x55882a0ac480 2024-02-18T11:40:34.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.615+0000 7f9984ff9640 1 -- 172.21.15.12:0/1977202366 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9980018040 con 0x55882a0ac480 2024-02-18T11:40:34.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.615+0000 7f9984ff9640 1 -- 172.21.15.12:0/1977202366 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9980012740 con 0x55882a0ac480 2024-02-18T11:40:34.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.616+0000 7f9997f7e740 1 -- 172.21.15.12:0/1977202366 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55882a0ac480 msgr2=0x55882a0b50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:34.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.616+0000 7f9997f7e740 1 --2- 172.21.15.12:0/1977202366 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55882a0ac480 0x55882a0b50f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x55882a0ad6d0 tx=0x7f9980005f90 comp rx=0 tx=0).stop 2024-02-18T11:40:34.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.616+0000 7f9997f7e740 1 -- 172.21.15.12:0/1977202366 shutdown_connections 2024-02-18T11:40:34.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.616+0000 7f9997f7e740 1 --2- 172.21.15.12:0/1977202366 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55882a0ace70 0x55882a0bda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.616+0000 7f9997f7e740 1 --2- 172.21.15.12:0/1977202366 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55882a0b1fe0 0x55882a0cb2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.616+0000 7f9997f7e740 1 --2- 172.21.15.12:0/1977202366 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55882a0ac480 0x55882a0b50f0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.616+0000 7f9997f7e740 1 -- 172.21.15.12:0/1977202366 >> 172.21.15.12:0/1977202366 conn(0x55882a0a35e0 msgr2=0x55882a0a39c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:34.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.616+0000 7f9997f7e740 1 -- 172.21.15.12:0/1977202366 shutdown_connections 2024-02-18T11:40:34.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.616+0000 7f9997f7e740 1 -- 172.21.15.12:0/1977202366 wait complete. 2024-02-18T11:40:34.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.616+0000 7f9997f7e740 1 -- start start 2024-02-18T11:40:34.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.616+0000 7f9997f7e740 1 Processor -- start 2024-02-18T11:40:34.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.617+0000 7f9997f7e740 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55882a0ac480 0x55882a0ba890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:34.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.617+0000 7f9997f7e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55882a0ace70 0x55882a0badd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:34.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.617+0000 7f9997f7e740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55882a0b1fe0 0x55882a118a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:34.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.617+0000 7f9997f7e740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55882a0bb390 con 0x55882a0b1fe0 2024-02-18T11:40:34.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.617+0000 7f9997f7e740 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x55882a0bb500 con 0x55882a0ac480 2024-02-18T11:40:34.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.617+0000 7f9997f7e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55882a118fc0 con 0x55882a0ace70 2024-02-18T11:40:34.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.617+0000 7f9997f7e740 10 client.0 _ll_register_callbacks cb 0x7f9980009590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-18T11:40:34.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.617+0000 7f998f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55882a0b1fe0 0x55882a118a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:34.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.617+0000 7f998f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55882a0b1fe0 0x55882a118a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48862/0 (socket says 172.21.15.12:48862) 2024-02-18T11:40:34.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.617+0000 7f998f7fe640 1 -- 172.21.15.12:0/2684695565 learned_addr learned my addr 172.21.15.12:0/2684695565 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:40:34.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.617+0000 7f998effd640 1 --2- 172.21.15.12:0/2684695565 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55882a0ac480 0x55882a0ba890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:34.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.617+0000 7f998f7fe640 1 -- 172.21.15.12:0/2684695565 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55882a0ace70 msgr2=0x55882a0badd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:40:34.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.617+0000 7f998f7fe640 1 --2- 172.21.15.12:0/2684695565 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55882a0ace70 0x55882a0badd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.617+0000 7f998f7fe640 1 -- 172.21.15.12:0/2684695565 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55882a0ac480 msgr2=0x55882a0ba890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:40:34.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.617+0000 7f998f7fe640 1 --2- 172.21.15.12:0/2684695565 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55882a0ac480 0x55882a0ba890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.617+0000 7f998f7fe640 1 -- 172.21.15.12:0/2684695565 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9980009ad0 con 0x55882a0b1fe0 2024-02-18T11:40:34.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.617+0000 7f998effd640 1 --2- 172.21.15.12:0/2684695565 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55882a0ac480 0x55882a0ba890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:40:34.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.617+0000 7f998f7fe640 1 --2- 172.21.15.12:0/2684695565 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55882a0b1fe0 0x55882a118a00 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f9988034f50 tx=0x7f99880bda80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9daccc5e8089dc18 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:34.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.617+0000 7f99767fc640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-02-18T11:40:34.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.617+0000 7f99767fc640 1 -- 172.21.15.12:0/2684695565 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99880042d0 con 0x55882a0b1fe0 2024-02-18T11:40:34.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.617+0000 7f99767fc640 1 -- 172.21.15.12:0/2684695565 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f99880c67b0 con 0x55882a0b1fe0 2024-02-18T11:40:34.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.617+0000 7f9997f7e740 1 -- 172.21.15.12:0/2684695565 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55882a0c3e30 con 0x55882a0b1fe0 2024-02-18T11:40:34.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.617+0000 7f99767fc640 1 -- 172.21.15.12:0/2684695565 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9988050300 con 0x55882a0b1fe0 2024-02-18T11:40:34.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.617+0000 7f9997f7e740 10 client.15240 fetch_fsmap learned FSMap version 20 2024-02-18T11:40:34.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.617+0000 7f9997f7e740 1 -- 172.21.15.12:0/2684695565 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x558829fd0780 con 0x55882a0b1fe0 2024-02-18T11:40:34.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.617+0000 7f99767fc640 1 -- 172.21.15.12:0/2684695565 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55882a0c3e30 con 0x55882a0b1fe0 2024-02-18T11:40:34.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.618+0000 7f99767fc640 1 -- 172.21.15.12:0/2684695565 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f99880f1170 con 0x55882a0b1fe0 2024-02-18T11:40:34.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.618+0000 7f9997f7e740 10 client.15240 fetch_fsmap finished waiting for FSMap version 20 2024-02-18T11:40:34.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.618+0000 7f9997f7e740 10 client.15240 resolve_mds: resolved name 'b' to daemon mds.b 2024-02-18T11:40:34.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.618+0000 7f9997f7e740 20 client.15240 populate_metadata read hostname 'smithi012' 2024-02-18T11:40:34.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.618+0000 7f9997f7e740 1 --2- 172.21.15.12:0/2684695565 >> [v2:172.21.15.158:6838/1127200849,v1:172.21.15.158:6839/1127200849] conn(0x55882a11e0e0 0x55882a11a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:34.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.618+0000 7f9997f7e740 4 client.15240 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "status"}] 2024-02-18T11:40:34.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.618+0000 7f9997f7e740 1 -- 172.21.15.12:0/2684695565 --> [v2:172.21.15.158:6838/1127200849,v1:172.21.15.158:6839/1127200849] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f9978011c00 con 0x55882a11e0e0 2024-02-18T11:40:34.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.618+0000 7f998effd640 1 --2- 172.21.15.12:0/2684695565 >> [v2:172.21.15.158:6838/1127200849,v1:172.21.15.158:6839/1127200849] conn(0x55882a11e0e0 0x55882a11a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:34.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.618+0000 7f99767fc640 10 client.15240 ms_handle_connect on v2:172.21.15.158:6838/1127200849 2024-02-18T11:40:34.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.618+0000 7f998effd640 1 --2- 172.21.15.12:0/2684695565 >> [v2:172.21.15.158:6838/1127200849,v1:172.21.15.158:6839/1127200849] conn(0x55882a11e0e0 0x55882a11a5f0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89afa05cabd6091 server_cookie=8f89cf4e02ed3553 in_seq=0 out_seq=0 2024-02-18T11:40:34.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.619+0000 7f99767fc640 1 -- 172.21.15.12:0/2684695565 <== mds.0 v2:172.21.15.158:6838/1127200849 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x7f9978011c00 con 0x55882a11e0e0 2024-02-18T11:40:34.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.619+0000 7f99767fc640 10 client.15240 handle_command_reply: tid=0 2024-02-18T11:40:34.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.619+0000 7f9997f7e740 1 client.15240 shutdown 2024-02-18T11:40:34.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.619+0000 7f9997f7e740 1 -- 172.21.15.12:0/2684695565 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55882a0b1fe0 msgr2=0x55882a118a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:34.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.619+0000 7f9997f7e740 1 --2- 172.21.15.12:0/2684695565 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55882a0b1fe0 0x55882a118a00 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f9988034f50 tx=0x7f99880bda80 comp rx=0 tx=0).stop 2024-02-18T11:40:34.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.619+0000 7f9997f7e740 1 -- 172.21.15.12:0/2684695565 shutdown_connections 2024-02-18T11:40:34.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.619+0000 7f9997f7e740 1 --2- 172.21.15.12:0/2684695565 >> [v2:172.21.15.158:6838/1127200849,v1:172.21.15.158:6839/1127200849] conn(0x55882a11e0e0 0x55882a11a5f0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.619+0000 7f9997f7e740 1 --2- 172.21.15.12:0/2684695565 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55882a0ace70 0x55882a0badd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.619+0000 7f9997f7e740 1 --2- 172.21.15.12:0/2684695565 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55882a0b1fe0 0x55882a118a00 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.619+0000 7f99767fc640 0 client.15240 ms_handle_reset on v2:172.21.15.158:6838/1127200849 2024-02-18T11:40:34.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.619+0000 7f9997f7e740 1 --2- 172.21.15.12:0/2684695565 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55882a0ac480 0x55882a0ba890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.619+0000 7f9997f7e740 1 -- 172.21.15.12:0/2684695565 >> 172.21.15.12:0/2684695565 conn(0x55882a0a35e0 msgr2=0x55882a0b50f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:34.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.619+0000 7f9997f7e740 1 -- 172.21.15.12:0/2684695565 shutdown_connections 2024-02-18T11:40:34.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.619+0000 7f9997f7e740 1 -- 172.21.15.12:0/2684695565 wait complete. 2024-02-18T11:40:34.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.619+0000 7f9997f7e740 20 client.15240 trim_cache size 0 max 16384 2024-02-18T11:40:34.621 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-02-18T11:40:34.621 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_fsid": "0c8bb0de-ce51-11ee-95ba-87774f69a715", 2024-02-18T11:40:34.621 INFO:teuthology.orchestra.run.smithi012.stdout: "whoami": 0, 2024-02-18T11:40:34.621 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 24524, 2024-02-18T11:40:34.621 INFO:teuthology.orchestra.run.smithi012.stdout: "want_state": "up:active", 2024-02-18T11:40:34.621 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "up:active", 2024-02-18T11:40:34.621 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs", 2024-02-18T11:40:34.621 INFO:teuthology.orchestra.run.smithi012.stdout: "rank_uptime": 106.78177013299999, 2024-02-18T11:40:34.621 INFO:teuthology.orchestra.run.smithi012.stdout: "mdsmap_epoch": 20, 2024-02-18T11:40:34.621 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch": 89, 2024-02-18T11:40:34.621 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch_barrier": 87, 2024-02-18T11:40:34.621 INFO:teuthology.orchestra.run.smithi012.stdout: "uptime": 112.29872099799999 2024-02-18T11:40:34.621 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-02-18T11:40:34.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.619+0000 7f99952d4640 1 -- 172.21.15.12:0/3929252273 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f997c0a2750 msgr2=0x7f997c0a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:34.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.619+0000 7f99952d4640 1 --2- 172.21.15.12:0/3929252273 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f997c0a2750 0x7f997c0a4c10 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9978009e70 tx=0x7f9978014660 comp rx=0 tx=0).stop 2024-02-18T11:40:34.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.619+0000 7f99952d4640 1 -- 172.21.15.12:0/3929252273 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9990101c70 msgr2=0x7f999019e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:34.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.619+0000 7f99952d4640 1 --2- 172.21.15.12:0/3929252273 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9990101c70 0x7f999019e880 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f9988037040 tx=0x7f9988034a90 comp rx=0 tx=0).stop 2024-02-18T11:40:34.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.619+0000 7f99952d4640 1 -- 172.21.15.12:0/3929252273 shutdown_connections 2024-02-18T11:40:34.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.619+0000 7f99952d4640 1 --2- 172.21.15.12:0/3929252273 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f997c0a2750 0x7f997c0a4c10 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.619+0000 7f99952d4640 1 --2- 172.21.15.12:0/3929252273 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9990101c70 0x7f999019e880 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.619+0000 7f99952d4640 1 --2- 172.21.15.12:0/3929252273 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f99901012b0 0x7f9990197810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.619+0000 7f99952d4640 1 --2- 172.21.15.12:0/3929252273 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99901000c0 0x7f99901972d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:34.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.619+0000 7f99952d4640 1 -- 172.21.15.12:0/3929252273 >> 172.21.15.12:0/3929252273 conn(0x7f99900fb650 msgr2=0x7f99900fd0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:34.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.619+0000 7f99952d4640 1 -- 172.21.15.12:0/3929252273 shutdown_connections 2024-02-18T11:40:34.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:34.619+0000 7f99952d4640 1 -- 172.21.15.12:0/3929252273 wait complete. 2024-02-18T11:40:34.624 INFO:teuthology.run_tasks:Running task kclient... 2024-02-18T11:40:34.649 INFO:tasks.kclient:Mounting kernel clients... 2024-02-18T11:40:34.649 INFO:tasks.kclient:config is {'client.0': {}} 2024-02-18T11:40:34.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:40:34.650 DEBUG:teuthology.orchestra.run.smithi012:> ip netns list 2024-02-18T11:40:34.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:40:34.665 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link delete ceph-brx 2024-02-18T11:40:34.733 INFO:teuthology.orchestra.run.smithi012.stderr:Cannot find device "ceph-brx" 2024-02-18T11:40:34.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-18T11:40:34.735 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'wsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-02-18T11:40:34.736 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-02-18T11:40:34.736 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-02-18T11:40:34.736 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph fs ls 2024-02-18T11:40:35.015 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T11:40:36.380 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:40:36 smithi012 ceph-mon[32460]: pgmap v341: 97 pgs: 97 active+clean; 640 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-02-18T11:40:36.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:40:36 smithi158 ceph-mon[36626]: pgmap v341: 97 pgs: 97 active+clean; 640 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-02-18T11:40:36.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:40:36 smithi188 ceph-mon[36925]: pgmap v341: 97 pgs: 97 active+clean; 640 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-02-18T11:40:36.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:36.772+0000 7f5401aad700 1 -- 172.21.15.12:0/1485332573 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f53fc100be0 msgr2=0x7f53fc101060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:36.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:36.772+0000 7f5401aad700 1 --2- 172.21.15.12:0/1485332573 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f53fc100be0 0x7f53fc101060 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f53e40099c0 tx=0x7f53e4009cd0 comp rx=0 tx=0).stop 2024-02-18T11:40:36.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:36.772+0000 7f5401aad700 1 -- 172.21.15.12:0/1485332573 shutdown_connections 2024-02-18T11:40:36.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:36.772+0000 7f5401aad700 1 --2- 172.21.15.12:0/1485332573 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f53fc1015a0 0x7f53fc10d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:36.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:36.772+0000 7f5401aad700 1 --2- 172.21.15.12:0/1485332573 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f53fc106c20 0x7f53fc107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:36.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:36.772+0000 7f5401aad700 1 --2- 172.21.15.12:0/1485332573 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f53fc100be0 0x7f53fc101060 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:36.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:36.772+0000 7f5401aad700 1 -- 172.21.15.12:0/1485332573 >> 172.21.15.12:0/1485332573 conn(0x7f53fc076c00 msgr2=0x7f53fc077010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:36.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:36.772+0000 7f5401aad700 1 -- 172.21.15.12:0/1485332573 shutdown_connections 2024-02-18T11:40:36.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:36.772+0000 7f5401aad700 1 -- 172.21.15.12:0/1485332573 wait complete. 2024-02-18T11:40:36.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:36.772+0000 7f5401aad700 1 Processor -- start 2024-02-18T11:40:36.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:36.773+0000 7f5401aad700 1 -- start start 2024-02-18T11:40:36.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:36.773+0000 7f5401aad700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f53fc100be0 0x7f53fc198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:36.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:36.773+0000 7f5401aad700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f53fc1015a0 0x7f53fc198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:36.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:36.773+0000 7f5401aad700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f53fc106c20 0x7f53fc19ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:36.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:36.773+0000 7f5401aad700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53fc19d030 con 0x7f53fc106c20 2024-02-18T11:40:36.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:36.773+0000 7f5401aad700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53fc19d1a0 con 0x7f53fc100be0 2024-02-18T11:40:36.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:36.773+0000 7f5401aad700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53fc19d360 con 0x7f53fc1015a0 2024-02-18T11:40:36.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:36.773+0000 7f53fb7fe700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f53fc100be0 0x7f53fc198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:36.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:36.773+0000 7f53fb7fe700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f53fc100be0 0x7f53fc198150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.158:3300/0 says I am v2:172.21.15.12:55314/0 (socket says 172.21.15.12:55314) 2024-02-18T11:40:36.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:36.773+0000 7f53fb7fe700 1 -- 172.21.15.12:0/2360525749 learned_addr learned my addr 172.21.15.12:0/2360525749 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:40:36.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:36.773+0000 7f53fbfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f53fc106c20 0x7f53fc19ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:36.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:36.773+0000 7f53faffd700 1 --2- 172.21.15.12:0/2360525749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f53fc1015a0 0x7f53fc198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:36.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:36.773+0000 7f53fbfff700 1 -- 172.21.15.12:0/2360525749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f53fc1015a0 msgr2=0x7f53fc198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:36.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:36.773+0000 7f53fbfff700 1 --2- 172.21.15.12:0/2360525749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f53fc1015a0 0x7f53fc198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:36.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:36.773+0000 7f53fbfff700 1 -- 172.21.15.12:0/2360525749 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f53fc100be0 msgr2=0x7f53fc198150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:36.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:36.773+0000 7f53fbfff700 1 --2- 172.21.15.12:0/2360525749 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f53fc100be0 0x7f53fc198150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:36.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:36.773+0000 7f53fbfff700 1 -- 172.21.15.12:0/2360525749 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53e40095f0 con 0x7f53fc106c20 2024-02-18T11:40:36.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:36.773+0000 7f53fb7fe700 1 --2- 172.21.15.12:0/2360525749 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f53fc100be0 0x7f53fc198150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-18T11:40:36.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:36.774+0000 7f53fbfff700 1 --2- 172.21.15.12:0/2360525749 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f53fc106c20 0x7f53fc19ca20 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f53f400b3f0 tx=0x7f53f400b700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:36.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:36.774+0000 7f53f8ff9700 1 -- 172.21.15.12:0/2360525749 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f4011630 con 0x7f53fc106c20 2024-02-18T11:40:36.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:36.774+0000 7f53f8ff9700 1 -- 172.21.15.12:0/2360525749 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f53f4011c70 con 0x7f53fc106c20 2024-02-18T11:40:36.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:36.774+0000 7f5401aad700 1 -- 172.21.15.12:0/2360525749 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53fc19d640 con 0x7f53fc106c20 2024-02-18T11:40:36.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:36.774+0000 7f53f8ff9700 1 -- 172.21.15.12:0/2360525749 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f4010d70 con 0x7f53fc106c20 2024-02-18T11:40:36.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:36.774+0000 7f5401aad700 1 -- 172.21.15.12:0/2360525749 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53fc1a4ea0 con 0x7f53fc106c20 2024-02-18T11:40:36.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:36.775+0000 7f5401aad700 1 -- 172.21.15.12:0/2360525749 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53fc068970 con 0x7f53fc106c20 2024-02-18T11:40:36.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:36.776+0000 7f53f8ff9700 1 -- 172.21.15.12:0/2360525749 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) v1 ==== 137245+0+0 (secure 0 0 0) 0x7f53f4010410 con 0x7f53fc106c20 2024-02-18T11:40:36.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:36.776+0000 7f53f8ff9700 1 --2- 172.21.15.12:0/2360525749 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f53e80a27e0 0x7f53e80a4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:36.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:36.776+0000 7f53fb7fe700 1 --2- 172.21.15.12:0/2360525749 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f53e80a27e0 0x7f53e80a4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:36.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:36.777+0000 7f53f8ff9700 1 -- 172.21.15.12:0/2360525749 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f53f40cdd90 con 0x7f53fc106c20 2024-02-18T11:40:36.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:36.777+0000 7f53fb7fe700 1 --2- 172.21.15.12:0/2360525749 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f53e80a27e0 0x7f53e80a4ca0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f53ec009fd0 tx=0x7f53ec009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:36.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:36.781+0000 7f53f8ff9700 1 -- 172.21.15.12:0/2360525749 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f53f4096b30 con 0x7f53fc106c20 2024-02-18T11:40:37.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:37.030+0000 7f5401aad700 1 -- 172.21.15.12:0/2360525749 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f53fc1994b0 con 0x7f53fc106c20 2024-02-18T11:40:37.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:37.030+0000 7f53f8ff9700 1 -- 172.21.15.12:0/2360525749 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v20) v1 ==== 53+0+73 (secure 0 0 0) 0x7f53f4010ed0 con 0x7f53fc106c20 2024-02-18T11:40:37.024 INFO:teuthology.orchestra.run.smithi012.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-02-18T11:40:37.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:37.032+0000 7f5401aad700 1 -- 172.21.15.12:0/2360525749 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f53e80a27e0 msgr2=0x7f53e80a4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:37.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:37.032+0000 7f5401aad700 1 --2- 172.21.15.12:0/2360525749 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f53e80a27e0 0x7f53e80a4ca0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f53ec009fd0 tx=0x7f53ec009040 comp rx=0 tx=0).stop 2024-02-18T11:40:37.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:37.032+0000 7f5401aad700 1 -- 172.21.15.12:0/2360525749 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f53fc106c20 msgr2=0x7f53fc19ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:37.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:37.032+0000 7f5401aad700 1 --2- 172.21.15.12:0/2360525749 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f53fc106c20 0x7f53fc19ca20 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f53f400b3f0 tx=0x7f53f400b700 comp rx=0 tx=0).stop 2024-02-18T11:40:37.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:37.032+0000 7f5401aad700 1 -- 172.21.15.12:0/2360525749 shutdown_connections 2024-02-18T11:40:37.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:37.032+0000 7f5401aad700 1 --2- 172.21.15.12:0/2360525749 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f53e80a27e0 0x7f53e80a4ca0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:37.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:37.032+0000 7f5401aad700 1 --2- 172.21.15.12:0/2360525749 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f53fc106c20 0x7f53fc19ca20 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:37.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:37.032+0000 7f5401aad700 1 --2- 172.21.15.12:0/2360525749 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f53fc100be0 0x7f53fc198150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:37.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:37.032+0000 7f5401aad700 1 --2- 172.21.15.12:0/2360525749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f53fc1015a0 0x7f53fc198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:37.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:37.032+0000 7f5401aad700 1 -- 172.21.15.12:0/2360525749 >> 172.21.15.12:0/2360525749 conn(0x7f53fc076c00 msgr2=0x7f53fc0ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:37.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:37.032+0000 7f5401aad700 1 -- 172.21.15.12:0/2360525749 shutdown_connections 2024-02-18T11:40:37.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:37.032+0000 7f5401aad700 1 -- 172.21.15.12:0/2360525749 wait complete. 2024-02-18T11:40:37.352 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:40:37 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/2360525749' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-18T11:40:37.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:40:37 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/2360525749' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-18T11:40:37.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:40:37 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/2360525749' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-18T11:40:37.784 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-02-18T11:40:37.784 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-18T11:40:37.784 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi012.front.sepia.ceph.com 2024-02-18T11:40:37.784 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-02-18T11:40:37.784 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-02-18T11:40:37.784 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-02-18T11:40:37.785 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-02-18T11:40:37.785 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-02-18T11:40:37.785 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-02-18T11:40:37.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:40:37.785 DEBUG:teuthology.orchestra.run.smithi012:> ip addr 2024-02-18T11:40:37.801 INFO:teuthology.orchestra.run.smithi012.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-18T11:40:37.801 INFO:teuthology.orchestra.run.smithi012.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-18T11:40:37.801 INFO:teuthology.orchestra.run.smithi012.stdout: inet 127.0.0.1/8 scope host lo 2024-02-18T11:40:37.801 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2024-02-18T11:40:37.801 INFO:teuthology.orchestra.run.smithi012.stdout: inet6 ::1/128 scope host 2024-02-18T11:40:37.801 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2024-02-18T11:40:37.802 INFO:teuthology.orchestra.run.smithi012.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-18T11:40:37.802 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether 0c:c4:7a:6b:fd:a4 brd ff:ff:ff:ff:ff:ff 2024-02-18T11:40:37.802 INFO:teuthology.orchestra.run.smithi012.stdout: altname enp7s0f0 2024-02-18T11:40:37.802 INFO:teuthology.orchestra.run.smithi012.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-18T11:40:37.802 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether 0c:c4:7a:6b:fd:a5 brd ff:ff:ff:ff:ff:ff 2024-02-18T11:40:37.802 INFO:teuthology.orchestra.run.smithi012.stdout: altname enp7s0f1 2024-02-18T11:40:37.802 INFO:teuthology.orchestra.run.smithi012.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-18T11:40:37.802 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether ac:1f:6b:f8:25:ea brd ff:ff:ff:ff:ff:ff 2024-02-18T11:40:37.802 INFO:teuthology.orchestra.run.smithi012.stdout: altname enp1s0f0 2024-02-18T11:40:37.802 INFO:teuthology.orchestra.run.smithi012.stdout: inet 172.21.15.12/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-02-18T11:40:37.802 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft 42170sec preferred_lft 42170sec 2024-02-18T11:40:37.802 INFO:teuthology.orchestra.run.smithi012.stdout: inet6 fe80::ae1f:6bff:fef8:25ea/64 scope link noprefixroute 2024-02-18T11:40:37.802 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2024-02-18T11:40:37.802 INFO:teuthology.orchestra.run.smithi012.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-18T11:40:37.802 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether ac:1f:6b:f8:25:eb brd ff:ff:ff:ff:ff:ff 2024-02-18T11:40:37.803 INFO:teuthology.orchestra.run.smithi012.stdout: altname enp1s0f1 2024-02-18T11:40:37.803 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-02-18T11:40:37.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:40:37.804 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec bash -c ' 2024-02-18T11:40:37.804 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2024-02-18T11:40:37.804 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link add name ceph-brx type bridge 2024-02-18T11:40:37.804 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip addr flush dev ceph-brx 2024-02-18T11:40:37.804 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link set ceph-brx up 2024-02-18T11:40:37.804 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-02-18T11:40:37.804 DEBUG:teuthology.orchestra.run.smithi012:> ') 2024-02-18T11:40:37.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:40:37.977 DEBUG:teuthology.orchestra.run.smithi012:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-18T11:40:38.004 INFO:teuthology.orchestra.run.smithi012.stdout:1 2024-02-18T11:40:38.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:40:38.006 DEBUG:teuthology.orchestra.run.smithi012:> route 2024-02-18T11:40:38.082 INFO:teuthology.orchestra.run.smithi012.stdout:Kernel IP routing table 2024-02-18T11:40:38.082 INFO:teuthology.orchestra.run.smithi012.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-18T11:40:38.082 INFO:teuthology.orchestra.run.smithi012.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-02-18T11:40:38.082 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-02-18T11:40:38.082 INFO:teuthology.orchestra.run.smithi012.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-18T11:40:38.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:40:38.083 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec bash -c ' 2024-02-18T11:40:38.083 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2024-02-18T11:40:38.083 DEBUG:teuthology.orchestra.run.smithi012:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-02-18T11:40:38.083 DEBUG:teuthology.orchestra.run.smithi012:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-02-18T11:40:38.083 DEBUG:teuthology.orchestra.run.smithi012:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-02-18T11:40:38.083 DEBUG:teuthology.orchestra.run.smithi012:> ') 2024-02-18T11:40:38.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:40:38.336 DEBUG:teuthology.orchestra.run.smithi012:> ip netns list 2024-02-18T11:40:38.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:40:38.351 DEBUG:teuthology.orchestra.run.smithi012:> ip netns list-id 2024-02-18T11:40:38.406 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:40:38.406 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec bash -c ' 2024-02-18T11:40:38.406 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2024-02-18T11:40:38.406 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-18T11:40:38.406 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-02-18T11:40:38.406 DEBUG:teuthology.orchestra.run.smithi012:> ') 2024-02-18T11:40:38.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:40:38 smithi012 ceph-mon[32460]: pgmap v342: 97 pgs: 97 active+clean; 640 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 341 B/s wr, 7 op/s 2024-02-18T11:40:38.465 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-02-18T11:40:38.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:40:38.465 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec bash -c ' 2024-02-18T11:40:38.465 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2024-02-18T11:40:38.465 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-02-18T11:40:38.465 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-02-18T11:40:38.465 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-02-18T11:40:38.465 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-02-18T11:40:38.465 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-02-18T11:40:38.465 DEBUG:teuthology.orchestra.run.smithi012:> ') 2024-02-18T11:40:38.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:40:38 smithi158 ceph-mon[36626]: pgmap v342: 97 pgs: 97 active+clean; 640 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 341 B/s wr, 7 op/s 2024-02-18T11:40:38.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:40:38.604 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec bash -c ' 2024-02-18T11:40:38.604 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2024-02-18T11:40:38.604 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link set brx.0 up 2024-02-18T11:40:38.604 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link set dev brx.0 master ceph-brx 2024-02-18T11:40:38.604 DEBUG:teuthology.orchestra.run.smithi012:> ') 2024-02-18T11:40:38.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:40:38 smithi188 ceph-mon[36925]: pgmap v342: 97 pgs: 97 active+clean; 640 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 341 B/s wr, 7 op/s 2024-02-18T11:40:38.646 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-18T11:40:38.646 DEBUG:teuthology.orchestra.run.smithi012:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-02-18T11:40:38.704 INFO:teuthology.orchestra.run.smithi012.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-02-18T11:40:38.705 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-18T11:40:38.705 DEBUG:teuthology.orchestra.run.smithi012:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-02-18T11:40:38.759 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-02-18T11:40:38.759 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-02-18T11:40:38.759 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/5fe0f653-6870-4e9c-aa7a-5f0a419fa44a 2024-02-18T11:40:38.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:40:38.759 DEBUG:teuthology.orchestra.run.smithi012:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/5fe0f653-6870-4e9c-aa7a-5f0a419fa44a /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,wsync,nofallback 2024-02-18T11:40:38.843 INFO:teuthology.orchestra.run.smithi012.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,wsync,nofallback 2024-02-18T11:40:38.843 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,wsync". 2024-02-18T11:40:38.965 INFO:teuthology.orchestra.run.smithi012.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,wsync,nofallback 2024-02-18T11:40:38.965 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,wsync". 2024-02-18T11:40:38.965 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: resolved to: "172.21.15.12:6789,172.21.15.158:6789,172.21.15.188:6789" 2024-02-18T11:40:38.965 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: trying mount with new device syntax: 0@0c8bb0de-ce51-11ee-95ba-87774f69a715.cephfs=/volumes/qa/sv_1/5fe0f653-6870-4e9c-aa7a-5f0a419fa44a 2024-02-18T11:40:38.965 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,wsync,name=0,key=0,mon_addr=172.21.15.12:6789/172.21.15.158:6789/172.21.15.188:6789" will pass to kernel 2024-02-18T11:40:38.966 INFO:teuthology.orchestra.run.smithi012.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-02-18T11:40:38.966 INFO:teuthology.orchestra.run.smithi012.stdout: You just mounted a file system that supports labels which does not 2024-02-18T11:40:38.966 INFO:teuthology.orchestra.run.smithi012.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-02-18T11:40:38.966 INFO:teuthology.orchestra.run.smithi012.stdout: applications will generate AVC messages and not be allowed access to 2024-02-18T11:40:38.966 INFO:teuthology.orchestra.run.smithi012.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-02-18T11:40:38.967 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-02-18T11:40:38.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:40:38.968 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-02-18T11:40:39.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:40:39.037 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-18T11:40:39.093 INFO:teuthology.orchestra.run.smithi012.stdout:client24919 2024-02-18T11:40:39.093 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-02-18T11:40:39.093 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-18T11:40:39.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:40:39.149 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-18T11:40:39.206 INFO:teuthology.orchestra.run.smithi012.stdout:client24919 2024-02-18T11:40:39.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:40:39.207 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-18T11:40:39.263 INFO:teuthology.orchestra.run.smithi012.stdout:client24919 2024-02-18T11:40:39.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:40:39.264 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715.client24919/status') 2024-02-18T11:40:39.330 INFO:teuthology.orchestra.run.smithi012.stdout:instance: client.24919 (3)172.21.15.12:0/1205966129 2024-02-18T11:40:39.330 INFO:teuthology.orchestra.run.smithi012.stdout:blocklisted: false 2024-02-18T11:40:39.332 INFO:teuthology.orchestra.run.smithi012.stderr:0+1 records in 2024-02-18T11:40:39.332 INFO:teuthology.orchestra.run.smithi012.stderr:0+1 records out 2024-02-18T11:40:39.332 INFO:teuthology.orchestra.run.smithi012.stderr:71 bytes copied, 2.136e-05 s, 3.3 MB/s 2024-02-18T11:40:39.333 INFO:teuthology.run_tasks:Running task exec... 2024-02-18T11:40:39.348 INFO:teuthology.task.exec:Executing custom commands... 2024-02-18T11:40:39.348 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi012.front.sepia.ceph.com 2024-02-18T11:40:39.348 DEBUG:teuthology.orchestra.run.smithi012:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-02-18T11:40:39.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:39.481+0000 7f9db132e640 1 -- 172.21.15.12:0/783473205 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9dac1012f0 msgr2=0x7f9dac101770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:39.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:39.481+0000 7f9db132e640 1 --2- 172.21.15.12:0/783473205 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9dac1012f0 0x7f9dac101770 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f9d94009900 tx=0x7f9d9402bc30 comp rx=0 tx=0).stop 2024-02-18T11:40:39.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:39.481+0000 7f9db132e640 1 -- 172.21.15.12:0/783473205 shutdown_connections 2024-02-18T11:40:39.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:39.481+0000 7f9db132e640 1 --2- 172.21.15.12:0/783473205 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9dac101cb0 0x7f9dac108540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:39.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:39.481+0000 7f9db132e640 1 --2- 172.21.15.12:0/783473205 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9dac1012f0 0x7f9dac101770 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:39.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:39.481+0000 7f9db132e640 1 --2- 172.21.15.12:0/783473205 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9dac100100 0x7f9dac100520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:39.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:39.481+0000 7f9db132e640 1 -- 172.21.15.12:0/783473205 >> 172.21.15.12:0/783473205 conn(0x7f9dac0fb670 msgr2=0x7f9dac0fdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:39.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:39.481+0000 7f9db132e640 1 -- 172.21.15.12:0/783473205 shutdown_connections 2024-02-18T11:40:39.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:39.481+0000 7f9db132e640 1 -- 172.21.15.12:0/783473205 wait complete. 2024-02-18T11:40:39.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:39.482+0000 7f9db132e640 1 Processor -- start 2024-02-18T11:40:39.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:39.482+0000 7f9db132e640 1 -- start start 2024-02-18T11:40:39.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:39.482+0000 7f9db132e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9dac100100 0x7f9dac1972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:39.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:39.482+0000 7f9db132e640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9dac1012f0 0x7f9dac1977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:39.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:39.482+0000 7f9db132e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9dac101cb0 0x7f9dac19e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:39.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:39.482+0000 7f9db132e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dac19ed90 con 0x7f9dac100100 2024-02-18T11:40:39.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:39.482+0000 7f9db132e640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dac19ef00 con 0x7f9dac1012f0 2024-02-18T11:40:39.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:39.482+0000 7f9db132e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dac19f0c0 con 0x7f9dac101cb0 2024-02-18T11:40:39.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:39.482+0000 7f9daaffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9dac100100 0x7f9dac1972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:39.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:39.483+0000 7f9daaffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9dac100100 0x7f9dac1972a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48888/0 (socket says 172.21.15.12:48888) 2024-02-18T11:40:39.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:39.483+0000 7f9daaffd640 1 -- 172.21.15.12:0/4212436032 learned_addr learned my addr 172.21.15.12:0/4212436032 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:40:39.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:39.483+0000 7f9da27fc640 1 --2- 172.21.15.12:0/4212436032 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9dac1012f0 0x7f9dac1977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:39.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:39.483+0000 7f9dab7fe640 1 --2- 172.21.15.12:0/4212436032 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9dac101cb0 0x7f9dac19e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:39.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:39.483+0000 7f9daaffd640 1 -- 172.21.15.12:0/4212436032 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9dac101cb0 msgr2=0x7f9dac19e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:39.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:39.483+0000 7f9daaffd640 1 --2- 172.21.15.12:0/4212436032 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9dac101cb0 0x7f9dac19e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:39.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:39.483+0000 7f9daaffd640 1 -- 172.21.15.12:0/4212436032 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9dac1012f0 msgr2=0x7f9dac1977e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:39.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:39.483+0000 7f9daaffd640 1 --2- 172.21.15.12:0/4212436032 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9dac1012f0 0x7f9dac1977e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:39.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:39.483+0000 7f9daaffd640 1 -- 172.21.15.12:0/4212436032 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d94009560 con 0x7f9dac100100 2024-02-18T11:40:39.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:39.483+0000 7f9dab7fe640 1 --2- 172.21.15.12:0/4212436032 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9dac101cb0 0x7f9dac19e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:40:39.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:39.483+0000 7f9daaffd640 1 --2- 172.21.15.12:0/4212436032 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9dac100100 0x7f9dac1972a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c00b450 tx=0x7f9d9c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:39.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:39.483+0000 7f9da8ff9640 1 -- 172.21.15.12:0/4212436032 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c011890 con 0x7f9dac100100 2024-02-18T11:40:39.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:39.483+0000 7f9da8ff9640 1 -- 172.21.15.12:0/4212436032 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9d9c0103e0 con 0x7f9dac100100 2024-02-18T11:40:39.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:39.483+0000 7f9db132e640 1 -- 172.21.15.12:0/4212436032 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dac19f3a0 con 0x7f9dac100100 2024-02-18T11:40:39.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:39.483+0000 7f9da8ff9640 1 -- 172.21.15.12:0/4212436032 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c00f580 con 0x7f9dac100100 2024-02-18T11:40:39.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:39.483+0000 7f9db132e640 1 -- 172.21.15.12:0/4212436032 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dac19f900 con 0x7f9dac100100 2024-02-18T11:40:39.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:39.484+0000 7f9db132e640 1 -- 172.21.15.12:0/4212436032 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dac10bb70 con 0x7f9dac100100 2024-02-18T11:40:39.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:39.485+0000 7f9da8ff9640 1 -- 172.21.15.12:0/4212436032 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) v1 ==== 137245+0+0 (secure 0 0 0) 0x7f9d9c0108e0 con 0x7f9dac100100 2024-02-18T11:40:39.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:39.486+0000 7f9da8ff9640 1 --2- 172.21.15.12:0/4212436032 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f9d980a27f0 0x7f9d980a4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:39.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:39.486+0000 7f9da8ff9640 1 -- 172.21.15.12:0/4212436032 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9d9c0ce340 con 0x7f9dac100100 2024-02-18T11:40:39.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:39.486+0000 7f9da27fc640 1 --2- 172.21.15.12:0/4212436032 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f9d980a27f0 0x7f9d980a4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:39.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:39.486+0000 7f9da27fc640 1 --2- 172.21.15.12:0/4212436032 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f9d980a27f0 0x7f9d980a4cb0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9d94009230 tx=0x7f9d9402c740 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:39.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:39.490+0000 7f9da8ff9640 1 -- 172.21.15.12:0/4212436032 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9d9c097080 con 0x7f9dac100100 2024-02-18T11:40:39.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:39.711+0000 7f9db132e640 1 -- 172.21.15.12:0/4212436032 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) v1 -- 0x7f9dac10bd00 con 0x7f9dac100100 2024-02-18T11:40:40.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:40.206+0000 7f9da8ff9640 1 -- 172.21.15.12:0/4212436032 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v21) v1 ==== 118+0+0 (secure 0 0 0) 0x7f9d9c09a740 con 0x7f9dac100100 2024-02-18T11:40:40.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:40.208+0000 7f9db132e640 1 -- 172.21.15.12:0/4212436032 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f9d980a27f0 msgr2=0x7f9d980a4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:40.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:40.208+0000 7f9db132e640 1 --2- 172.21.15.12:0/4212436032 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f9d980a27f0 0x7f9d980a4cb0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9d94009230 tx=0x7f9d9402c740 comp rx=0 tx=0).stop 2024-02-18T11:40:40.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:40.208+0000 7f9db132e640 1 -- 172.21.15.12:0/4212436032 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9dac100100 msgr2=0x7f9dac1972a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:40.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:40.208+0000 7f9db132e640 1 --2- 172.21.15.12:0/4212436032 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9dac100100 0x7f9dac1972a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c00b450 tx=0x7f9d9c00b910 comp rx=0 tx=0).stop 2024-02-18T11:40:40.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:40.209+0000 7f9db132e640 1 -- 172.21.15.12:0/4212436032 shutdown_connections 2024-02-18T11:40:40.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:40.209+0000 7f9db132e640 1 --2- 172.21.15.12:0/4212436032 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f9d980a27f0 0x7f9d980a4cb0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:40.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:40.209+0000 7f9db132e640 1 --2- 172.21.15.12:0/4212436032 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9dac101cb0 0x7f9dac19e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:40.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:40.209+0000 7f9db132e640 1 --2- 172.21.15.12:0/4212436032 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9dac1012f0 0x7f9dac1977e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:40.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:40.209+0000 7f9db132e640 1 --2- 172.21.15.12:0/4212436032 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9dac100100 0x7f9dac1972a0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:40.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:40.209+0000 7f9db132e640 1 -- 172.21.15.12:0/4212436032 >> 172.21.15.12:0/4212436032 conn(0x7f9dac0fb670 msgr2=0x7f9dac0fd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:40.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:40.209+0000 7f9db132e640 1 -- 172.21.15.12:0/4212436032 shutdown_connections 2024-02-18T11:40:40.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:40.209+0000 7f9db132e640 1 -- 172.21.15.12:0/4212436032 wait complete. 2024-02-18T11:40:40.215 DEBUG:teuthology.orchestra.run.smithi012:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-02-18T11:40:40.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:40.329+0000 7f323622e640 1 -- 172.21.15.12:0/2336189079 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3230101cb0 msgr2=0x7f32301040a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:40.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:40.329+0000 7f323622e640 1 --2- 172.21.15.12:0/2336189079 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3230101cb0 0x7f32301040a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f3220009900 tx=0x7f322002bc30 comp rx=0 tx=0).stop 2024-02-18T11:40:40.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:40.329+0000 7f323622e640 1 -- 172.21.15.12:0/2336189079 shutdown_connections 2024-02-18T11:40:40.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:40.329+0000 7f323622e640 1 --2- 172.21.15.12:0/2336189079 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32301045e0 0x7f32301069d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:40.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:40.329+0000 7f323622e640 1 --2- 172.21.15.12:0/2336189079 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3230106f10 0x7f3230109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:40.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:40.329+0000 7f323622e640 1 --2- 172.21.15.12:0/2336189079 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3230101cb0 0x7f32301040a0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:40.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:40.329+0000 7f323622e640 1 -- 172.21.15.12:0/2336189079 >> 172.21.15.12:0/2336189079 conn(0x7f32300fb6a0 msgr2=0x7f32300fdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:40.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:40.329+0000 7f323622e640 1 -- 172.21.15.12:0/2336189079 shutdown_connections 2024-02-18T11:40:40.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:40.329+0000 7f323622e640 1 -- 172.21.15.12:0/2336189079 wait complete. 2024-02-18T11:40:40.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:40.330+0000 7f323622e640 1 Processor -- start 2024-02-18T11:40:40.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:40.330+0000 7f323622e640 1 -- start start 2024-02-18T11:40:40.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:40.330+0000 7f323622e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3230101cb0 0x7f32301971f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:40.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:40.330+0000 7f323622e640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f32301045e0 0x7f3230197730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:40.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:40.330+0000 7f323622e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3230106f10 0x7f323019e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:40.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:40.330+0000 7f323622e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323019ece0 con 0x7f3230101cb0 2024-02-18T11:40:40.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:40.330+0000 7f323622e640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323019ee50 con 0x7f32301045e0 2024-02-18T11:40:40.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:40.330+0000 7f323622e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323019f010 con 0x7f3230106f10 2024-02-18T11:40:40.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:40.330+0000 7f322f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3230101cb0 0x7f32301971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:40.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:40.330+0000 7f322f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3230101cb0 0x7f32301971f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48912/0 (socket says 172.21.15.12:48912) 2024-02-18T11:40:40.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:40.330+0000 7f322f7fe640 1 -- 172.21.15.12:0/3080896847 learned_addr learned my addr 172.21.15.12:0/3080896847 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:40:40.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:40.330+0000 7f322ffff640 1 --2- 172.21.15.12:0/3080896847 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3230106f10 0x7f323019e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:40.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:40.330+0000 7f322f7fe640 1 -- 172.21.15.12:0/3080896847 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3230106f10 msgr2=0x7f323019e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:40.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:40.330+0000 7f322f7fe640 1 --2- 172.21.15.12:0/3080896847 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3230106f10 0x7f323019e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:40.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:40.330+0000 7f322f7fe640 1 -- 172.21.15.12:0/3080896847 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f32301045e0 msgr2=0x7f3230197730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:40.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:40.330+0000 7f322f7fe640 1 --2- 172.21.15.12:0/3080896847 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f32301045e0 0x7f3230197730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:40.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:40.331+0000 7f322f7fe640 1 -- 172.21.15.12:0/3080896847 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3220009560 con 0x7f3230101cb0 2024-02-18T11:40:40.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:40.331+0000 7f322ffff640 1 --2- 172.21.15.12:0/3080896847 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3230106f10 0x7f323019e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:40:40.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:40.331+0000 7f322f7fe640 1 --2- 172.21.15.12:0/3080896847 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3230101cb0 0x7f32301971f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3220005ab0 tx=0x7f32200042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:40.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:40.331+0000 7f322cff9640 1 -- 172.21.15.12:0/3080896847 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32200044e0 con 0x7f3230101cb0 2024-02-18T11:40:40.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:40.331+0000 7f322cff9640 1 -- 172.21.15.12:0/3080896847 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f32200054e0 con 0x7f3230101cb0 2024-02-18T11:40:40.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:40.331+0000 7f323622e640 1 -- 172.21.15.12:0/3080896847 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f323019f290 con 0x7f3230101cb0 2024-02-18T11:40:40.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:40.331+0000 7f322cff9640 1 -- 172.21.15.12:0/3080896847 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f322003e450 con 0x7f3230101cb0 2024-02-18T11:40:40.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:40.331+0000 7f323622e640 1 -- 172.21.15.12:0/3080896847 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f323019f870 con 0x7f3230101cb0 2024-02-18T11:40:40.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:40.332+0000 7f323622e640 1 -- 172.21.15.12:0/3080896847 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32300fd190 con 0x7f3230101cb0 2024-02-18T11:40:40.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:40.333+0000 7f322cff9640 1 -- 172.21.15.12:0/3080896847 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) v1 ==== 137245+0+0 (secure 0 0 0) 0x7f32200035f0 con 0x7f3230101cb0 2024-02-18T11:40:40.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:40.333+0000 7f322cff9640 1 --2- 172.21.15.12:0/3080896847 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f32180a2870 0x7f32180a4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:40:40.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:40.334+0000 7f322effd640 1 --2- 172.21.15.12:0/3080896847 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f32180a2870 0x7f32180a4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:40:40.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:40.334+0000 7f322cff9640 1 -- 172.21.15.12:0/3080896847 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f32200f0b00 con 0x7f3230101cb0 2024-02-18T11:40:40.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:40.334+0000 7f322effd640 1 --2- 172.21.15.12:0/3080896847 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f32180a2870 0x7f32180a4d30 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3230198700 tx=0x7f3224006c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:40:40.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:40.338+0000 7f322cff9640 1 -- 172.21.15.12:0/3080896847 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f32200f2e60 con 0x7f3230101cb0 2024-02-18T11:40:40.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:40:40 smithi158 ceph-mon[36626]: pgmap v343: 97 pgs: 97 active+clean; 640 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-18T11:40:40.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:40:40 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/4212436032' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-02-18T11:40:40.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:40.546+0000 7f323622e640 1 -- 172.21.15.12:0/3080896847 --> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) v1 -- 0x7f323002ce80 con 0x7f32180a2870 2024-02-18T11:40:40.545 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:40:40 smithi012 ceph-mon[32460]: pgmap v343: 97 pgs: 97 active+clean; 640 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-18T11:40:40.545 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:40:40 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/4212436032' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-02-18T11:40:40.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:40.567+0000 7f322cff9640 1 -- 172.21.15.12:0/3080896847 <== mgr.14152 v2:172.21.15.12:6800/1153736685 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3 (secure 0 0 0) 0x7f323002ce80 con 0x7f32180a2870 2024-02-18T11:40:40.561 INFO:teuthology.orchestra.run.smithi012.stdout:{} 2024-02-18T11:40:40.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:40.569+0000 7f323622e640 1 -- 172.21.15.12:0/3080896847 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f32180a2870 msgr2=0x7f32180a4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:40.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:40.569+0000 7f323622e640 1 --2- 172.21.15.12:0/3080896847 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f32180a2870 0x7f32180a4d30 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3230198700 tx=0x7f3224006c90 comp rx=0 tx=0).stop 2024-02-18T11:40:40.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:40.569+0000 7f323622e640 1 -- 172.21.15.12:0/3080896847 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3230101cb0 msgr2=0x7f32301971f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:40:40.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:40.569+0000 7f323622e640 1 --2- 172.21.15.12:0/3080896847 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3230101cb0 0x7f32301971f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3220005ab0 tx=0x7f32200042f0 comp rx=0 tx=0).stop 2024-02-18T11:40:40.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:40.569+0000 7f323622e640 1 -- 172.21.15.12:0/3080896847 shutdown_connections 2024-02-18T11:40:40.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:40.569+0000 7f323622e640 1 --2- 172.21.15.12:0/3080896847 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f32180a2870 0x7f32180a4d30 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:40.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:40.569+0000 7f323622e640 1 --2- 172.21.15.12:0/3080896847 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3230106f10 0x7f323019e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:40.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:40.569+0000 7f323622e640 1 --2- 172.21.15.12:0/3080896847 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f32301045e0 0x7f3230197730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:40.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:40.569+0000 7f323622e640 1 --2- 172.21.15.12:0/3080896847 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3230101cb0 0x7f32301971f0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:40:40.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:40.569+0000 7f323622e640 1 -- 172.21.15.12:0/3080896847 >> 172.21.15.12:0/3080896847 conn(0x7f32300fb6a0 msgr2=0x7f32301025f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:40:40.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:40.569+0000 7f323622e640 1 -- 172.21.15.12:0/3080896847 shutdown_connections 2024-02-18T11:40:40.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:40:40.569+0000 7f323622e640 1 -- 172.21.15.12:0/3080896847 wait complete. 2024-02-18T11:40:40.576 INFO:teuthology.run_tasks:Running task check-counter... 2024-02-18T11:40:40.587 INFO:teuthology.run_tasks:Running task workunit... 2024-02-18T11:40:40.659 INFO:tasks.workunit:Pulling workunits from ref 1c920afa8316ea21877f3186850254947531ea22 2024-02-18T11:40:40.660 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-02-18T11:40:40.660 INFO:tasks.workunit:timeout=3h 2024-02-18T11:40:40.660 INFO:tasks.workunit:cleanup=True 2024-02-18T11:40:40.660 DEBUG:teuthology.orchestra.run.smithi012:> stat -- /home/ubuntu/cephtest/mnt.0 2024-02-18T11:40:40.661 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:40:40 smithi188 ceph-mon[36925]: pgmap v343: 97 pgs: 97 active+clean; 640 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-18T11:40:40.661 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:40:40 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/4212436032' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-02-18T11:40:40.676 INFO:teuthology.orchestra.run.smithi012.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-02-18T11:40:40.676 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-02-18T11:40:40.676 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 77h/119d Inode: 1099511628283 Links: 2 2024-02-18T11:40:40.676 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-18T11:40:40.676 INFO:teuthology.orchestra.run.smithi012.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-02-18T11:40:40.677 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2024-02-18 11:38:52.966015348 +0000 2024-02-18T11:40:40.677 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2024-02-18 11:38:52.966015348 +0000 2024-02-18T11:40:40.677 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2024-02-18 11:40:39.039432580 +0000 2024-02-18T11:40:40.677 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: 2024-02-18 11:38:52.966015348 +0000 2024-02-18T11:40:40.677 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-02-18T11:40:40.677 DEBUG:teuthology.orchestra.run.smithi012:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-02-18T11:40:40.754 DEBUG:teuthology.orchestra.run.smithi012:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 1c920afa8316ea21877f3186850254947531ea22 2024-02-18T11:40:40.810 INFO:tasks.workunit.client.0.smithi012.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-02-18T11:40:40.922 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:40:40 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:40:40.550+0000 7fb34a383700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-18T11:40:40.923 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:40:40 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:40:40.550+0000 7fb34a383700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-18T11:40:40.923 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:40:40 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:40:40.550+0000 7fb34a383700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-18T11:40:40.923 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:40:40 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:40:40.550+0000 7fb34a383700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-18T11:40:40.923 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:40:40 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:40:40.550+0000 7fb34a383700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-18T11:40:41.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:40:41 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/4212436032' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-02-18T11:40:41.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:40:41 smithi158 ceph-mon[36626]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-18T11:40:41.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:40:41 smithi158 ceph-mon[36626]: from='client.15258 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:40:41.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:40:41 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/4212436032' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-02-18T11:40:41.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:40:41 smithi188 ceph-mon[36925]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-18T11:40:41.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:40:41 smithi188 ceph-mon[36925]: from='client.15258 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:40:41.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:40:41 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/4212436032' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-02-18T11:40:41.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:40:41 smithi012 ceph-mon[32460]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-18T11:40:41.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:40:41 smithi012 ceph-mon[32460]: from='client.15258 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T11:40:42.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:40:42 smithi158 ceph-mon[36626]: pgmap v344: 97 pgs: 97 active+clean; 652 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-02-18T11:40:42.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:40:42 smithi188 ceph-mon[36925]: pgmap v344: 97 pgs: 97 active+clean; 652 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-02-18T11:40:42.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:40:42 smithi012 ceph-mon[32460]: pgmap v344: 97 pgs: 97 active+clean; 652 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-02-18T11:40:43.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:40:43 smithi158 ceph-mon[36626]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-02-18T11:40:43.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:40:43 smithi188 ceph-mon[36925]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-02-18T11:40:43.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:40:43 smithi012 ceph-mon[32460]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-02-18T11:40:44.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:40:44 smithi158 ceph-mon[36626]: pgmap v345: 97 pgs: 97 active+clean; 653 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-02-18T11:40:44.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:40:44 smithi188 ceph-mon[36925]: pgmap v345: 97 pgs: 97 active+clean; 653 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-02-18T11:40:44.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:40:44 smithi012 ceph-mon[32460]: pgmap v345: 97 pgs: 97 active+clean; 653 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-02-18T11:40:46.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:40:46 smithi158 ceph-mon[36626]: pgmap v346: 97 pgs: 97 active+clean; 653 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 2.0 KiB/s wr, 6 op/s 2024-02-18T11:40:46.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:40:46 smithi188 ceph-mon[36925]: pgmap v346: 97 pgs: 97 active+clean; 653 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 2.0 KiB/s wr, 6 op/s 2024-02-18T11:40:46.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:40:46 smithi012 ceph-mon[32460]: pgmap v346: 97 pgs: 97 active+clean; 653 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 2.0 KiB/s wr, 6 op/s 2024-02-18T11:40:48.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:40:48 smithi188 ceph-mon[36925]: pgmap v347: 97 pgs: 97 active+clean; 653 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-02-18T11:40:48.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:40:48 smithi012 ceph-mon[32460]: pgmap v347: 97 pgs: 97 active+clean; 653 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-02-18T11:40:48.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:40:48 smithi158 ceph-mon[36626]: pgmap v347: 97 pgs: 97 active+clean; 653 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-02-18T11:40:50.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:40:50 smithi188 ceph-mon[36925]: pgmap v348: 97 pgs: 97 active+clean; 653 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-02-18T11:40:50.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:40:50 smithi012 ceph-mon[32460]: pgmap v348: 97 pgs: 97 active+clean; 653 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-02-18T11:40:50.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:40:50 smithi158 ceph-mon[36626]: pgmap v348: 97 pgs: 97 active+clean; 653 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-02-18T11:40:52.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:40:52 smithi188 ceph-mon[36925]: pgmap v349: 97 pgs: 97 active+clean; 655 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-02-18T11:40:52.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:40:52 smithi012 ceph-mon[32460]: pgmap v349: 97 pgs: 97 active+clean; 655 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-02-18T11:40:52.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:40:52 smithi158 ceph-mon[36626]: pgmap v349: 97 pgs: 97 active+clean; 655 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-02-18T11:40:54.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:40:54 smithi188 ceph-mon[36925]: pgmap v350: 97 pgs: 97 active+clean; 655 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 853 B/s wr, 7 op/s 2024-02-18T11:40:54.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:40:54 smithi012 ceph-mon[32460]: pgmap v350: 97 pgs: 97 active+clean; 655 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 853 B/s wr, 7 op/s 2024-02-18T11:40:54.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:40:54 smithi158 ceph-mon[36626]: pgmap v350: 97 pgs: 97 active+clean; 655 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 853 B/s wr, 7 op/s 2024-02-18T11:40:56.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:40:56 smithi188 ceph-mon[36925]: pgmap v351: 97 pgs: 97 active+clean; 655 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 5 op/s 2024-02-18T11:40:56.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:40:56 smithi012 ceph-mon[32460]: pgmap v351: 97 pgs: 97 active+clean; 655 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 5 op/s 2024-02-18T11:40:56.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:40:56 smithi158 ceph-mon[36626]: pgmap v351: 97 pgs: 97 active+clean; 655 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 5 op/s 2024-02-18T11:40:58.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:40:58 smithi188 ceph-mon[36925]: pgmap v352: 97 pgs: 97 active+clean; 655 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 170 B/s wr, 6 op/s 2024-02-18T11:40:58.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:40:58 smithi012 ceph-mon[32460]: pgmap v352: 97 pgs: 97 active+clean; 655 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 170 B/s wr, 6 op/s 2024-02-18T11:40:58.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:40:58 smithi158 ceph-mon[36626]: pgmap v352: 97 pgs: 97 active+clean; 655 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 170 B/s wr, 6 op/s 2024-02-18T11:41:00.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:41:00 smithi188 ceph-mon[36925]: pgmap v353: 97 pgs: 97 active+clean; 655 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-02-18T11:41:00.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:41:00 smithi012 ceph-mon[32460]: pgmap v353: 97 pgs: 97 active+clean; 655 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-02-18T11:41:00.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:41:00 smithi158 ceph-mon[36626]: pgmap v353: 97 pgs: 97 active+clean; 655 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-02-18T11:41:02.842 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:41:02 smithi188 ceph-mon[36925]: pgmap v354: 97 pgs: 97 active+clean; 655 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 6 op/s 2024-02-18T11:41:02.842 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:41:02 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:41:02.909 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:41:02 smithi158 ceph-mon[36626]: pgmap v354: 97 pgs: 97 active+clean; 655 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 6 op/s 2024-02-18T11:41:02.909 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:41:02 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:41:02.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:41:02 smithi012 ceph-mon[32460]: pgmap v354: 97 pgs: 97 active+clean; 655 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 6 op/s 2024-02-18T11:41:02.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:41:02 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:41:04.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:41:04 smithi188 ceph-mon[36925]: pgmap v355: 97 pgs: 97 active+clean; 655 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-18T11:41:04.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:41:04 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:41:04.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:41:04 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:41:04.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:41:04 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:41:04.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:41:04 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:41:04.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:41:04 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:41:04.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:41:04 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:41:04.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:41:04 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:41:04.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:41:04 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:41:04.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:41:04 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:41:04.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:41:04 smithi012 ceph-mon[32460]: pgmap v355: 97 pgs: 97 active+clean; 655 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-18T11:41:04.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:41:04 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:41:04.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:41:04 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:41:04.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:41:04 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:41:04.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:41:04 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:41:04.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:41:04 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:41:04.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:41:04 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:41:04.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:41:04 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:41:04.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:41:04 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:41:04.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:41:04 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:41:04.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:41:04 smithi158 ceph-mon[36626]: pgmap v355: 97 pgs: 97 active+clean; 655 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-18T11:41:04.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:41:04 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:41:04.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:41:04 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:41:04.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:41:04 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:41:04.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:41:04 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:41:04.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:41:04 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:41:04.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:41:04 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:41:04.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:41:04 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:41:04.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:41:04 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:41:04.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:41:04 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:41:06.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:41:06 smithi188 ceph-mon[36925]: pgmap v356: 97 pgs: 97 active+clean; 655 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-18T11:41:06.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:41:06 smithi012 ceph-mon[32460]: pgmap v356: 97 pgs: 97 active+clean; 655 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-18T11:41:06.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:41:06 smithi158 ceph-mon[36626]: pgmap v356: 97 pgs: 97 active+clean; 655 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-18T11:41:08.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:41:08 smithi188 ceph-mon[36925]: pgmap v357: 97 pgs: 97 active+clean; 655 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-02-18T11:41:08.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:41:08 smithi012 ceph-mon[32460]: pgmap v357: 97 pgs: 97 active+clean; 655 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-02-18T11:41:08.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:41:08 smithi158 ceph-mon[36626]: pgmap v357: 97 pgs: 97 active+clean; 655 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-02-18T11:41:10.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:41:10 smithi188 ceph-mon[36925]: pgmap v358: 97 pgs: 97 active+clean; 655 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-18T11:41:10.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:41:10 smithi012 ceph-mon[32460]: pgmap v358: 97 pgs: 97 active+clean; 655 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-18T11:41:10.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:41:10 smithi158 ceph-mon[36626]: pgmap v358: 97 pgs: 97 active+clean; 655 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-18T11:41:12.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:41:12 smithi188 ceph-mon[36925]: pgmap v359: 97 pgs: 97 active+clean; 655 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-18T11:41:12.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:41:12 smithi012 ceph-mon[32460]: pgmap v359: 97 pgs: 97 active+clean; 655 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-18T11:41:12.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:41:12 smithi158 ceph-mon[36626]: pgmap v359: 97 pgs: 97 active+clean; 655 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-18T11:41:14.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:41:14 smithi188 ceph-mon[36925]: pgmap v360: 97 pgs: 97 active+clean; 655 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-18T11:41:14.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:41:14 smithi012 ceph-mon[32460]: pgmap v360: 97 pgs: 97 active+clean; 655 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-18T11:41:14.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:41:14 smithi158 ceph-mon[36626]: pgmap v360: 97 pgs: 97 active+clean; 655 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-18T11:41:16.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:41:16 smithi188 ceph-mon[36925]: pgmap v361: 97 pgs: 97 active+clean; 655 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-18T11:41:16.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:41:16 smithi012 ceph-mon[32460]: pgmap v361: 97 pgs: 97 active+clean; 655 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-18T11:41:16.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:41:16 smithi158 ceph-mon[36626]: pgmap v361: 97 pgs: 97 active+clean; 655 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-18T11:41:18.855 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:41:18 smithi188 ceph-mon[36925]: pgmap v362: 97 pgs: 97 active+clean; 655 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-02-18T11:41:18.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:41:18 smithi012 ceph-mon[32460]: pgmap v362: 97 pgs: 97 active+clean; 655 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-02-18T11:41:18.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:41:18 smithi158 ceph-mon[36626]: pgmap v362: 97 pgs: 97 active+clean; 655 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-02-18T11:41:20.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:41:20 smithi188 ceph-mon[36925]: pgmap v363: 97 pgs: 97 active+clean; 655 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-18T11:41:20.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:41:20 smithi012 ceph-mon[32460]: pgmap v363: 97 pgs: 97 active+clean; 655 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-18T11:41:20.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:41:20 smithi158 ceph-mon[36626]: pgmap v363: 97 pgs: 97 active+clean; 655 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-18T11:41:22.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:41:22 smithi012 ceph-mon[32460]: pgmap v364: 97 pgs: 97 active+clean; 661 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 6 op/s 2024-02-18T11:41:22.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:41:22 smithi158 ceph-mon[36626]: pgmap v364: 97 pgs: 97 active+clean; 661 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 6 op/s 2024-02-18T11:41:23.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:41:22 smithi188 ceph-mon[36925]: pgmap v364: 97 pgs: 97 active+clean; 661 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 6 op/s 2024-02-18T11:41:24.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:41:24 smithi012 ceph-mon[32460]: pgmap v365: 97 pgs: 97 active+clean; 661 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 682 B/s wr, 6 op/s 2024-02-18T11:41:24.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:41:24 smithi158 ceph-mon[36626]: pgmap v365: 97 pgs: 97 active+clean; 661 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 682 B/s wr, 6 op/s 2024-02-18T11:41:25.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:41:24 smithi188 ceph-mon[36925]: pgmap v365: 97 pgs: 97 active+clean; 661 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 682 B/s wr, 6 op/s 2024-02-18T11:41:26.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:41:26 smithi012 ceph-mon[32460]: pgmap v366: 97 pgs: 97 active+clean; 661 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 682 B/s wr, 5 op/s 2024-02-18T11:41:26.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:41:26 smithi158 ceph-mon[36626]: pgmap v366: 97 pgs: 97 active+clean; 661 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 682 B/s wr, 5 op/s 2024-02-18T11:41:27.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:41:26 smithi188 ceph-mon[36925]: pgmap v366: 97 pgs: 97 active+clean; 661 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 682 B/s wr, 5 op/s 2024-02-18T11:41:28.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:41:28 smithi012 ceph-mon[32460]: pgmap v367: 97 pgs: 97 active+clean; 661 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 6 op/s 2024-02-18T11:41:28.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:41:28 smithi158 ceph-mon[36626]: pgmap v367: 97 pgs: 97 active+clean; 661 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 6 op/s 2024-02-18T11:41:29.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:41:28 smithi188 ceph-mon[36925]: pgmap v367: 97 pgs: 97 active+clean; 661 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 6 op/s 2024-02-18T11:41:30.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:41:30 smithi012 ceph-mon[32460]: pgmap v368: 97 pgs: 97 active+clean; 661 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 682 B/s wr, 6 op/s 2024-02-18T11:41:30.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:41:30 smithi158 ceph-mon[36626]: pgmap v368: 97 pgs: 97 active+clean; 661 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 682 B/s wr, 6 op/s 2024-02-18T11:41:31.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:41:30 smithi188 ceph-mon[36925]: pgmap v368: 97 pgs: 97 active+clean; 661 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 682 B/s wr, 6 op/s 2024-02-18T11:41:32.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:41:32 smithi012 ceph-mon[32460]: pgmap v369: 97 pgs: 97 active+clean; 661 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 682 B/s wr, 6 op/s 2024-02-18T11:41:32.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:41:32 smithi158 ceph-mon[36626]: pgmap v369: 97 pgs: 97 active+clean; 661 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 682 B/s wr, 6 op/s 2024-02-18T11:41:33.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:41:32 smithi188 ceph-mon[36925]: pgmap v369: 97 pgs: 97 active+clean; 661 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 682 B/s wr, 6 op/s 2024-02-18T11:41:34.906 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:41:34 smithi188 ceph-mon[36925]: pgmap v370: 97 pgs: 97 active+clean; 661 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 85 B/s wr, 6 op/s 2024-02-18T11:41:34.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:41:34 smithi012 ceph-mon[32460]: pgmap v370: 97 pgs: 97 active+clean; 661 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 85 B/s wr, 6 op/s 2024-02-18T11:41:34.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:41:34 smithi158 ceph-mon[36626]: pgmap v370: 97 pgs: 97 active+clean; 661 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 85 B/s wr, 6 op/s 2024-02-18T11:41:36.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:41:36 smithi012 ceph-mon[32460]: pgmap v371: 97 pgs: 97 active+clean; 661 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-18T11:41:36.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:41:36 smithi158 ceph-mon[36626]: pgmap v371: 97 pgs: 97 active+clean; 661 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-18T11:41:37.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:41:36 smithi188 ceph-mon[36925]: pgmap v371: 97 pgs: 97 active+clean; 661 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-18T11:41:38.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:41:38 smithi012 ceph-mon[32460]: pgmap v372: 97 pgs: 97 active+clean; 661 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-02-18T11:41:38.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:41:38 smithi158 ceph-mon[36626]: pgmap v372: 97 pgs: 97 active+clean; 661 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-02-18T11:41:39.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:41:38 smithi188 ceph-mon[36925]: pgmap v372: 97 pgs: 97 active+clean; 661 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-02-18T11:41:40.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:41:40 smithi012 ceph-mon[32460]: pgmap v373: 97 pgs: 97 active+clean; 661 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-18T11:41:40.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:41:40 smithi158 ceph-mon[36626]: pgmap v373: 97 pgs: 97 active+clean; 661 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-18T11:41:41.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:41:40 smithi188 ceph-mon[36925]: pgmap v373: 97 pgs: 97 active+clean; 661 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-18T11:41:42.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:41:42 smithi012 ceph-mon[32460]: pgmap v374: 97 pgs: 97 active+clean; 661 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-18T11:41:42.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:41:42 smithi158 ceph-mon[36626]: pgmap v374: 97 pgs: 97 active+clean; 661 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-18T11:41:43.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:41:42 smithi188 ceph-mon[36925]: pgmap v374: 97 pgs: 97 active+clean; 661 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-18T11:41:44.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:41:44 smithi012 ceph-mon[32460]: pgmap v375: 97 pgs: 97 active+clean; 661 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-18T11:41:44.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:41:44 smithi158 ceph-mon[36626]: pgmap v375: 97 pgs: 97 active+clean; 661 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-18T11:41:45.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:41:44 smithi188 ceph-mon[36925]: pgmap v375: 97 pgs: 97 active+clean; 661 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-18T11:41:46.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:41:46 smithi012 ceph-mon[32460]: pgmap v376: 97 pgs: 97 active+clean; 661 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-18T11:41:46.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:41:46 smithi158 ceph-mon[36626]: pgmap v376: 97 pgs: 97 active+clean; 661 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-18T11:41:47.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:41:46 smithi188 ceph-mon[36925]: pgmap v376: 97 pgs: 97 active+clean; 661 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-18T11:41:48.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:41:48 smithi012 ceph-mon[32460]: pgmap v377: 97 pgs: 97 active+clean; 661 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-02-18T11:41:48.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:41:48 smithi158 ceph-mon[36626]: pgmap v377: 97 pgs: 97 active+clean; 661 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-02-18T11:41:49.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:41:48 smithi188 ceph-mon[36925]: pgmap v377: 97 pgs: 97 active+clean; 661 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-02-18T11:41:50.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:41:50 smithi012 ceph-mon[32460]: pgmap v378: 97 pgs: 97 active+clean; 661 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-18T11:41:50.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:41:50 smithi158 ceph-mon[36626]: pgmap v378: 97 pgs: 97 active+clean; 661 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-18T11:41:51.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:41:50 smithi188 ceph-mon[36925]: pgmap v378: 97 pgs: 97 active+clean; 661 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-18T11:41:52.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:41:52 smithi012 ceph-mon[32460]: pgmap v379: 97 pgs: 97 active+clean; 661 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-18T11:41:52.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:41:52 smithi158 ceph-mon[36626]: pgmap v379: 97 pgs: 97 active+clean; 661 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-18T11:41:53.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:41:52 smithi188 ceph-mon[36925]: pgmap v379: 97 pgs: 97 active+clean; 661 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-18T11:41:53.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:41:53 smithi012 ceph-mon[32460]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-02-18T11:41:53.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:41:53 smithi158 ceph-mon[36626]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-02-18T11:41:54.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:41:53 smithi188 ceph-mon[36925]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-02-18T11:41:54.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:41:54 smithi012 ceph-mon[32460]: pgmap v380: 97 pgs: 97 active+clean; 661 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-02-18T11:41:54.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:41:54 smithi158 ceph-mon[36626]: pgmap v380: 97 pgs: 97 active+clean; 661 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-02-18T11:41:55.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:41:54 smithi188 ceph-mon[36925]: pgmap v380: 97 pgs: 97 active+clean; 661 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-02-18T11:41:55.458 INFO:tasks.workunit.client.0.smithi012.stderr:Updating files: 96% (11803/12253) Updating files: 97% (11886/12253) Updating files: 98% (12008/12253) Updating files: 99% (12131/12253) Updating files: 100% (12253/12253) Updating files: 100% (12253/12253), done. 2024-02-18T11:41:55.920 INFO:tasks.workunit.client.0.smithi012.stderr:Note: switching to '1c920afa8316ea21877f3186850254947531ea22'. 2024-02-18T11:41:55.920 INFO:tasks.workunit.client.0.smithi012.stderr: 2024-02-18T11:41:55.920 INFO:tasks.workunit.client.0.smithi012.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-02-18T11:41:55.921 INFO:tasks.workunit.client.0.smithi012.stderr:changes and commit them, and you can discard any commits you make in this 2024-02-18T11:41:55.921 INFO:tasks.workunit.client.0.smithi012.stderr:state without impacting any branches by switching back to a branch. 2024-02-18T11:41:55.921 INFO:tasks.workunit.client.0.smithi012.stderr: 2024-02-18T11:41:55.921 INFO:tasks.workunit.client.0.smithi012.stderr:If you want to create a new branch to retain commits you create, you may 2024-02-18T11:41:55.921 INFO:tasks.workunit.client.0.smithi012.stderr:do so (now or later) by using -c with the switch command. Example: 2024-02-18T11:41:55.921 INFO:tasks.workunit.client.0.smithi012.stderr: 2024-02-18T11:41:55.921 INFO:tasks.workunit.client.0.smithi012.stderr: git switch -c 2024-02-18T11:41:55.921 INFO:tasks.workunit.client.0.smithi012.stderr: 2024-02-18T11:41:55.921 INFO:tasks.workunit.client.0.smithi012.stderr:Or undo this operation with: 2024-02-18T11:41:55.921 INFO:tasks.workunit.client.0.smithi012.stderr: 2024-02-18T11:41:55.921 INFO:tasks.workunit.client.0.smithi012.stderr: git switch - 2024-02-18T11:41:55.921 INFO:tasks.workunit.client.0.smithi012.stderr: 2024-02-18T11:41:55.921 INFO:tasks.workunit.client.0.smithi012.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-02-18T11:41:55.921 INFO:tasks.workunit.client.0.smithi012.stderr: 2024-02-18T11:41:55.921 INFO:tasks.workunit.client.0.smithi012.stderr:HEAD is now at 1c920afa831 Merge PR #52670 into wip-vshankar-testing-20240217.015652 2024-02-18T11:41:55.923 DEBUG:teuthology.orchestra.run.smithi012:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-02-18T11:41:55.984 INFO:tasks.workunit.client.0.smithi012.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-02-18T11:41:55.986 INFO:tasks.workunit.client.0.smithi012.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-02-18T11:41:55.986 INFO:tasks.workunit.client.0.smithi012.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-02-18T11:41:56.717 INFO:tasks.workunit.client.0.smithi012.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-02-18T11:41:56.784 INFO:tasks.workunit.client.0.smithi012.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-02-18T11:41:56.823 INFO:tasks.workunit.client.0.smithi012.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-02-18T11:41:56.825 INFO:tasks.workunit.client.0.smithi012.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-02-18T11:41:56.825 INFO:tasks.workunit.client.0.smithi012.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-02-18T11:41:56.863 INFO:tasks.workunit.client.0.smithi012.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-02-18T11:41:56.866 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-02-18T11:41:56.866 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-02-18T11:41:56.922 INFO:tasks.workunit:Running workunits matching suites/iogen.sh on client.0... 2024-02-18T11:41:56.923 INFO:tasks.workunit:Running workunit suites/iogen.sh... 2024-02-18T11:41:56.923 DEBUG:teuthology.orchestra.run.smithi012:workunit test suites/iogen.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1c920afa8316ea21877f3186850254947531ea22 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/iogen.sh 2024-02-18T11:41:56.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:41:56 smithi012 ceph-mon[32460]: pgmap v381: 97 pgs: 97 active+clean; 661 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-02-18T11:41:56.944 INFO:tasks.workunit.client.0.smithi012.stdout:getting iogen 2024-02-18T11:41:56.944 INFO:tasks.workunit.client.0.smithi012.stderr:+ echo 'getting iogen' 2024-02-18T11:41:56.944 INFO:tasks.workunit.client.0.smithi012.stderr:+ wget http://download.ceph.com/qa/iogen_3.1p0.tar 2024-02-18T11:41:56.965 INFO:tasks.workunit.client.0.smithi012.stderr:--2024-02-18 11:41:56-- http://download.ceph.com/qa/iogen_3.1p0.tar 2024-02-18T11:41:56.965 INFO:tasks.workunit.client.0.smithi012.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2024-02-18T11:41:56.987 INFO:tasks.workunit.client.0.smithi012.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2024-02-18T11:41:56.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:41:56 smithi158 ceph-mon[36626]: pgmap v381: 97 pgs: 97 active+clean; 661 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-02-18T11:41:57.007 INFO:tasks.workunit.client.0.smithi012.stderr:HTTP request sent, awaiting response... 200 OK 2024-02-18T11:41:57.007 INFO:tasks.workunit.client.0.smithi012.stderr:Length: 8071 (7.9K) [application/octet-stream] 2024-02-18T11:41:57.008 INFO:tasks.workunit.client.0.smithi012.stderr:Saving to: ‘iogen_3.1p0.tar’ 2024-02-18T11:41:57.011 INFO:tasks.workunit.client.0.smithi012.stderr: 2024-02-18T11:41:57.011 INFO:tasks.workunit.client.0.smithi012.stderr: 0K ....... 100% 1.40G=0s 2024-02-18T11:41:57.012 INFO:tasks.workunit.client.0.smithi012.stderr: 2024-02-18T11:41:57.012 INFO:tasks.workunit.client.0.smithi012.stderr:2024-02-18 11:41:57 (1.40 GB/s) - ‘iogen_3.1p0.tar’ saved [8071/8071] 2024-02-18T11:41:57.012 INFO:tasks.workunit.client.0.smithi012.stderr: 2024-02-18T11:41:57.012 INFO:tasks.workunit.client.0.smithi012.stderr:+ tar -xvzf iogen_3.1p0.tar 2024-02-18T11:41:57.026 INFO:tasks.workunit.client.0.smithi012.stdout:iogen_3.1p0 2024-02-18T11:41:57.027 INFO:tasks.workunit.client.0.smithi012.stdout:iogen_3.1p0/src 2024-02-18T11:41:57.028 INFO:tasks.workunit.client.0.smithi012.stdout:iogen_3.1p0/src/iogen.8 2024-02-18T11:41:57.029 INFO:tasks.workunit.client.0.smithi012.stdout:iogen_3.1p0/src/iogen.c 2024-02-18T11:41:57.030 INFO:tasks.workunit.client.0.smithi012.stdout:iogen_3.1p0/README 2024-02-18T11:41:57.031 INFO:tasks.workunit.client.0.smithi012.stdout:iogen_3.1p0/Makefile.bsd 2024-02-18T11:41:57.031 INFO:tasks.workunit.client.0.smithi012.stdout:iogen_3.1p0/Makefile 2024-02-18T11:41:57.033 INFO:tasks.workunit.client.0.smithi012.stderr:+ cd iogen_3.1p0 2024-02-18T11:41:57.033 INFO:tasks.workunit.client.0.smithi012.stdout:making iogen 2024-02-18T11:41:57.033 INFO:tasks.workunit.client.0.smithi012.stderr:+ echo 'making iogen' 2024-02-18T11:41:57.033 INFO:tasks.workunit.client.0.smithi012.stderr:+ make 2024-02-18T11:41:57.040 INFO:tasks.workunit.client.0.smithi012.stdout:gcc -D_FILE_OFFSET_BITS=64 -DVERSION=\"3.1p0\" -Dstrlcpy=strncpy -Darc4random=rand -O2 src/iogen.c -o iogen 2024-02-18T11:41:57.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:41:56 smithi188 ceph-mon[36925]: pgmap v381: 97 pgs: 97 active+clean; 661 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-02-18T11:41:57.286 INFO:tasks.workunit.client.0.smithi012.stdout:nroff -mandoc -Tascii src/iogen.8 > iogen.cat8 2024-02-18T11:41:57.373 INFO:tasks.workunit.client.0.smithi012.stdout:running iogen 2024-02-18T11:41:57.374 INFO:tasks.workunit.client.0.smithi012.stderr:+ echo 'running iogen' 2024-02-18T11:41:57.374 INFO:tasks.workunit.client.0.smithi012.stderr:+ ./iogen -n 5 -s 2g 2024-02-18T11:41:57.374 INFO:tasks.workunit.client.0.smithi012.stderr:child spawned 83476 2024-02-18T11:41:57.374 INFO:tasks.workunit.client.0.smithi012.stderr:child spawned 83477 2024-02-18T11:41:57.374 INFO:tasks.workunit.client.0.smithi012.stderr:child spawned 83478 2024-02-18T11:41:57.375 INFO:tasks.workunit.client.0.smithi012.stderr:child spawned 83479 2024-02-18T11:41:57.375 INFO:tasks.workunit.client.0.smithi012.stderr:child spawned 83480 2024-02-18T11:41:57.375 INFO:tasks.workunit.client.0.smithi012.stdout:sleep for 10 min 2024-02-18T11:41:57.375 INFO:tasks.workunit.client.0.smithi012.stderr:+ echo 'sleep for 10 min' 2024-02-18T11:41:57.375 INFO:tasks.workunit.client.0.smithi012.stderr:+ sleep 600 2024-02-18T11:41:59.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:41:58 smithi188 ceph-mon[36925]: pgmap v382: 97 pgs: 97 active+clean; 661 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 7 op/s 2024-02-18T11:41:59.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:41:58 smithi012 ceph-mon[32460]: pgmap v382: 97 pgs: 97 active+clean; 661 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 7 op/s 2024-02-18T11:41:59.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:41:58 smithi158 ceph-mon[36626]: pgmap v382: 97 pgs: 97 active+clean; 661 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 7 op/s 2024-02-18T11:42:01.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:42:00 smithi188 ceph-mon[36925]: pgmap v383: 97 pgs: 97 active+clean; 61 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5.0 MiB/s wr, 8 op/s 2024-02-18T11:42:01.172 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:42:00 smithi012 ceph-mon[32460]: pgmap v383: 97 pgs: 97 active+clean; 61 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5.0 MiB/s wr, 8 op/s 2024-02-18T11:42:01.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:42:00 smithi158 ceph-mon[36626]: pgmap v383: 97 pgs: 97 active+clean; 61 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5.0 MiB/s wr, 8 op/s 2024-02-18T11:42:03.172 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:42:02 smithi012 ceph-mon[32460]: pgmap v384: 97 pgs: 97 active+clean; 713 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 59 MiB/s wr, 93 op/s 2024-02-18T11:42:03.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:42:02 smithi158 ceph-mon[36626]: pgmap v384: 97 pgs: 97 active+clean; 713 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 59 MiB/s wr, 93 op/s 2024-02-18T11:42:03.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:42:02 smithi188 ceph-mon[36925]: pgmap v384: 97 pgs: 97 active+clean; 713 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 59 MiB/s wr, 93 op/s 2024-02-18T11:42:04.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:42:04 smithi188 ceph-mon[36925]: pgmap v385: 97 pgs: 97 active+clean; 2.9 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 246 MiB/s wr, 232 op/s 2024-02-18T11:42:04.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:42:04 smithi012 ceph-mon[32460]: pgmap v385: 97 pgs: 97 active+clean; 2.9 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 246 MiB/s wr, 232 op/s 2024-02-18T11:42:04.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:42:04 smithi158 ceph-mon[36626]: pgmap v385: 97 pgs: 97 active+clean; 2.9 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 246 MiB/s wr, 232 op/s 2024-02-18T11:42:05.423 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:42:05 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:42:05.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:42:05 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:42:05.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:42:05 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:42:06.744 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:42:06 smithi158 ceph-mon[36626]: pgmap v386: 97 pgs: 97 active+clean; 3.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 302 MiB/s wr, 245 op/s 2024-02-18T11:42:06.744 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:42:06 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:42:06.744 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:42:06 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:42:06.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:42:06 smithi188 ceph-mon[36925]: pgmap v386: 97 pgs: 97 active+clean; 3.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 302 MiB/s wr, 245 op/s 2024-02-18T11:42:06.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:42:06 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:42:06.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:42:06 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:42:06.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:42:06 smithi012 ceph-mon[32460]: pgmap v386: 97 pgs: 97 active+clean; 3.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 302 MiB/s wr, 245 op/s 2024-02-18T11:42:06.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:42:06 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:42:06.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:42:06 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:42:07.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:42:07 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:42:07.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:42:07 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:42:07.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:42:07 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:42:07.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:42:07 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:42:07.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:42:07 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:42:07.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:42:07 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:42:07.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:42:07 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:42:07.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:42:07 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:42:07.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:42:07 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:42:09.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:42:08 smithi188 ceph-mon[36925]: pgmap v387: 97 pgs: 97 active+clean; 6.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 530 MiB/s wr, 423 op/s 2024-02-18T11:42:09.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:42:08 smithi012 ceph-mon[32460]: pgmap v387: 97 pgs: 97 active+clean; 6.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 530 MiB/s wr, 423 op/s 2024-02-18T11:42:09.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:42:08 smithi158 ceph-mon[36626]: pgmap v387: 97 pgs: 97 active+clean; 6.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 530 MiB/s wr, 423 op/s 2024-02-18T11:42:11.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:42:10 smithi188 ceph-mon[36925]: pgmap v388: 97 pgs: 97 active+clean; 7.6 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 657 MiB/s wr, 512 op/s 2024-02-18T11:42:11.172 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:42:10 smithi012 ceph-mon[32460]: pgmap v388: 97 pgs: 97 active+clean; 7.6 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 657 MiB/s wr, 512 op/s 2024-02-18T11:42:11.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:42:10 smithi158 ceph-mon[36626]: pgmap v388: 97 pgs: 97 active+clean; 7.6 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 657 MiB/s wr, 512 op/s 2024-02-18T11:42:15.744 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:42:15 smithi158 ceph-mon[36626]: pgmap v389: 97 pgs: 97 active+clean; 8.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 694 MiB/s wr, 593 op/s 2024-02-18T11:42:15.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:42:15 smithi188 ceph-mon[36925]: pgmap v389: 97 pgs: 97 active+clean; 8.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 694 MiB/s wr, 593 op/s 2024-02-18T11:42:15.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:42:15 smithi012 ceph-mon[32460]: pgmap v389: 97 pgs: 97 active+clean; 8.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 694 MiB/s wr, 593 op/s 2024-02-18T11:42:16.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:42:16 smithi188 ceph-mon[36925]: pgmap v390: 97 pgs: 97 active+clean; 8.6 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 727 MiB/s wr, 814 op/s 2024-02-18T11:42:16.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:42:16 smithi188 ceph-mon[36925]: pgmap v391: 97 pgs: 97 active+clean; 8.7 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 549 MiB/s wr, 700 op/s 2024-02-18T11:42:16.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:42:16 smithi012 ceph-mon[32460]: pgmap v390: 97 pgs: 97 active+clean; 8.6 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 727 MiB/s wr, 814 op/s 2024-02-18T11:42:16.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:42:16 smithi012 ceph-mon[32460]: pgmap v391: 97 pgs: 97 active+clean; 8.7 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 549 MiB/s wr, 700 op/s 2024-02-18T11:42:16.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:42:16 smithi158 ceph-mon[36626]: pgmap v390: 97 pgs: 97 active+clean; 8.6 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 727 MiB/s wr, 814 op/s 2024-02-18T11:42:16.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:42:16 smithi158 ceph-mon[36626]: pgmap v391: 97 pgs: 97 active+clean; 8.7 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 549 MiB/s wr, 700 op/s 2024-02-18T11:42:19.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:42:18 smithi188 ceph-mon[36925]: pgmap v392: 97 pgs: 97 active+clean; 9.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 601 MiB/s wr, 1.01k op/s 2024-02-18T11:42:19.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:42:18 smithi012 ceph-mon[32460]: pgmap v392: 97 pgs: 97 active+clean; 9.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 601 MiB/s wr, 1.01k op/s 2024-02-18T11:42:19.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:42:18 smithi158 ceph-mon[36626]: pgmap v392: 97 pgs: 97 active+clean; 9.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 601 MiB/s wr, 1.01k op/s 2024-02-18T11:42:21.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:42:20 smithi188 ceph-mon[36925]: pgmap v393: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 424 MiB/s wr, 916 op/s 2024-02-18T11:42:21.172 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:42:20 smithi012 ceph-mon[32460]: pgmap v393: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 424 MiB/s wr, 916 op/s 2024-02-18T11:42:21.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:42:20 smithi158 ceph-mon[36626]: pgmap v393: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 424 MiB/s wr, 916 op/s 2024-02-18T11:42:23.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:42:22 smithi012 ceph-mon[32460]: pgmap v394: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 457 MiB/s wr, 1.16k op/s 2024-02-18T11:42:23.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:42:22 smithi158 ceph-mon[36626]: pgmap v394: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 457 MiB/s wr, 1.16k op/s 2024-02-18T11:42:23.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:42:22 smithi188 ceph-mon[36925]: pgmap v394: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 457 MiB/s wr, 1.16k op/s 2024-02-18T11:42:25.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:42:25 smithi012 ceph-mon[32460]: pgmap v395: 97 pgs: 97 active+clean; 10 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 574 MiB/s wr, 1.37k op/s 2024-02-18T11:42:25.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:42:25 smithi188 ceph-mon[36925]: pgmap v395: 97 pgs: 97 active+clean; 10 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 574 MiB/s wr, 1.37k op/s 2024-02-18T11:42:25.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:42:25 smithi158 ceph-mon[36626]: pgmap v395: 97 pgs: 97 active+clean; 10 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 574 MiB/s wr, 1.37k op/s 2024-02-18T11:42:26.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:42:26 smithi012 ceph-mon[32460]: pgmap v396: 97 pgs: 97 active+clean; 10 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 551 MiB/s wr, 1.17k op/s 2024-02-18T11:42:26.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:42:26 smithi158 ceph-mon[36626]: pgmap v396: 97 pgs: 97 active+clean; 10 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 551 MiB/s wr, 1.17k op/s 2024-02-18T11:42:26.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:42:26 smithi188 ceph-mon[36925]: pgmap v396: 97 pgs: 97 active+clean; 10 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 551 MiB/s wr, 1.17k op/s 2024-02-18T11:42:28.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:42:28 smithi158 ceph-mon[36626]: pgmap v397: 97 pgs: 97 active+clean; 10 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 799 MiB/s wr, 1.62k op/s 2024-02-18T11:42:28.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:42:28 smithi188 ceph-mon[36925]: pgmap v397: 97 pgs: 97 active+clean; 10 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 799 MiB/s wr, 1.62k op/s 2024-02-18T11:42:28.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:42:28 smithi012 ceph-mon[32460]: pgmap v397: 97 pgs: 97 active+clean; 10 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 799 MiB/s wr, 1.62k op/s 2024-02-18T11:42:30.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:42:30 smithi158 ceph-mon[36626]: pgmap v398: 97 pgs: 97 active+clean; 10 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 802 MiB/s wr, 1.64k op/s 2024-02-18T11:42:30.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:42:30 smithi188 ceph-mon[36925]: pgmap v398: 97 pgs: 97 active+clean; 10 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 802 MiB/s wr, 1.64k op/s 2024-02-18T11:42:30.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:42:30 smithi012 ceph-mon[32460]: pgmap v398: 97 pgs: 97 active+clean; 10 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 802 MiB/s wr, 1.64k op/s 2024-02-18T11:42:32.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:42:32 smithi158 ceph-mon[36626]: pgmap v399: 97 pgs: 97 active+clean; 10 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 911 MiB/s wr, 2.08k op/s 2024-02-18T11:42:33.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:42:32 smithi188 ceph-mon[36925]: pgmap v399: 97 pgs: 97 active+clean; 10 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 911 MiB/s wr, 2.08k op/s 2024-02-18T11:42:33.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:42:32 smithi012 ceph-mon[32460]: pgmap v399: 97 pgs: 97 active+clean; 10 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 911 MiB/s wr, 2.08k op/s 2024-02-18T11:42:35.172 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:42:34 smithi012 ceph-mon[32460]: pgmap v400: 97 pgs: 97 active+clean; 10 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 909 MiB/s wr, 2.00k op/s 2024-02-18T11:42:35.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:42:34 smithi158 ceph-mon[36626]: pgmap v400: 97 pgs: 97 active+clean; 10 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 909 MiB/s wr, 2.00k op/s 2024-02-18T11:42:35.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:42:34 smithi188 ceph-mon[36925]: pgmap v400: 97 pgs: 97 active+clean; 10 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 909 MiB/s wr, 2.00k op/s 2024-02-18T11:42:37.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:42:37 smithi012 ceph-mon[32460]: pgmap v401: 97 pgs: 97 active+clean; 10 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 803 MiB/s wr, 1.81k op/s 2024-02-18T11:42:37.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:42:37 smithi188 ceph-mon[36925]: pgmap v401: 97 pgs: 97 active+clean; 10 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 803 MiB/s wr, 1.81k op/s 2024-02-18T11:42:37.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:42:37 smithi158 ceph-mon[36626]: pgmap v401: 97 pgs: 97 active+clean; 10 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 803 MiB/s wr, 1.81k op/s 2024-02-18T11:42:38.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:42:38 smithi188 ceph-mon[36925]: pgmap v402: 97 pgs: 97 active+clean; 10 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 965 MiB/s wr, 2.38k op/s 2024-02-18T11:42:38.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:42:38 smithi012 ceph-mon[32460]: pgmap v402: 97 pgs: 97 active+clean; 10 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 965 MiB/s wr, 2.38k op/s 2024-02-18T11:42:38.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:42:38 smithi158 ceph-mon[36626]: pgmap v402: 97 pgs: 97 active+clean; 10 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 965 MiB/s wr, 2.38k op/s 2024-02-18T11:42:40.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:42:40 smithi158 ceph-mon[36626]: pgmap v403: 97 pgs: 97 active+clean; 10 GiB data, 58 GiB used, 1014 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 825 MiB/s wr, 2.22k op/s 2024-02-18T11:42:40.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:42:40 smithi188 ceph-mon[36925]: pgmap v403: 97 pgs: 97 active+clean; 10 GiB data, 58 GiB used, 1014 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 825 MiB/s wr, 2.22k op/s 2024-02-18T11:42:40.925 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:42:40 smithi012 ceph-mon[32460]: pgmap v403: 97 pgs: 97 active+clean; 10 GiB data, 58 GiB used, 1014 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 825 MiB/s wr, 2.22k op/s 2024-02-18T11:42:42.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:42:42 smithi188 ceph-mon[36925]: pgmap v404: 97 pgs: 97 active+clean; 10 GiB data, 60 GiB used, 1013 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 884 MiB/s wr, 2.09k op/s 2024-02-18T11:42:42.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:42:42 smithi012 ceph-mon[32460]: pgmap v404: 97 pgs: 97 active+clean; 10 GiB data, 60 GiB used, 1013 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 884 MiB/s wr, 2.09k op/s 2024-02-18T11:42:42.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:42:42 smithi158 ceph-mon[36626]: pgmap v404: 97 pgs: 97 active+clean; 10 GiB data, 60 GiB used, 1013 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 884 MiB/s wr, 2.09k op/s 2024-02-18T11:42:44.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:42:44 smithi158 ceph-mon[36626]: pgmap v405: 97 pgs: 97 active+clean; 10 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 888 MiB/s wr, 1.82k op/s 2024-02-18T11:42:45.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:42:44 smithi188 ceph-mon[36925]: pgmap v405: 97 pgs: 97 active+clean; 10 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 888 MiB/s wr, 1.82k op/s 2024-02-18T11:42:45.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:42:44 smithi012 ceph-mon[32460]: pgmap v405: 97 pgs: 97 active+clean; 10 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 888 MiB/s wr, 1.82k op/s 2024-02-18T11:42:47.180 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:42:46 smithi012 ceph-mon[32460]: pgmap v406: 97 pgs: 97 active+clean; 10 GiB data, 66 GiB used, 1007 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 789 MiB/s wr, 1.70k op/s 2024-02-18T11:42:47.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:42:46 smithi158 ceph-mon[36626]: pgmap v406: 97 pgs: 97 active+clean; 10 GiB data, 66 GiB used, 1007 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 789 MiB/s wr, 1.70k op/s 2024-02-18T11:42:47.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:42:46 smithi188 ceph-mon[36925]: pgmap v406: 97 pgs: 97 active+clean; 10 GiB data, 66 GiB used, 1007 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 789 MiB/s wr, 1.70k op/s 2024-02-18T11:42:49.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:42:49 smithi012 ceph-mon[32460]: pgmap v407: 97 pgs: 97 active+clean; 10 GiB data, 70 GiB used, 1003 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 983 MiB/s wr, 2.45k op/s 2024-02-18T11:42:49.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:42:49 smithi158 ceph-mon[36626]: pgmap v407: 97 pgs: 97 active+clean; 10 GiB data, 70 GiB used, 1003 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 983 MiB/s wr, 2.45k op/s 2024-02-18T11:42:49.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:42:49 smithi188 ceph-mon[36925]: pgmap v407: 97 pgs: 97 active+clean; 10 GiB data, 70 GiB used, 1003 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 983 MiB/s wr, 2.45k op/s 2024-02-18T11:42:50.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:42:50 smithi158 ceph-mon[36626]: pgmap v408: 97 pgs: 97 active+clean; 10 GiB data, 72 GiB used, 1001 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 874 MiB/s wr, 2.17k op/s 2024-02-18T11:42:50.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:42:50 smithi188 ceph-mon[36925]: pgmap v408: 97 pgs: 97 active+clean; 10 GiB data, 72 GiB used, 1001 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 874 MiB/s wr, 2.17k op/s 2024-02-18T11:42:50.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:42:50 smithi012 ceph-mon[32460]: pgmap v408: 97 pgs: 97 active+clean; 10 GiB data, 72 GiB used, 1001 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 874 MiB/s wr, 2.17k op/s 2024-02-18T11:42:52.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:42:52 smithi158 ceph-mon[36626]: pgmap v409: 97 pgs: 97 active+clean; 10 GiB data, 74 GiB used, 999 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 919 MiB/s wr, 2.25k op/s 2024-02-18T11:42:53.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:42:52 smithi188 ceph-mon[36925]: pgmap v409: 97 pgs: 97 active+clean; 10 GiB data, 74 GiB used, 999 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 919 MiB/s wr, 2.25k op/s 2024-02-18T11:42:53.172 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:42:52 smithi012 ceph-mon[32460]: pgmap v409: 97 pgs: 97 active+clean; 10 GiB data, 74 GiB used, 999 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 919 MiB/s wr, 2.25k op/s 2024-02-18T11:42:55.172 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:42:54 smithi012 ceph-mon[32460]: pgmap v410: 97 pgs: 97 active+clean; 10 GiB data, 78 GiB used, 995 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 903 MiB/s wr, 2.20k op/s 2024-02-18T11:42:55.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:42:54 smithi158 ceph-mon[36626]: pgmap v410: 97 pgs: 97 active+clean; 10 GiB data, 78 GiB used, 995 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 903 MiB/s wr, 2.20k op/s 2024-02-18T11:42:55.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:42:54 smithi188 ceph-mon[36925]: pgmap v410: 97 pgs: 97 active+clean; 10 GiB data, 78 GiB used, 995 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 903 MiB/s wr, 2.20k op/s 2024-02-18T11:42:56.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:42:56 smithi188 ceph-mon[36925]: pgmap v411: 97 pgs: 97 active+clean; 10 GiB data, 80 GiB used, 993 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 794 MiB/s wr, 2.02k op/s 2024-02-18T11:42:56.423 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:42:56 smithi012 ceph-mon[32460]: pgmap v411: 97 pgs: 97 active+clean; 10 GiB data, 80 GiB used, 993 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 794 MiB/s wr, 2.02k op/s 2024-02-18T11:42:56.495 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:42:56 smithi158 ceph-mon[36626]: pgmap v411: 97 pgs: 97 active+clean; 10 GiB data, 80 GiB used, 993 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 794 MiB/s wr, 2.02k op/s 2024-02-18T11:42:58.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:42:58 smithi158 ceph-mon[36626]: pgmap v412: 97 pgs: 97 active+clean; 10 GiB data, 82 GiB used, 990 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 979 MiB/s wr, 2.35k op/s 2024-02-18T11:42:58.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:42:58 smithi188 ceph-mon[36925]: pgmap v412: 97 pgs: 97 active+clean; 10 GiB data, 82 GiB used, 990 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 979 MiB/s wr, 2.35k op/s 2024-02-18T11:42:58.675 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:42:58 smithi012 ceph-mon[32460]: pgmap v412: 97 pgs: 97 active+clean; 10 GiB data, 82 GiB used, 990 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 979 MiB/s wr, 2.35k op/s 2024-02-18T11:43:00.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:43:00 smithi188 ceph-mon[36925]: pgmap v413: 97 pgs: 97 active+clean; 10 GiB data, 86 GiB used, 987 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 844 MiB/s wr, 1.71k op/s 2024-02-18T11:43:00.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:43:00 smithi012 ceph-mon[32460]: pgmap v413: 97 pgs: 97 active+clean; 10 GiB data, 86 GiB used, 987 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 844 MiB/s wr, 1.71k op/s 2024-02-18T11:43:00.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:43:00 smithi158 ceph-mon[36626]: pgmap v413: 97 pgs: 97 active+clean; 10 GiB data, 86 GiB used, 987 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 844 MiB/s wr, 1.71k op/s 2024-02-18T11:43:02.744 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:43:02 smithi158 ceph-mon[36626]: pgmap v414: 97 pgs: 97 active+clean; 10 GiB data, 88 GiB used, 985 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 906 MiB/s wr, 1.52k op/s 2024-02-18T11:43:02.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:43:02 smithi188 ceph-mon[36925]: pgmap v414: 97 pgs: 97 active+clean; 10 GiB data, 88 GiB used, 985 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 906 MiB/s wr, 1.52k op/s 2024-02-18T11:43:02.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:43:02 smithi012 ceph-mon[32460]: pgmap v414: 97 pgs: 97 active+clean; 10 GiB data, 88 GiB used, 985 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 906 MiB/s wr, 1.52k op/s 2024-02-18T11:43:04.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:43:04 smithi188 ceph-mon[36925]: pgmap v415: 97 pgs: 97 active+clean; 10 GiB data, 94 GiB used, 979 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 911 MiB/s wr, 1.36k op/s 2024-02-18T11:43:04.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:43:04 smithi012 ceph-mon[32460]: pgmap v415: 97 pgs: 97 active+clean; 10 GiB data, 94 GiB used, 979 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 911 MiB/s wr, 1.36k op/s 2024-02-18T11:43:04.993 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:43:04 smithi158 ceph-mon[36626]: pgmap v415: 97 pgs: 97 active+clean; 10 GiB data, 94 GiB used, 979 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 911 MiB/s wr, 1.36k op/s 2024-02-18T11:43:07.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:43:06 smithi012 ceph-mon[32460]: pgmap v416: 97 pgs: 97 active+clean; 10 GiB data, 94 GiB used, 979 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 816 MiB/s wr, 1.29k op/s 2024-02-18T11:43:07.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:43:06 smithi158 ceph-mon[36626]: pgmap v416: 97 pgs: 97 active+clean; 10 GiB data, 94 GiB used, 979 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 816 MiB/s wr, 1.29k op/s 2024-02-18T11:43:07.355 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:43:06 smithi188 ceph-mon[36925]: pgmap v416: 97 pgs: 97 active+clean; 10 GiB data, 94 GiB used, 979 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 816 MiB/s wr, 1.29k op/s 2024-02-18T11:43:08.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:43:07 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:43:08.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:43:07 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:43:08.432 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:43:07 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:43:09.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:43:09 smithi158 ceph-mon[36626]: pgmap v417: 97 pgs: 97 active+clean; 10 GiB data, 97 GiB used, 976 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 959 MiB/s wr, 1.51k op/s 2024-02-18T11:43:09.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:43:09 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:43:09.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:43:09 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:43:09.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:43:09 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:43:09.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:43:09 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:43:09.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:43:09 smithi188 ceph-mon[36925]: pgmap v417: 97 pgs: 97 active+clean; 10 GiB data, 97 GiB used, 976 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 959 MiB/s wr, 1.51k op/s 2024-02-18T11:43:09.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:43:09 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:43:09.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:43:09 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:43:09.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:43:09 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:43:09.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:43:09 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:43:09.423 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:43:09 smithi012 ceph-mon[32460]: pgmap v417: 97 pgs: 97 active+clean; 10 GiB data, 97 GiB used, 976 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 959 MiB/s wr, 1.51k op/s 2024-02-18T11:43:09.423 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:43:09 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:43:09.423 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:43:09 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:43:09.423 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:43:09 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:43:09.423 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:43:09 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:43:10.425 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:43:10 smithi012 ceph-mon[32460]: pgmap v418: 97 pgs: 97 active+clean; 10 GiB data, 101 GiB used, 972 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 854 MiB/s wr, 1.22k op/s 2024-02-18T11:43:10.425 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:43:10 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:43:10.426 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:43:10 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:43:10.426 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:43:10 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:43:10.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:43:10 smithi158 ceph-mon[36626]: pgmap v418: 97 pgs: 97 active+clean; 10 GiB data, 101 GiB used, 972 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 854 MiB/s wr, 1.22k op/s 2024-02-18T11:43:10.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:43:10 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:43:10.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:43:10 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:43:10.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:43:10 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:43:10.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:43:10 smithi188 ceph-mon[36925]: pgmap v418: 97 pgs: 97 active+clean; 10 GiB data, 101 GiB used, 972 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 854 MiB/s wr, 1.22k op/s 2024-02-18T11:43:10.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:43:10 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:43:10.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:43:10 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:43:10.604 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:43:10 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:43:12.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:43:12 smithi188 ceph-mon[36925]: pgmap v419: 97 pgs: 97 active+clean; 10 GiB data, 103 GiB used, 970 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 907 MiB/s wr, 1.24k op/s 2024-02-18T11:43:12.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:43:12 smithi012 ceph-mon[32460]: pgmap v419: 97 pgs: 97 active+clean; 10 GiB data, 103 GiB used, 970 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 907 MiB/s wr, 1.24k op/s 2024-02-18T11:43:12.744 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:43:12 smithi158 ceph-mon[36626]: pgmap v419: 97 pgs: 97 active+clean; 10 GiB data, 103 GiB used, 970 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 907 MiB/s wr, 1.24k op/s 2024-02-18T11:43:14.678 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:43:14 smithi012 ceph-mon[32460]: pgmap v420: 97 pgs: 97 active+clean; 10 GiB data, 106 GiB used, 967 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 885 MiB/s wr, 1.23k op/s 2024-02-18T11:43:14.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:43:14 smithi158 ceph-mon[36626]: pgmap v420: 97 pgs: 97 active+clean; 10 GiB data, 106 GiB used, 967 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 885 MiB/s wr, 1.23k op/s 2024-02-18T11:43:14.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:43:14 smithi188 ceph-mon[36925]: pgmap v420: 97 pgs: 97 active+clean; 10 GiB data, 106 GiB used, 967 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 885 MiB/s wr, 1.23k op/s 2024-02-18T11:43:16.744 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:43:16 smithi158 ceph-mon[36626]: pgmap v421: 97 pgs: 97 active+clean; 10 GiB data, 108 GiB used, 964 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 783 MiB/s wr, 1.08k op/s 2024-02-18T11:43:16.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:43:16 smithi188 ceph-mon[36925]: pgmap v421: 97 pgs: 97 active+clean; 10 GiB data, 108 GiB used, 964 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 783 MiB/s wr, 1.08k op/s 2024-02-18T11:43:16.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:43:16 smithi012 ceph-mon[32460]: pgmap v421: 97 pgs: 97 active+clean; 10 GiB data, 108 GiB used, 964 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 783 MiB/s wr, 1.08k op/s 2024-02-18T11:43:19.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:43:18 smithi188 ceph-mon[36925]: pgmap v422: 97 pgs: 97 active+clean; 10 GiB data, 110 GiB used, 962 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 891 MiB/s wr, 1.23k op/s 2024-02-18T11:43:19.172 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:43:18 smithi012 ceph-mon[32460]: pgmap v422: 97 pgs: 97 active+clean; 10 GiB data, 110 GiB used, 962 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 891 MiB/s wr, 1.23k op/s 2024-02-18T11:43:19.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:43:18 smithi158 ceph-mon[36626]: pgmap v422: 97 pgs: 97 active+clean; 10 GiB data, 110 GiB used, 962 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 891 MiB/s wr, 1.23k op/s 2024-02-18T11:43:21.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:43:20 smithi012 ceph-mon[32460]: pgmap v423: 97 pgs: 97 active+clean; 10 GiB data, 116 GiB used, 956 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 886 MiB/s wr, 1.22k op/s 2024-02-18T11:43:21.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:43:20 smithi158 ceph-mon[36626]: pgmap v423: 97 pgs: 97 active+clean; 10 GiB data, 116 GiB used, 956 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 886 MiB/s wr, 1.22k op/s 2024-02-18T11:43:21.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:43:20 smithi188 ceph-mon[36925]: pgmap v423: 97 pgs: 97 active+clean; 10 GiB data, 116 GiB used, 956 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 886 MiB/s wr, 1.22k op/s 2024-02-18T11:43:23.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:43:23 smithi188 ceph-mon[36925]: pgmap v424: 97 pgs: 97 active+clean; 10 GiB data, 118 GiB used, 954 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 912 MiB/s wr, 1.28k op/s 2024-02-18T11:43:23.427 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:43:23 smithi012 ceph-mon[32460]: pgmap v424: 97 pgs: 97 active+clean; 10 GiB data, 118 GiB used, 954 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 912 MiB/s wr, 1.28k op/s 2024-02-18T11:43:23.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:43:23 smithi158 ceph-mon[36626]: pgmap v424: 97 pgs: 97 active+clean; 10 GiB data, 118 GiB used, 954 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 912 MiB/s wr, 1.28k op/s 2024-02-18T11:43:24.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:43:24 smithi012 ceph-mon[32460]: pgmap v425: 97 pgs: 97 active+clean; 10 GiB data, 124 GiB used, 949 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 905 MiB/s wr, 1.27k op/s 2024-02-18T11:43:24.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:43:24 smithi158 ceph-mon[36626]: pgmap v425: 97 pgs: 97 active+clean; 10 GiB data, 124 GiB used, 949 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 905 MiB/s wr, 1.27k op/s 2024-02-18T11:43:24.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:43:24 smithi188 ceph-mon[36925]: pgmap v425: 97 pgs: 97 active+clean; 10 GiB data, 124 GiB used, 949 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 905 MiB/s wr, 1.27k op/s 2024-02-18T11:43:26.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:43:26 smithi158 ceph-mon[36626]: pgmap v426: 97 pgs: 97 active+clean; 10 GiB data, 126 GiB used, 947 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 812 MiB/s wr, 1.16k op/s 2024-02-18T11:43:26.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:43:26 smithi188 ceph-mon[36925]: pgmap v426: 97 pgs: 97 active+clean; 10 GiB data, 126 GiB used, 947 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 812 MiB/s wr, 1.16k op/s 2024-02-18T11:43:26.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:43:26 smithi012 ceph-mon[32460]: pgmap v426: 97 pgs: 97 active+clean; 10 GiB data, 126 GiB used, 947 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 812 MiB/s wr, 1.16k op/s 2024-02-18T11:43:28.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:43:28 smithi188 ceph-mon[36925]: pgmap v427: 97 pgs: 97 active+clean; 10 GiB data, 128 GiB used, 945 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 908 MiB/s wr, 1.45k op/s 2024-02-18T11:43:28.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:43:28 smithi012 ceph-mon[32460]: pgmap v427: 97 pgs: 97 active+clean; 10 GiB data, 128 GiB used, 945 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 908 MiB/s wr, 1.45k op/s 2024-02-18T11:43:28.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:43:28 smithi158 ceph-mon[36626]: pgmap v427: 97 pgs: 97 active+clean; 10 GiB data, 128 GiB used, 945 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 908 MiB/s wr, 1.45k op/s 2024-02-18T11:43:30.744 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:43:30 smithi158 ceph-mon[36626]: pgmap v428: 97 pgs: 97 active+clean; 10 GiB data, 133 GiB used, 940 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 919 MiB/s wr, 1.70k op/s 2024-02-18T11:43:30.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:43:30 smithi188 ceph-mon[36925]: pgmap v428: 97 pgs: 97 active+clean; 10 GiB data, 133 GiB used, 940 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 919 MiB/s wr, 1.70k op/s 2024-02-18T11:43:30.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:43:30 smithi012 ceph-mon[32460]: pgmap v428: 97 pgs: 97 active+clean; 10 GiB data, 133 GiB used, 940 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 919 MiB/s wr, 1.70k op/s 2024-02-18T11:43:32.993 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:43:32 smithi158 ceph-mon[36626]: pgmap v429: 97 pgs: 97 active+clean; 10 GiB data, 133 GiB used, 940 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 877 MiB/s wr, 1.73k op/s 2024-02-18T11:43:33.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:43:32 smithi188 ceph-mon[36925]: pgmap v429: 97 pgs: 97 active+clean; 10 GiB data, 133 GiB used, 940 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 877 MiB/s wr, 1.73k op/s 2024-02-18T11:43:33.172 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:43:32 smithi012 ceph-mon[32460]: pgmap v429: 97 pgs: 97 active+clean; 10 GiB data, 133 GiB used, 940 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 877 MiB/s wr, 1.73k op/s 2024-02-18T11:43:35.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:43:34 smithi188 ceph-mon[36925]: pgmap v430: 97 pgs: 97 active+clean; 10 GiB data, 133 GiB used, 940 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 875 MiB/s wr, 1.79k op/s 2024-02-18T11:43:35.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:43:34 smithi012 ceph-mon[32460]: pgmap v430: 97 pgs: 97 active+clean; 10 GiB data, 133 GiB used, 940 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 875 MiB/s wr, 1.79k op/s 2024-02-18T11:43:35.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:43:34 smithi158 ceph-mon[36626]: pgmap v430: 97 pgs: 97 active+clean; 10 GiB data, 133 GiB used, 940 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 875 MiB/s wr, 1.79k op/s 2024-02-18T11:43:37.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:43:36 smithi012 ceph-mon[32460]: pgmap v431: 97 pgs: 97 active+clean; 10 GiB data, 132 GiB used, 940 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 774 MiB/s wr, 1.64k op/s 2024-02-18T11:43:37.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:43:36 smithi158 ceph-mon[36626]: pgmap v431: 97 pgs: 97 active+clean; 10 GiB data, 132 GiB used, 940 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 774 MiB/s wr, 1.64k op/s 2024-02-18T11:43:37.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:43:36 smithi188 ceph-mon[36925]: pgmap v431: 97 pgs: 97 active+clean; 10 GiB data, 132 GiB used, 940 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 774 MiB/s wr, 1.64k op/s 2024-02-18T11:43:39.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:43:39 smithi188 ceph-mon[36925]: pgmap v432: 97 pgs: 97 active+clean; 10 GiB data, 134 GiB used, 938 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 886 MiB/s wr, 1.76k op/s 2024-02-18T11:43:39.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:43:39 smithi012 ceph-mon[32460]: pgmap v432: 97 pgs: 97 active+clean; 10 GiB data, 134 GiB used, 938 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 886 MiB/s wr, 1.76k op/s 2024-02-18T11:43:39.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:43:39 smithi158 ceph-mon[36626]: pgmap v432: 97 pgs: 97 active+clean; 10 GiB data, 134 GiB used, 938 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 886 MiB/s wr, 1.76k op/s 2024-02-18T11:43:40.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:43:40 smithi188 ceph-mon[36925]: pgmap v433: 97 pgs: 97 active+clean; 10 GiB data, 137 GiB used, 936 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 910 MiB/s wr, 1.67k op/s 2024-02-18T11:43:40.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:43:40 smithi012 ceph-mon[32460]: pgmap v433: 97 pgs: 97 active+clean; 10 GiB data, 137 GiB used, 936 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 910 MiB/s wr, 1.67k op/s 2024-02-18T11:43:40.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:43:40 smithi158 ceph-mon[36626]: pgmap v433: 97 pgs: 97 active+clean; 10 GiB data, 137 GiB used, 936 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 910 MiB/s wr, 1.67k op/s 2024-02-18T11:43:42.993 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:43:42 smithi158 ceph-mon[36626]: pgmap v434: 97 pgs: 97 active+clean; 10 GiB data, 139 GiB used, 934 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 883 MiB/s wr, 1.42k op/s 2024-02-18T11:43:43.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:43:42 smithi188 ceph-mon[36925]: pgmap v434: 97 pgs: 97 active+clean; 10 GiB data, 139 GiB used, 934 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 883 MiB/s wr, 1.42k op/s 2024-02-18T11:43:43.172 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:43:42 smithi012 ceph-mon[32460]: pgmap v434: 97 pgs: 97 active+clean; 10 GiB data, 139 GiB used, 934 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 883 MiB/s wr, 1.42k op/s 2024-02-18T11:43:45.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:43:44 smithi188 ceph-mon[36925]: pgmap v435: 97 pgs: 97 active+clean; 10 GiB data, 139 GiB used, 933 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 896 MiB/s wr, 1.51k op/s 2024-02-18T11:43:45.172 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:43:44 smithi012 ceph-mon[32460]: pgmap v435: 97 pgs: 97 active+clean; 10 GiB data, 139 GiB used, 933 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 896 MiB/s wr, 1.51k op/s 2024-02-18T11:43:45.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:43:44 smithi158 ceph-mon[36626]: pgmap v435: 97 pgs: 97 active+clean; 10 GiB data, 139 GiB used, 933 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 896 MiB/s wr, 1.51k op/s 2024-02-18T11:43:47.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:43:46 smithi012 ceph-mon[32460]: pgmap v436: 97 pgs: 97 active+clean; 10 GiB data, 141 GiB used, 931 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 790 MiB/s wr, 1.36k op/s 2024-02-18T11:43:47.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:43:46 smithi158 ceph-mon[36626]: pgmap v436: 97 pgs: 97 active+clean; 10 GiB data, 141 GiB used, 931 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 790 MiB/s wr, 1.36k op/s 2024-02-18T11:43:47.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:43:46 smithi188 ceph-mon[36925]: pgmap v436: 97 pgs: 97 active+clean; 10 GiB data, 141 GiB used, 931 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 790 MiB/s wr, 1.36k op/s 2024-02-18T11:43:49.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:43:49 smithi188 ceph-mon[36925]: pgmap v437: 97 pgs: 97 active+clean; 10 GiB data, 139 GiB used, 934 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 888 MiB/s wr, 1.63k op/s 2024-02-18T11:43:49.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:43:49 smithi012 ceph-mon[32460]: pgmap v437: 97 pgs: 97 active+clean; 10 GiB data, 139 GiB used, 934 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 888 MiB/s wr, 1.63k op/s 2024-02-18T11:43:49.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:43:49 smithi158 ceph-mon[36626]: pgmap v437: 97 pgs: 97 active+clean; 10 GiB data, 139 GiB used, 934 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 888 MiB/s wr, 1.63k op/s 2024-02-18T11:43:50.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:43:50 smithi012 ceph-mon[32460]: pgmap v438: 97 pgs: 97 active+clean; 10 GiB data, 133 GiB used, 940 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 895 MiB/s wr, 1.82k op/s 2024-02-18T11:43:50.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:43:50 smithi158 ceph-mon[36626]: pgmap v438: 97 pgs: 97 active+clean; 10 GiB data, 133 GiB used, 940 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 895 MiB/s wr, 1.82k op/s 2024-02-18T11:43:50.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:43:50 smithi188 ceph-mon[36925]: pgmap v438: 97 pgs: 97 active+clean; 10 GiB data, 133 GiB used, 940 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 895 MiB/s wr, 1.82k op/s 2024-02-18T11:43:52.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:43:52 smithi188 ceph-mon[36925]: pgmap v439: 97 pgs: 97 active+clean; 10 GiB data, 135 GiB used, 937 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 888 MiB/s wr, 1.97k op/s 2024-02-18T11:43:52.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:43:52 smithi012 ceph-mon[32460]: pgmap v439: 97 pgs: 97 active+clean; 10 GiB data, 135 GiB used, 937 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 888 MiB/s wr, 1.97k op/s 2024-02-18T11:43:52.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:43:52 smithi158 ceph-mon[36626]: pgmap v439: 97 pgs: 97 active+clean; 10 GiB data, 135 GiB used, 937 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 888 MiB/s wr, 1.97k op/s 2024-02-18T11:43:54.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:43:54 smithi188 ceph-mon[36925]: pgmap v440: 97 pgs: 97 active+clean; 10 GiB data, 140 GiB used, 932 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 913 MiB/s wr, 1.95k op/s 2024-02-18T11:43:54.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:43:54 smithi012 ceph-mon[32460]: pgmap v440: 97 pgs: 97 active+clean; 10 GiB data, 140 GiB used, 932 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 913 MiB/s wr, 1.95k op/s 2024-02-18T11:43:54.993 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:43:54 smithi158 ceph-mon[36626]: pgmap v440: 97 pgs: 97 active+clean; 10 GiB data, 140 GiB used, 932 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 913 MiB/s wr, 1.95k op/s 2024-02-18T11:43:56.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:43:56 smithi188 ceph-mon[36925]: pgmap v441: 97 pgs: 97 active+clean; 10 GiB data, 143 GiB used, 930 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 803 MiB/s wr, 1.60k op/s 2024-02-18T11:43:56.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:43:56 smithi012 ceph-mon[32460]: pgmap v441: 97 pgs: 97 active+clean; 10 GiB data, 143 GiB used, 930 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 803 MiB/s wr, 1.60k op/s 2024-02-18T11:43:56.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:43:56 smithi158 ceph-mon[36626]: pgmap v441: 97 pgs: 97 active+clean; 10 GiB data, 143 GiB used, 930 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 803 MiB/s wr, 1.60k op/s 2024-02-18T11:43:58.993 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:43:58 smithi158 ceph-mon[36626]: pgmap v442: 97 pgs: 97 active+clean; 10 GiB data, 144 GiB used, 928 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 901 MiB/s wr, 1.68k op/s 2024-02-18T11:43:59.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:43:58 smithi188 ceph-mon[36925]: pgmap v442: 97 pgs: 97 active+clean; 10 GiB data, 144 GiB used, 928 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 901 MiB/s wr, 1.68k op/s 2024-02-18T11:43:59.172 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:43:58 smithi012 ceph-mon[32460]: pgmap v442: 97 pgs: 97 active+clean; 10 GiB data, 144 GiB used, 928 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 901 MiB/s wr, 1.68k op/s 2024-02-18T11:44:01.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:44:00 smithi158 ceph-mon[36626]: pgmap v443: 97 pgs: 97 active+clean; 10 GiB data, 150 GiB used, 923 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 926 MiB/s wr, 1.69k op/s 2024-02-18T11:44:01.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:44:01 smithi188 ceph-mon[36925]: pgmap v443: 97 pgs: 97 active+clean; 10 GiB data, 150 GiB used, 923 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 926 MiB/s wr, 1.69k op/s 2024-02-18T11:44:01.426 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:44:00 smithi012 ceph-mon[32460]: pgmap v443: 97 pgs: 97 active+clean; 10 GiB data, 150 GiB used, 923 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 926 MiB/s wr, 1.69k op/s 2024-02-18T11:44:02.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:44:02 smithi158 ceph-mon[36626]: pgmap v444: 97 pgs: 97 active+clean; 10 GiB data, 152 GiB used, 921 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 911 MiB/s wr, 1.73k op/s 2024-02-18T11:44:02.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:44:02 smithi188 ceph-mon[36925]: pgmap v444: 97 pgs: 97 active+clean; 10 GiB data, 152 GiB used, 921 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 911 MiB/s wr, 1.73k op/s 2024-02-18T11:44:02.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:44:02 smithi012 ceph-mon[32460]: pgmap v444: 97 pgs: 97 active+clean; 10 GiB data, 152 GiB used, 921 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 911 MiB/s wr, 1.73k op/s 2024-02-18T11:44:04.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:44:04 smithi188 ceph-mon[36925]: pgmap v445: 97 pgs: 97 active+clean; 10 GiB data, 157 GiB used, 915 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 897 MiB/s wr, 1.68k op/s 2024-02-18T11:44:04.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:44:04 smithi012 ceph-mon[32460]: pgmap v445: 97 pgs: 97 active+clean; 10 GiB data, 157 GiB used, 915 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 897 MiB/s wr, 1.68k op/s 2024-02-18T11:44:04.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:44:04 smithi158 ceph-mon[36626]: pgmap v445: 97 pgs: 97 active+clean; 10 GiB data, 157 GiB used, 915 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 897 MiB/s wr, 1.68k op/s 2024-02-18T11:44:06.993 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:44:06 smithi158 ceph-mon[36626]: pgmap v446: 97 pgs: 97 active+clean; 10 GiB data, 159 GiB used, 913 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 775 MiB/s wr, 1.53k op/s 2024-02-18T11:44:07.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:44:06 smithi188 ceph-mon[36925]: pgmap v446: 97 pgs: 97 active+clean; 10 GiB data, 159 GiB used, 913 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 775 MiB/s wr, 1.53k op/s 2024-02-18T11:44:07.172 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:44:06 smithi012 ceph-mon[32460]: pgmap v446: 97 pgs: 97 active+clean; 10 GiB data, 159 GiB used, 913 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 775 MiB/s wr, 1.53k op/s 2024-02-18T11:44:09.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:44:08 smithi188 ceph-mon[36925]: pgmap v447: 97 pgs: 97 active+clean; 10 GiB data, 161 GiB used, 911 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 877 MiB/s wr, 1.62k op/s 2024-02-18T11:44:09.183 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:44:08 smithi012 ceph-mon[32460]: pgmap v447: 97 pgs: 97 active+clean; 10 GiB data, 161 GiB used, 911 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 877 MiB/s wr, 1.62k op/s 2024-02-18T11:44:09.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:44:08 smithi158 ceph-mon[36626]: pgmap v447: 97 pgs: 97 active+clean; 10 GiB data, 161 GiB used, 911 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 877 MiB/s wr, 1.62k op/s 2024-02-18T11:44:11.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:44:10 smithi012 ceph-mon[32460]: pgmap v448: 97 pgs: 97 active+clean; 10 GiB data, 166 GiB used, 907 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 868 MiB/s wr, 1.61k op/s 2024-02-18T11:44:11.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:44:10 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:44:11.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:44:10 smithi158 ceph-mon[36626]: pgmap v448: 97 pgs: 97 active+clean; 10 GiB data, 166 GiB used, 907 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 868 MiB/s wr, 1.61k op/s 2024-02-18T11:44:11.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:44:10 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:44:11.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:44:10 smithi188 ceph-mon[36925]: pgmap v448: 97 pgs: 97 active+clean; 10 GiB data, 166 GiB used, 907 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 868 MiB/s wr, 1.61k op/s 2024-02-18T11:44:11.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:44:10 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:44:12.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:44:12 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:44:12.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:44:12 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:44:12.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:44:12 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:44:12.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:44:12 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:44:12.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:44:12 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:44:12.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:44:12 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:44:13.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:44:13 smithi188 ceph-mon[36925]: pgmap v449: 97 pgs: 97 active+clean; 10 GiB data, 169 GiB used, 904 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 902 MiB/s wr, 1.60k op/s 2024-02-18T11:44:13.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:44:13 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:44:13.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:44:13 smithi012 ceph-mon[32460]: pgmap v449: 97 pgs: 97 active+clean; 10 GiB data, 169 GiB used, 904 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 902 MiB/s wr, 1.60k op/s 2024-02-18T11:44:13.423 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:44:13 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:44:13.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:44:13 smithi158 ceph-mon[36626]: pgmap v449: 97 pgs: 97 active+clean; 10 GiB data, 169 GiB used, 904 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 902 MiB/s wr, 1.60k op/s 2024-02-18T11:44:13.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:44:13 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:44:14.423 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:44:14 smithi012 ceph-mon[32460]: pgmap v450: 97 pgs: 97 active+clean; 10 GiB data, 174 GiB used, 899 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 900 MiB/s wr, 1.55k op/s 2024-02-18T11:44:14.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:44:14 smithi158 ceph-mon[36626]: pgmap v450: 97 pgs: 97 active+clean; 10 GiB data, 174 GiB used, 899 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 900 MiB/s wr, 1.55k op/s 2024-02-18T11:44:14.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:44:14 smithi188 ceph-mon[36925]: pgmap v450: 97 pgs: 97 active+clean; 10 GiB data, 174 GiB used, 899 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 900 MiB/s wr, 1.55k op/s 2024-02-18T11:44:16.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:44:16 smithi158 ceph-mon[36626]: pgmap v451: 97 pgs: 97 active+clean; 10 GiB data, 176 GiB used, 896 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 797 MiB/s wr, 1.33k op/s 2024-02-18T11:44:16.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:44:16 smithi188 ceph-mon[36925]: pgmap v451: 97 pgs: 97 active+clean; 10 GiB data, 176 GiB used, 896 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 797 MiB/s wr, 1.33k op/s 2024-02-18T11:44:16.675 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:44:16 smithi012 ceph-mon[32460]: pgmap v451: 97 pgs: 97 active+clean; 10 GiB data, 176 GiB used, 896 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 797 MiB/s wr, 1.33k op/s 2024-02-18T11:44:18.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:44:18 smithi012 ceph-mon[32460]: pgmap v452: 97 pgs: 97 active+clean; 10 GiB data, 178 GiB used, 894 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 903 MiB/s wr, 1.68k op/s 2024-02-18T11:44:18.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:44:18 smithi158 ceph-mon[36626]: pgmap v452: 97 pgs: 97 active+clean; 10 GiB data, 178 GiB used, 894 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 903 MiB/s wr, 1.68k op/s 2024-02-18T11:44:18.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:44:18 smithi188 ceph-mon[36925]: pgmap v452: 97 pgs: 97 active+clean; 10 GiB data, 178 GiB used, 894 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 903 MiB/s wr, 1.68k op/s 2024-02-18T11:44:20.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:44:20 smithi188 ceph-mon[36925]: pgmap v453: 97 pgs: 97 active+clean; 10 GiB data, 183 GiB used, 890 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 886 MiB/s wr, 1.69k op/s 2024-02-18T11:44:20.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:44:20 smithi012 ceph-mon[32460]: pgmap v453: 97 pgs: 97 active+clean; 10 GiB data, 183 GiB used, 890 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 886 MiB/s wr, 1.69k op/s 2024-02-18T11:44:20.993 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:44:20 smithi158 ceph-mon[36626]: pgmap v453: 97 pgs: 97 active+clean; 10 GiB data, 183 GiB used, 890 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 886 MiB/s wr, 1.69k op/s 2024-02-18T11:44:22.993 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:44:22 smithi158 ceph-mon[36626]: pgmap v454: 97 pgs: 97 active+clean; 10 GiB data, 186 GiB used, 887 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 935 MiB/s wr, 1.71k op/s 2024-02-18T11:44:23.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:44:22 smithi188 ceph-mon[36925]: pgmap v454: 97 pgs: 97 active+clean; 10 GiB data, 186 GiB used, 887 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 935 MiB/s wr, 1.71k op/s 2024-02-18T11:44:23.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:44:22 smithi012 ceph-mon[32460]: pgmap v454: 97 pgs: 97 active+clean; 10 GiB data, 186 GiB used, 887 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 935 MiB/s wr, 1.71k op/s 2024-02-18T11:44:25.172 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:44:24 smithi012 ceph-mon[32460]: pgmap v455: 97 pgs: 97 active+clean; 10 GiB data, 191 GiB used, 882 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 875 MiB/s wr, 1.61k op/s 2024-02-18T11:44:25.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:44:24 smithi158 ceph-mon[36626]: pgmap v455: 97 pgs: 97 active+clean; 10 GiB data, 191 GiB used, 882 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 875 MiB/s wr, 1.61k op/s 2024-02-18T11:44:25.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:44:24 smithi188 ceph-mon[36925]: pgmap v455: 97 pgs: 97 active+clean; 10 GiB data, 191 GiB used, 882 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 875 MiB/s wr, 1.61k op/s 2024-02-18T11:44:26.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:44:26 smithi012 ceph-mon[32460]: pgmap v456: 97 pgs: 97 active+clean; 10 GiB data, 193 GiB used, 879 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 769 MiB/s wr, 1.28k op/s 2024-02-18T11:44:26.744 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:44:26 smithi158 ceph-mon[36626]: pgmap v456: 97 pgs: 97 active+clean; 10 GiB data, 193 GiB used, 879 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 769 MiB/s wr, 1.28k op/s 2024-02-18T11:44:26.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:44:26 smithi188 ceph-mon[36925]: pgmap v456: 97 pgs: 97 active+clean; 10 GiB data, 193 GiB used, 879 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 769 MiB/s wr, 1.28k op/s 2024-02-18T11:44:28.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:44:28 smithi012 ceph-mon[32460]: pgmap v457: 97 pgs: 97 active+clean; 10 GiB data, 195 GiB used, 878 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 862 MiB/s wr, 1.31k op/s 2024-02-18T11:44:28.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:44:28 smithi158 ceph-mon[36626]: pgmap v457: 97 pgs: 97 active+clean; 10 GiB data, 195 GiB used, 878 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 862 MiB/s wr, 1.31k op/s 2024-02-18T11:44:28.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:44:28 smithi188 ceph-mon[36925]: pgmap v457: 97 pgs: 97 active+clean; 10 GiB data, 195 GiB used, 878 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 862 MiB/s wr, 1.31k op/s 2024-02-18T11:44:30.744 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:44:30 smithi158 ceph-mon[36626]: pgmap v458: 97 pgs: 97 active+clean; 10 GiB data, 199 GiB used, 874 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 837 MiB/s wr, 1.06k op/s 2024-02-18T11:44:30.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:44:30 smithi188 ceph-mon[36925]: pgmap v458: 97 pgs: 97 active+clean; 10 GiB data, 199 GiB used, 874 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 837 MiB/s wr, 1.06k op/s 2024-02-18T11:44:30.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:44:30 smithi012 ceph-mon[32460]: pgmap v458: 97 pgs: 97 active+clean; 10 GiB data, 199 GiB used, 874 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 837 MiB/s wr, 1.06k op/s 2024-02-18T11:44:32.930 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:44:32 smithi012 ceph-mon[32460]: pgmap v459: 97 pgs: 97 active+clean; 10 GiB data, 202 GiB used, 871 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 883 MiB/s wr, 1.09k op/s 2024-02-18T11:44:32.993 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:44:32 smithi158 ceph-mon[36626]: pgmap v459: 97 pgs: 97 active+clean; 10 GiB data, 202 GiB used, 871 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 883 MiB/s wr, 1.09k op/s 2024-02-18T11:44:33.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:44:32 smithi188 ceph-mon[36925]: pgmap v459: 97 pgs: 97 active+clean; 10 GiB data, 202 GiB used, 871 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 883 MiB/s wr, 1.09k op/s 2024-02-18T11:44:35.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:44:35 smithi188 ceph-mon[36925]: pgmap v460: 97 pgs: 97 active+clean; 10 GiB data, 199 GiB used, 874 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 852 MiB/s wr, 1.06k op/s 2024-02-18T11:44:35.425 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:44:35 smithi012 ceph-mon[32460]: pgmap v460: 97 pgs: 97 active+clean; 10 GiB data, 199 GiB used, 874 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 852 MiB/s wr, 1.06k op/s 2024-02-18T11:44:35.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:44:35 smithi158 ceph-mon[36626]: pgmap v460: 97 pgs: 97 active+clean; 10 GiB data, 199 GiB used, 874 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 852 MiB/s wr, 1.06k op/s 2024-02-18T11:44:36.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:44:36 smithi188 ceph-mon[36925]: pgmap v461: 97 pgs: 97 active+clean; 10 GiB data, 201 GiB used, 872 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 758 MiB/s wr, 907 op/s 2024-02-18T11:44:36.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:44:36 smithi012 ceph-mon[32460]: pgmap v461: 97 pgs: 97 active+clean; 10 GiB data, 201 GiB used, 872 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 758 MiB/s wr, 907 op/s 2024-02-18T11:44:36.744 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:44:36 smithi158 ceph-mon[36626]: pgmap v461: 97 pgs: 97 active+clean; 10 GiB data, 201 GiB used, 872 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 758 MiB/s wr, 907 op/s 2024-02-18T11:44:38.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:44:38 smithi158 ceph-mon[36626]: pgmap v462: 97 pgs: 97 active+clean; 10 GiB data, 197 GiB used, 876 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 848 MiB/s wr, 1.01k op/s 2024-02-18T11:44:38.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:44:38 smithi188 ceph-mon[36925]: pgmap v462: 97 pgs: 97 active+clean; 10 GiB data, 197 GiB used, 876 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 848 MiB/s wr, 1.01k op/s 2024-02-18T11:44:38.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:44:38 smithi012 ceph-mon[32460]: pgmap v462: 97 pgs: 97 active+clean; 10 GiB data, 197 GiB used, 876 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 848 MiB/s wr, 1.01k op/s 2024-02-18T11:44:40.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:44:40 smithi188 ceph-mon[36925]: pgmap v463: 97 pgs: 97 active+clean; 10 GiB data, 201 GiB used, 872 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 832 MiB/s wr, 970 op/s 2024-02-18T11:44:40.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:44:40 smithi012 ceph-mon[32460]: pgmap v463: 97 pgs: 97 active+clean; 10 GiB data, 201 GiB used, 872 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 832 MiB/s wr, 970 op/s 2024-02-18T11:44:40.995 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:44:40 smithi158 ceph-mon[36626]: pgmap v463: 97 pgs: 97 active+clean; 10 GiB data, 201 GiB used, 872 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 832 MiB/s wr, 970 op/s 2024-02-18T11:44:43.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:44:42 smithi188 ceph-mon[36925]: pgmap v464: 97 pgs: 97 active+clean; 10 GiB data, 204 GiB used, 869 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 873 MiB/s wr, 1.02k op/s 2024-02-18T11:44:43.172 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:44:42 smithi012 ceph-mon[32460]: pgmap v464: 97 pgs: 97 active+clean; 10 GiB data, 204 GiB used, 869 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 873 MiB/s wr, 1.02k op/s 2024-02-18T11:44:43.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:44:42 smithi158 ceph-mon[36626]: pgmap v464: 97 pgs: 97 active+clean; 10 GiB data, 204 GiB used, 869 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 873 MiB/s wr, 1.02k op/s 2024-02-18T11:44:45.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:44:44 smithi012 ceph-mon[32460]: pgmap v465: 97 pgs: 97 active+clean; 10 GiB data, 209 GiB used, 864 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 844 MiB/s wr, 1.01k op/s 2024-02-18T11:44:45.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:44:44 smithi158 ceph-mon[36626]: pgmap v465: 97 pgs: 97 active+clean; 10 GiB data, 209 GiB used, 864 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 844 MiB/s wr, 1.01k op/s 2024-02-18T11:44:45.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:44:44 smithi188 ceph-mon[36925]: pgmap v465: 97 pgs: 97 active+clean; 10 GiB data, 209 GiB used, 864 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 844 MiB/s wr, 1.01k op/s 2024-02-18T11:44:47.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:44:47 smithi158 ceph-mon[36626]: pgmap v466: 97 pgs: 97 active+clean; 10 GiB data, 211 GiB used, 862 GiB / 1.0 TiB avail; 1.2 MiB/s rd, 741 MiB/s wr, 883 op/s 2024-02-18T11:44:47.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:44:47 smithi188 ceph-mon[36925]: pgmap v466: 97 pgs: 97 active+clean; 10 GiB data, 211 GiB used, 862 GiB / 1.0 TiB avail; 1.2 MiB/s rd, 741 MiB/s wr, 883 op/s 2024-02-18T11:44:47.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:44:47 smithi012 ceph-mon[32460]: pgmap v466: 97 pgs: 97 active+clean; 10 GiB data, 211 GiB used, 862 GiB / 1.0 TiB avail; 1.2 MiB/s rd, 741 MiB/s wr, 883 op/s 2024-02-18T11:44:48.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:44:48 smithi188 ceph-mon[36925]: pgmap v467: 97 pgs: 97 active+clean; 10 GiB data, 213 GiB used, 860 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 843 MiB/s wr, 1.01k op/s 2024-02-18T11:44:48.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:44:48 smithi012 ceph-mon[32460]: pgmap v467: 97 pgs: 97 active+clean; 10 GiB data, 213 GiB used, 860 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 843 MiB/s wr, 1.01k op/s 2024-02-18T11:44:48.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:44:48 smithi158 ceph-mon[36626]: pgmap v467: 97 pgs: 97 active+clean; 10 GiB data, 213 GiB used, 860 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 843 MiB/s wr, 1.01k op/s 2024-02-18T11:44:50.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:44:50 smithi188 ceph-mon[36925]: pgmap v468: 97 pgs: 97 active+clean; 10 GiB data, 217 GiB used, 856 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 831 MiB/s wr, 1.00k op/s 2024-02-18T11:44:50.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:44:50 smithi012 ceph-mon[32460]: pgmap v468: 97 pgs: 97 active+clean; 10 GiB data, 217 GiB used, 856 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 831 MiB/s wr, 1.00k op/s 2024-02-18T11:44:50.744 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:44:50 smithi158 ceph-mon[36626]: pgmap v468: 97 pgs: 97 active+clean; 10 GiB data, 217 GiB used, 856 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 831 MiB/s wr, 1.00k op/s 2024-02-18T11:44:52.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:44:52 smithi012 ceph-mon[32460]: pgmap v469: 97 pgs: 97 active+clean; 10 GiB data, 220 GiB used, 853 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 893 MiB/s wr, 1.15k op/s 2024-02-18T11:44:52.993 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:44:52 smithi158 ceph-mon[36626]: pgmap v469: 97 pgs: 97 active+clean; 10 GiB data, 220 GiB used, 853 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 893 MiB/s wr, 1.15k op/s 2024-02-18T11:44:53.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:44:52 smithi188 ceph-mon[36925]: pgmap v469: 97 pgs: 97 active+clean; 10 GiB data, 220 GiB used, 853 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 893 MiB/s wr, 1.15k op/s 2024-02-18T11:44:55.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:44:54 smithi012 ceph-mon[32460]: pgmap v470: 97 pgs: 97 active+clean; 10 GiB data, 225 GiB used, 847 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 876 MiB/s wr, 1.20k op/s 2024-02-18T11:44:55.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:44:54 smithi158 ceph-mon[36626]: pgmap v470: 97 pgs: 97 active+clean; 10 GiB data, 225 GiB used, 847 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 876 MiB/s wr, 1.20k op/s 2024-02-18T11:44:55.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:44:54 smithi188 ceph-mon[36925]: pgmap v470: 97 pgs: 97 active+clean; 10 GiB data, 225 GiB used, 847 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 876 MiB/s wr, 1.20k op/s 2024-02-18T11:44:57.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:44:57 smithi188 ceph-mon[36925]: pgmap v471: 97 pgs: 97 active+clean; 10 GiB data, 227 GiB used, 845 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 773 MiB/s wr, 1.10k op/s 2024-02-18T11:44:57.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:44:57 smithi012 ceph-mon[32460]: pgmap v471: 97 pgs: 97 active+clean; 10 GiB data, 227 GiB used, 845 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 773 MiB/s wr, 1.10k op/s 2024-02-18T11:44:57.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:44:57 smithi158 ceph-mon[36626]: pgmap v471: 97 pgs: 97 active+clean; 10 GiB data, 227 GiB used, 845 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 773 MiB/s wr, 1.10k op/s 2024-02-18T11:44:58.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:44:58 smithi158 ceph-mon[36626]: pgmap v472: 97 pgs: 97 active+clean; 10 GiB data, 218 GiB used, 855 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 868 MiB/s wr, 1.24k op/s 2024-02-18T11:44:58.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:44:58 smithi188 ceph-mon[36925]: pgmap v472: 97 pgs: 97 active+clean; 10 GiB data, 218 GiB used, 855 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 868 MiB/s wr, 1.24k op/s 2024-02-18T11:44:58.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:44:58 smithi012 ceph-mon[32460]: pgmap v472: 97 pgs: 97 active+clean; 10 GiB data, 218 GiB used, 855 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 868 MiB/s wr, 1.24k op/s 2024-02-18T11:45:00.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:45:00 smithi188 ceph-mon[36925]: pgmap v473: 97 pgs: 97 active+clean; 10 GiB data, 210 GiB used, 863 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 848 MiB/s wr, 1.24k op/s 2024-02-18T11:45:00.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:45:00 smithi012 ceph-mon[32460]: pgmap v473: 97 pgs: 97 active+clean; 10 GiB data, 210 GiB used, 863 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 848 MiB/s wr, 1.24k op/s 2024-02-18T11:45:00.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:45:00 smithi158 ceph-mon[36626]: pgmap v473: 97 pgs: 97 active+clean; 10 GiB data, 210 GiB used, 863 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 848 MiB/s wr, 1.24k op/s 2024-02-18T11:45:02.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:45:02 smithi012 ceph-mon[32460]: pgmap v474: 97 pgs: 97 active+clean; 10 GiB data, 190 GiB used, 883 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 911 MiB/s wr, 1.35k op/s 2024-02-18T11:45:02.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:45:02 smithi158 ceph-mon[36626]: pgmap v474: 97 pgs: 97 active+clean; 10 GiB data, 190 GiB used, 883 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 911 MiB/s wr, 1.35k op/s 2024-02-18T11:45:02.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:45:02 smithi188 ceph-mon[36925]: pgmap v474: 97 pgs: 97 active+clean; 10 GiB data, 190 GiB used, 883 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 911 MiB/s wr, 1.35k op/s 2024-02-18T11:45:04.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:45:04 smithi188 ceph-mon[36925]: pgmap v475: 97 pgs: 97 active+clean; 10 GiB data, 183 GiB used, 890 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 891 MiB/s wr, 1.33k op/s 2024-02-18T11:45:04.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:45:04 smithi012 ceph-mon[32460]: pgmap v475: 97 pgs: 97 active+clean; 10 GiB data, 183 GiB used, 890 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 891 MiB/s wr, 1.33k op/s 2024-02-18T11:45:04.993 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:45:04 smithi158 ceph-mon[36626]: pgmap v475: 97 pgs: 97 active+clean; 10 GiB data, 183 GiB used, 890 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 891 MiB/s wr, 1.33k op/s 2024-02-18T11:45:07.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:45:06 smithi188 ceph-mon[36925]: pgmap v476: 97 pgs: 97 active+clean; 10 GiB data, 185 GiB used, 888 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 792 MiB/s wr, 1.19k op/s 2024-02-18T11:45:07.172 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:45:06 smithi012 ceph-mon[32460]: pgmap v476: 97 pgs: 97 active+clean; 10 GiB data, 185 GiB used, 888 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 792 MiB/s wr, 1.19k op/s 2024-02-18T11:45:07.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:45:06 smithi158 ceph-mon[36626]: pgmap v476: 97 pgs: 97 active+clean; 10 GiB data, 185 GiB used, 888 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 792 MiB/s wr, 1.19k op/s 2024-02-18T11:45:09.176 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:45:08 smithi012 ceph-mon[32460]: pgmap v477: 97 pgs: 97 active+clean; 10 GiB data, 187 GiB used, 886 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 903 MiB/s wr, 1.54k op/s 2024-02-18T11:45:09.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:45:08 smithi158 ceph-mon[36626]: pgmap v477: 97 pgs: 97 active+clean; 10 GiB data, 187 GiB used, 886 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 903 MiB/s wr, 1.54k op/s 2024-02-18T11:45:09.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:45:08 smithi188 ceph-mon[36925]: pgmap v477: 97 pgs: 97 active+clean; 10 GiB data, 187 GiB used, 886 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 903 MiB/s wr, 1.54k op/s 2024-02-18T11:45:11.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:45:11 smithi188 ceph-mon[36925]: pgmap v478: 97 pgs: 97 active+clean; 10 GiB data, 191 GiB used, 882 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 887 MiB/s wr, 1.70k op/s 2024-02-18T11:45:11.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:45:11 smithi012 ceph-mon[32460]: pgmap v478: 97 pgs: 97 active+clean; 10 GiB data, 191 GiB used, 882 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 887 MiB/s wr, 1.70k op/s 2024-02-18T11:45:11.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:45:11 smithi158 ceph-mon[36626]: pgmap v478: 97 pgs: 97 active+clean; 10 GiB data, 191 GiB used, 882 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 887 MiB/s wr, 1.70k op/s 2024-02-18T11:45:12.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:45:12 smithi012 ceph-mon[32460]: pgmap v479: 97 pgs: 97 active+clean; 10 GiB data, 194 GiB used, 879 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 932 MiB/s wr, 1.78k op/s 2024-02-18T11:45:12.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:45:12 smithi158 ceph-mon[36626]: pgmap v479: 97 pgs: 97 active+clean; 10 GiB data, 194 GiB used, 879 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 932 MiB/s wr, 1.78k op/s 2024-02-18T11:45:12.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:45:12 smithi188 ceph-mon[36925]: pgmap v479: 97 pgs: 97 active+clean; 10 GiB data, 194 GiB used, 879 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 932 MiB/s wr, 1.78k op/s 2024-02-18T11:45:13.423 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:45:13 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:45:13.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:45:13 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:45:13.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:45:13 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:45:14.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:45:14 smithi188 ceph-mon[36925]: pgmap v480: 97 pgs: 97 active+clean; 10 GiB data, 200 GiB used, 873 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 897 MiB/s wr, 1.70k op/s 2024-02-18T11:45:14.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:45:14 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:45:14.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:45:14 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:45:14.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:45:14 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:45:14.677 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:45:14 smithi012 ceph-mon[32460]: pgmap v480: 97 pgs: 97 active+clean; 10 GiB data, 200 GiB used, 873 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 897 MiB/s wr, 1.70k op/s 2024-02-18T11:45:14.677 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:45:14 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:45:14.677 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:45:14 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:45:14.677 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:45:14 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:45:14.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:45:14 smithi158 ceph-mon[36626]: pgmap v480: 97 pgs: 97 active+clean; 10 GiB data, 200 GiB used, 873 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 897 MiB/s wr, 1.70k op/s 2024-02-18T11:45:14.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:45:14 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:45:14.744 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:45:14 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:45:14.744 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:45:14 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:45:16.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:45:16 smithi188 ceph-mon[36925]: pgmap v481: 97 pgs: 97 active+clean; 10 GiB data, 202 GiB used, 871 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 779 MiB/s wr, 1.52k op/s 2024-02-18T11:45:16.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:45:16 smithi012 ceph-mon[32460]: pgmap v481: 97 pgs: 97 active+clean; 10 GiB data, 202 GiB used, 871 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 779 MiB/s wr, 1.52k op/s 2024-02-18T11:45:16.993 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:45:16 smithi158 ceph-mon[36626]: pgmap v481: 97 pgs: 97 active+clean; 10 GiB data, 202 GiB used, 871 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 779 MiB/s wr, 1.52k op/s 2024-02-18T11:45:18.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:45:18 smithi012 ceph-mon[32460]: pgmap v482: 97 pgs: 97 active+clean; 10 GiB data, 204 GiB used, 869 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 881 MiB/s wr, 1.74k op/s 2024-02-18T11:45:18.993 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:45:18 smithi158 ceph-mon[36626]: pgmap v482: 97 pgs: 97 active+clean; 10 GiB data, 204 GiB used, 869 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 881 MiB/s wr, 1.74k op/s 2024-02-18T11:45:19.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:45:18 smithi188 ceph-mon[36925]: pgmap v482: 97 pgs: 97 active+clean; 10 GiB data, 204 GiB used, 869 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 881 MiB/s wr, 1.74k op/s 2024-02-18T11:45:21.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:45:20 smithi188 ceph-mon[36925]: pgmap v483: 97 pgs: 97 active+clean; 10 GiB data, 208 GiB used, 864 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 863 MiB/s wr, 1.68k op/s 2024-02-18T11:45:21.174 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:45:20 smithi012 ceph-mon[32460]: pgmap v483: 97 pgs: 97 active+clean; 10 GiB data, 208 GiB used, 864 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 863 MiB/s wr, 1.68k op/s 2024-02-18T11:45:21.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:45:20 smithi158 ceph-mon[36626]: pgmap v483: 97 pgs: 97 active+clean; 10 GiB data, 208 GiB used, 864 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 863 MiB/s wr, 1.68k op/s 2024-02-18T11:45:23.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:45:23 smithi188 ceph-mon[36925]: pgmap v484: 97 pgs: 97 active+clean; 10 GiB data, 212 GiB used, 861 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 939 MiB/s wr, 1.81k op/s 2024-02-18T11:45:23.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:45:23 smithi012 ceph-mon[32460]: pgmap v484: 97 pgs: 97 active+clean; 10 GiB data, 212 GiB used, 861 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 939 MiB/s wr, 1.81k op/s 2024-02-18T11:45:23.496 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:45:23 smithi158 ceph-mon[36626]: pgmap v484: 97 pgs: 97 active+clean; 10 GiB data, 212 GiB used, 861 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 939 MiB/s wr, 1.81k op/s 2024-02-18T11:45:24.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:45:24 smithi188 ceph-mon[36925]: pgmap v485: 97 pgs: 97 active+clean; 10 GiB data, 203 GiB used, 870 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 918 MiB/s wr, 1.72k op/s 2024-02-18T11:45:24.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:45:24 smithi012 ceph-mon[32460]: pgmap v485: 97 pgs: 97 active+clean; 10 GiB data, 203 GiB used, 870 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 918 MiB/s wr, 1.72k op/s 2024-02-18T11:45:24.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:45:24 smithi158 ceph-mon[36626]: pgmap v485: 97 pgs: 97 active+clean; 10 GiB data, 203 GiB used, 870 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 918 MiB/s wr, 1.72k op/s 2024-02-18T11:45:26.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:45:26 smithi012 ceph-mon[32460]: pgmap v486: 97 pgs: 97 active+clean; 10 GiB data, 205 GiB used, 868 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 811 MiB/s wr, 1.63k op/s 2024-02-18T11:45:26.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:45:26 smithi158 ceph-mon[36626]: pgmap v486: 97 pgs: 97 active+clean; 10 GiB data, 205 GiB used, 868 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 811 MiB/s wr, 1.63k op/s 2024-02-18T11:45:26.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:45:26 smithi188 ceph-mon[36925]: pgmap v486: 97 pgs: 97 active+clean; 10 GiB data, 205 GiB used, 868 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 811 MiB/s wr, 1.63k op/s 2024-02-18T11:45:28.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:45:28 smithi012 ceph-mon[32460]: pgmap v487: 97 pgs: 97 active+clean; 10 GiB data, 199 GiB used, 873 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 907 MiB/s wr, 1.85k op/s 2024-02-18T11:45:28.993 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:45:28 smithi158 ceph-mon[36626]: pgmap v487: 97 pgs: 97 active+clean; 10 GiB data, 199 GiB used, 873 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 907 MiB/s wr, 1.85k op/s 2024-02-18T11:45:29.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:45:28 smithi188 ceph-mon[36925]: pgmap v487: 97 pgs: 97 active+clean; 10 GiB data, 199 GiB used, 873 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 907 MiB/s wr, 1.85k op/s 2024-02-18T11:45:31.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:45:30 smithi188 ceph-mon[36925]: pgmap v488: 97 pgs: 97 active+clean; 10 GiB data, 204 GiB used, 869 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 884 MiB/s wr, 1.83k op/s 2024-02-18T11:45:31.172 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:45:30 smithi012 ceph-mon[32460]: pgmap v488: 97 pgs: 97 active+clean; 10 GiB data, 204 GiB used, 869 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 884 MiB/s wr, 1.83k op/s 2024-02-18T11:45:31.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:45:30 smithi158 ceph-mon[36626]: pgmap v488: 97 pgs: 97 active+clean; 10 GiB data, 204 GiB used, 869 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 884 MiB/s wr, 1.83k op/s 2024-02-18T11:45:33.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:45:32 smithi012 ceph-mon[32460]: pgmap v489: 97 pgs: 97 active+clean; 10 GiB data, 207 GiB used, 866 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 946 MiB/s wr, 1.76k op/s 2024-02-18T11:45:33.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:45:32 smithi158 ceph-mon[36626]: pgmap v489: 97 pgs: 97 active+clean; 10 GiB data, 207 GiB used, 866 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 946 MiB/s wr, 1.76k op/s 2024-02-18T11:45:33.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:45:32 smithi188 ceph-mon[36925]: pgmap v489: 97 pgs: 97 active+clean; 10 GiB data, 207 GiB used, 866 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 946 MiB/s wr, 1.76k op/s 2024-02-18T11:45:34.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:45:34 smithi158 ceph-mon[36626]: pgmap v490: 97 pgs: 97 active+clean; 10 GiB data, 211 GiB used, 861 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 863 MiB/s wr, 1.60k op/s 2024-02-18T11:45:34.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:45:34 smithi188 ceph-mon[36925]: pgmap v490: 97 pgs: 97 active+clean; 10 GiB data, 211 GiB used, 861 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 863 MiB/s wr, 1.60k op/s 2024-02-18T11:45:34.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:45:34 smithi012 ceph-mon[32460]: pgmap v490: 97 pgs: 97 active+clean; 10 GiB data, 211 GiB used, 861 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 863 MiB/s wr, 1.60k op/s 2024-02-18T11:45:36.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:45:36 smithi012 ceph-mon[32460]: pgmap v491: 97 pgs: 97 active+clean; 10 GiB data, 215 GiB used, 858 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 786 MiB/s wr, 1.64k op/s 2024-02-18T11:45:36.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:45:36 smithi158 ceph-mon[36626]: pgmap v491: 97 pgs: 97 active+clean; 10 GiB data, 215 GiB used, 858 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 786 MiB/s wr, 1.64k op/s 2024-02-18T11:45:36.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:45:36 smithi188 ceph-mon[36925]: pgmap v491: 97 pgs: 97 active+clean; 10 GiB data, 215 GiB used, 858 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 786 MiB/s wr, 1.64k op/s 2024-02-18T11:45:38.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:45:38 smithi012 ceph-mon[32460]: pgmap v492: 97 pgs: 97 active+clean; 10 GiB data, 217 GiB used, 856 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 895 MiB/s wr, 1.90k op/s 2024-02-18T11:45:38.993 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:45:38 smithi158 ceph-mon[36626]: pgmap v492: 97 pgs: 97 active+clean; 10 GiB data, 217 GiB used, 856 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 895 MiB/s wr, 1.90k op/s 2024-02-18T11:45:39.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:45:38 smithi188 ceph-mon[36925]: pgmap v492: 97 pgs: 97 active+clean; 10 GiB data, 217 GiB used, 856 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 895 MiB/s wr, 1.90k op/s 2024-02-18T11:45:41.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:45:40 smithi158 ceph-mon[36626]: pgmap v493: 97 pgs: 97 active+clean; 10 GiB data, 221 GiB used, 852 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 887 MiB/s wr, 1.77k op/s 2024-02-18T11:45:41.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:45:40 smithi188 ceph-mon[36925]: pgmap v493: 97 pgs: 97 active+clean; 10 GiB data, 221 GiB used, 852 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 887 MiB/s wr, 1.77k op/s 2024-02-18T11:45:41.423 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:45:40 smithi012 ceph-mon[32460]: pgmap v493: 97 pgs: 97 active+clean; 10 GiB data, 221 GiB used, 852 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 887 MiB/s wr, 1.77k op/s 2024-02-18T11:45:43.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:45:42 smithi158 ceph-mon[36626]: pgmap v494: 97 pgs: 97 active+clean; 10 GiB data, 224 GiB used, 848 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 941 MiB/s wr, 1.85k op/s 2024-02-18T11:45:43.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:45:42 smithi188 ceph-mon[36925]: pgmap v494: 97 pgs: 97 active+clean; 10 GiB data, 224 GiB used, 848 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 941 MiB/s wr, 1.85k op/s 2024-02-18T11:45:43.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:45:42 smithi012 ceph-mon[32460]: pgmap v494: 97 pgs: 97 active+clean; 10 GiB data, 224 GiB used, 848 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 941 MiB/s wr, 1.85k op/s 2024-02-18T11:45:45.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:45:45 smithi188 ceph-mon[36925]: pgmap v495: 97 pgs: 97 active+clean; 10 GiB data, 228 GiB used, 844 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 849 MiB/s wr, 1.79k op/s 2024-02-18T11:45:45.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:45:45 smithi012 ceph-mon[32460]: pgmap v495: 97 pgs: 97 active+clean; 10 GiB data, 228 GiB used, 844 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 849 MiB/s wr, 1.79k op/s 2024-02-18T11:45:45.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:45:45 smithi158 ceph-mon[36626]: pgmap v495: 97 pgs: 97 active+clean; 10 GiB data, 228 GiB used, 844 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 849 MiB/s wr, 1.79k op/s 2024-02-18T11:45:46.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:45:46 smithi012 ceph-mon[32460]: pgmap v496: 97 pgs: 97 active+clean; 10 GiB data, 220 GiB used, 853 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 808 MiB/s wr, 1.70k op/s 2024-02-18T11:45:46.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:45:46 smithi158 ceph-mon[36626]: pgmap v496: 97 pgs: 97 active+clean; 10 GiB data, 220 GiB used, 853 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 808 MiB/s wr, 1.70k op/s 2024-02-18T11:45:46.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:45:46 smithi188 ceph-mon[36925]: pgmap v496: 97 pgs: 97 active+clean; 10 GiB data, 220 GiB used, 853 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 808 MiB/s wr, 1.70k op/s 2024-02-18T11:45:48.676 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:45:48 smithi012 ceph-mon[32460]: pgmap v497: 97 pgs: 97 active+clean; 10 GiB data, 222 GiB used, 851 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 883 MiB/s wr, 1.88k op/s 2024-02-18T11:45:48.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:45:48 smithi158 ceph-mon[36626]: pgmap v497: 97 pgs: 97 active+clean; 10 GiB data, 222 GiB used, 851 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 883 MiB/s wr, 1.88k op/s 2024-02-18T11:45:48.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:45:48 smithi188 ceph-mon[36925]: pgmap v497: 97 pgs: 97 active+clean; 10 GiB data, 222 GiB used, 851 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 883 MiB/s wr, 1.88k op/s 2024-02-18T11:45:50.993 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:45:50 smithi158 ceph-mon[36626]: pgmap v498: 97 pgs: 97 active+clean; 10 GiB data, 226 GiB used, 846 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 871 MiB/s wr, 1.81k op/s 2024-02-18T11:45:51.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:45:50 smithi188 ceph-mon[36925]: pgmap v498: 97 pgs: 97 active+clean; 10 GiB data, 226 GiB used, 846 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 871 MiB/s wr, 1.81k op/s 2024-02-18T11:45:51.172 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:45:50 smithi012 ceph-mon[32460]: pgmap v498: 97 pgs: 97 active+clean; 10 GiB data, 226 GiB used, 846 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 871 MiB/s wr, 1.81k op/s 2024-02-18T11:45:53.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:45:52 smithi158 ceph-mon[36626]: pgmap v499: 97 pgs: 97 active+clean; 10 GiB data, 230 GiB used, 843 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 928 MiB/s wr, 1.91k op/s 2024-02-18T11:45:53.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:45:52 smithi188 ceph-mon[36925]: pgmap v499: 97 pgs: 97 active+clean; 10 GiB data, 230 GiB used, 843 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 928 MiB/s wr, 1.91k op/s 2024-02-18T11:45:53.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:45:52 smithi012 ceph-mon[32460]: pgmap v499: 97 pgs: 97 active+clean; 10 GiB data, 230 GiB used, 843 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 928 MiB/s wr, 1.91k op/s 2024-02-18T11:45:55.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:45:55 smithi012 ceph-mon[32460]: pgmap v500: 97 pgs: 97 active+clean; 10 GiB data, 234 GiB used, 839 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 860 MiB/s wr, 1.75k op/s 2024-02-18T11:45:55.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:45:55 smithi158 ceph-mon[36626]: pgmap v500: 97 pgs: 97 active+clean; 10 GiB data, 234 GiB used, 839 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 860 MiB/s wr, 1.75k op/s 2024-02-18T11:45:55.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:45:55 smithi188 ceph-mon[36925]: pgmap v500: 97 pgs: 97 active+clean; 10 GiB data, 234 GiB used, 839 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 860 MiB/s wr, 1.75k op/s 2024-02-18T11:45:56.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:45:56 smithi158 ceph-mon[36626]: pgmap v501: 97 pgs: 97 active+clean; 10 GiB data, 237 GiB used, 835 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 836 MiB/s wr, 1.74k op/s 2024-02-18T11:45:56.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:45:56 smithi188 ceph-mon[36925]: pgmap v501: 97 pgs: 97 active+clean; 10 GiB data, 237 GiB used, 835 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 836 MiB/s wr, 1.74k op/s 2024-02-18T11:45:56.675 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:45:56 smithi012 ceph-mon[32460]: pgmap v501: 97 pgs: 97 active+clean; 10 GiB data, 237 GiB used, 835 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 836 MiB/s wr, 1.74k op/s 2024-02-18T11:45:58.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:45:58 smithi012 ceph-mon[32460]: pgmap v502: 97 pgs: 97 active+clean; 10 GiB data, 239 GiB used, 834 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 915 MiB/s wr, 2.02k op/s 2024-02-18T11:45:58.993 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:45:58 smithi158 ceph-mon[36626]: pgmap v502: 97 pgs: 97 active+clean; 10 GiB data, 239 GiB used, 834 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 915 MiB/s wr, 2.02k op/s 2024-02-18T11:45:59.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:45:58 smithi188 ceph-mon[36925]: pgmap v502: 97 pgs: 97 active+clean; 10 GiB data, 239 GiB used, 834 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 915 MiB/s wr, 2.02k op/s 2024-02-18T11:46:00.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:46:00 smithi158 ceph-mon[36626]: pgmap v503: 97 pgs: 97 active+clean; 10 GiB data, 244 GiB used, 829 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 902 MiB/s wr, 2.02k op/s 2024-02-18T11:46:01.104 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:46:00 smithi188 ceph-mon[36925]: pgmap v503: 97 pgs: 97 active+clean; 10 GiB data, 244 GiB used, 829 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 902 MiB/s wr, 2.02k op/s 2024-02-18T11:46:01.172 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:46:00 smithi012 ceph-mon[32460]: pgmap v503: 97 pgs: 97 active+clean; 10 GiB data, 244 GiB used, 829 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 902 MiB/s wr, 2.02k op/s 2024-02-18T11:46:03.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:46:02 smithi188 ceph-mon[36925]: pgmap v504: 97 pgs: 97 active+clean; 10 GiB data, 247 GiB used, 826 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 932 MiB/s wr, 1.94k op/s 2024-02-18T11:46:03.172 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:46:02 smithi012 ceph-mon[32460]: pgmap v504: 97 pgs: 97 active+clean; 10 GiB data, 247 GiB used, 826 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 932 MiB/s wr, 1.94k op/s 2024-02-18T11:46:03.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:46:02 smithi158 ceph-mon[36626]: pgmap v504: 97 pgs: 97 active+clean; 10 GiB data, 247 GiB used, 826 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 932 MiB/s wr, 1.94k op/s 2024-02-18T11:46:05.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:46:05 smithi012 ceph-mon[32460]: pgmap v505: 97 pgs: 97 active+clean; 10 GiB data, 251 GiB used, 822 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 856 MiB/s wr, 1.84k op/s 2024-02-18T11:46:05.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:46:05 smithi158 ceph-mon[36626]: pgmap v505: 97 pgs: 97 active+clean; 10 GiB data, 251 GiB used, 822 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 856 MiB/s wr, 1.84k op/s 2024-02-18T11:46:05.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:46:05 smithi188 ceph-mon[36925]: pgmap v505: 97 pgs: 97 active+clean; 10 GiB data, 251 GiB used, 822 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 856 MiB/s wr, 1.84k op/s 2024-02-18T11:46:06.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:46:06 smithi188 ceph-mon[36925]: pgmap v506: 97 pgs: 97 active+clean; 10 GiB data, 254 GiB used, 819 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 818 MiB/s wr, 1.75k op/s 2024-02-18T11:46:06.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:46:06 smithi012 ceph-mon[32460]: pgmap v506: 97 pgs: 97 active+clean; 10 GiB data, 254 GiB used, 819 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 818 MiB/s wr, 1.75k op/s 2024-02-18T11:46:06.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:46:06 smithi158 ceph-mon[36626]: pgmap v506: 97 pgs: 97 active+clean; 10 GiB data, 254 GiB used, 819 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 818 MiB/s wr, 1.75k op/s 2024-02-18T11:46:08.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:46:08 smithi188 ceph-mon[36925]: pgmap v507: 97 pgs: 97 active+clean; 10 GiB data, 256 GiB used, 817 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 887 MiB/s wr, 1.80k op/s 2024-02-18T11:46:08.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:46:08 smithi012 ceph-mon[32460]: pgmap v507: 97 pgs: 97 active+clean; 10 GiB data, 256 GiB used, 817 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 887 MiB/s wr, 1.80k op/s 2024-02-18T11:46:08.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:46:08 smithi158 ceph-mon[36626]: pgmap v507: 97 pgs: 97 active+clean; 10 GiB data, 256 GiB used, 817 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 887 MiB/s wr, 1.80k op/s 2024-02-18T11:46:10.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:46:10 smithi012 ceph-mon[32460]: pgmap v508: 97 pgs: 97 active+clean; 10 GiB data, 260 GiB used, 812 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 862 MiB/s wr, 1.65k op/s 2024-02-18T11:46:10.993 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:46:10 smithi158 ceph-mon[36626]: pgmap v508: 97 pgs: 97 active+clean; 10 GiB data, 260 GiB used, 812 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 862 MiB/s wr, 1.65k op/s 2024-02-18T11:46:11.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:46:10 smithi188 ceph-mon[36925]: pgmap v508: 97 pgs: 97 active+clean; 10 GiB data, 260 GiB used, 812 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 862 MiB/s wr, 1.65k op/s 2024-02-18T11:46:13.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:46:12 smithi188 ceph-mon[36925]: pgmap v509: 97 pgs: 97 active+clean; 10 GiB data, 264 GiB used, 809 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 897 MiB/s wr, 1.74k op/s 2024-02-18T11:46:13.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:46:12 smithi012 ceph-mon[32460]: pgmap v509: 97 pgs: 97 active+clean; 10 GiB data, 264 GiB used, 809 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 897 MiB/s wr, 1.74k op/s 2024-02-18T11:46:13.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:46:12 smithi158 ceph-mon[36626]: pgmap v509: 97 pgs: 97 active+clean; 10 GiB data, 264 GiB used, 809 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 897 MiB/s wr, 1.74k op/s 2024-02-18T11:46:15.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:46:14 smithi158 ceph-mon[36626]: pgmap v510: 97 pgs: 97 active+clean; 10 GiB data, 267 GiB used, 806 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 824 MiB/s wr, 1.61k op/s 2024-02-18T11:46:15.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:46:14 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:46:15.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:46:15 smithi188 ceph-mon[36925]: pgmap v510: 97 pgs: 97 active+clean; 10 GiB data, 267 GiB used, 806 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 824 MiB/s wr, 1.61k op/s 2024-02-18T11:46:15.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:46:15 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:46:15.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:46:14 smithi012 ceph-mon[32460]: pgmap v510: 97 pgs: 97 active+clean; 10 GiB data, 267 GiB used, 806 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 824 MiB/s wr, 1.61k op/s 2024-02-18T11:46:15.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:46:14 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:46:17.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:46:17 smithi188 ceph-mon[36925]: pgmap v511: 97 pgs: 97 active+clean; 10 GiB data, 270 GiB used, 803 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 778 MiB/s wr, 1.53k op/s 2024-02-18T11:46:17.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:46:17 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:46:17.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:46:17 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:46:17.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:46:17 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:46:17.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:46:17 smithi012 ceph-mon[32460]: pgmap v511: 97 pgs: 97 active+clean; 10 GiB data, 270 GiB used, 803 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 778 MiB/s wr, 1.53k op/s 2024-02-18T11:46:17.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:46:17 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:46:17.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:46:17 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:46:17.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:46:17 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:46:17.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:46:17 smithi158 ceph-mon[36626]: pgmap v511: 97 pgs: 97 active+clean; 10 GiB data, 270 GiB used, 803 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 778 MiB/s wr, 1.53k op/s 2024-02-18T11:46:17.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:46:17 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:46:17.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:46:17 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:46:17.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:46:17 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:46:18.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:46:18 smithi188 ceph-mon[36925]: pgmap v512: 97 pgs: 97 active+clean; 10 GiB data, 272 GiB used, 801 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 846 MiB/s wr, 1.64k op/s 2024-02-18T11:46:18.675 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:46:18 smithi012 ceph-mon[32460]: pgmap v512: 97 pgs: 97 active+clean; 10 GiB data, 272 GiB used, 801 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 846 MiB/s wr, 1.64k op/s 2024-02-18T11:46:18.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:46:18 smithi158 ceph-mon[36626]: pgmap v512: 97 pgs: 97 active+clean; 10 GiB data, 272 GiB used, 801 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 846 MiB/s wr, 1.64k op/s 2024-02-18T11:46:20.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:46:20 smithi158 ceph-mon[36626]: pgmap v513: 97 pgs: 97 active+clean; 10 GiB data, 276 GiB used, 797 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 832 MiB/s wr, 1.58k op/s 2024-02-18T11:46:20.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:46:20 smithi188 ceph-mon[36925]: pgmap v513: 97 pgs: 97 active+clean; 10 GiB data, 276 GiB used, 797 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 832 MiB/s wr, 1.58k op/s 2024-02-18T11:46:20.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:46:20 smithi012 ceph-mon[32460]: pgmap v513: 97 pgs: 97 active+clean; 10 GiB data, 276 GiB used, 797 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 832 MiB/s wr, 1.58k op/s 2024-02-18T11:46:23.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:46:22 smithi188 ceph-mon[36925]: pgmap v514: 97 pgs: 97 active+clean; 10 GiB data, 279 GiB used, 793 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 890 MiB/s wr, 1.47k op/s 2024-02-18T11:46:23.172 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:46:22 smithi012 ceph-mon[32460]: pgmap v514: 97 pgs: 97 active+clean; 10 GiB data, 279 GiB used, 793 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 890 MiB/s wr, 1.47k op/s 2024-02-18T11:46:23.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:46:22 smithi158 ceph-mon[36626]: pgmap v514: 97 pgs: 97 active+clean; 10 GiB data, 279 GiB used, 793 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 890 MiB/s wr, 1.47k op/s 2024-02-18T11:46:25.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:46:25 smithi188 ceph-mon[36925]: pgmap v515: 97 pgs: 97 active+clean; 10 GiB data, 282 GiB used, 790 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 789 MiB/s wr, 1.12k op/s 2024-02-18T11:46:25.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:46:25 smithi012 ceph-mon[32460]: pgmap v515: 97 pgs: 97 active+clean; 10 GiB data, 282 GiB used, 790 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 789 MiB/s wr, 1.12k op/s 2024-02-18T11:46:25.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:46:25 smithi158 ceph-mon[36626]: pgmap v515: 97 pgs: 97 active+clean; 10 GiB data, 282 GiB used, 790 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 789 MiB/s wr, 1.12k op/s 2024-02-18T11:46:27.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:46:27 smithi158 ceph-mon[36626]: pgmap v516: 97 pgs: 97 active+clean; 10 GiB data, 271 GiB used, 802 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 768 MiB/s wr, 1.16k op/s 2024-02-18T11:46:27.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:46:27 smithi188 ceph-mon[36925]: pgmap v516: 97 pgs: 97 active+clean; 10 GiB data, 271 GiB used, 802 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 768 MiB/s wr, 1.16k op/s 2024-02-18T11:46:27.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:46:27 smithi012 ceph-mon[32460]: pgmap v516: 97 pgs: 97 active+clean; 10 GiB data, 271 GiB used, 802 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 768 MiB/s wr, 1.16k op/s 2024-02-18T11:46:28.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:46:28 smithi188 ceph-mon[36925]: pgmap v517: 97 pgs: 97 active+clean; 10 GiB data, 273 GiB used, 800 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 865 MiB/s wr, 1.38k op/s 2024-02-18T11:46:28.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:46:28 smithi012 ceph-mon[32460]: pgmap v517: 97 pgs: 97 active+clean; 10 GiB data, 273 GiB used, 800 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 865 MiB/s wr, 1.38k op/s 2024-02-18T11:46:28.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:46:28 smithi158 ceph-mon[36626]: pgmap v517: 97 pgs: 97 active+clean; 10 GiB data, 273 GiB used, 800 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 865 MiB/s wr, 1.38k op/s 2024-02-18T11:46:30.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:46:30 smithi012 ceph-mon[32460]: pgmap v518: 97 pgs: 97 active+clean; 10 GiB data, 277 GiB used, 796 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 851 MiB/s wr, 1.34k op/s 2024-02-18T11:46:30.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:46:30 smithi158 ceph-mon[36626]: pgmap v518: 97 pgs: 97 active+clean; 10 GiB data, 277 GiB used, 796 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 851 MiB/s wr, 1.34k op/s 2024-02-18T11:46:30.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:46:30 smithi188 ceph-mon[36925]: pgmap v518: 97 pgs: 97 active+clean; 10 GiB data, 277 GiB used, 796 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 851 MiB/s wr, 1.34k op/s 2024-02-18T11:46:32.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:46:32 smithi188 ceph-mon[36925]: pgmap v519: 97 pgs: 97 active+clean; 10 GiB data, 266 GiB used, 807 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 908 MiB/s wr, 1.39k op/s 2024-02-18T11:46:32.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:46:32 smithi012 ceph-mon[32460]: pgmap v519: 97 pgs: 97 active+clean; 10 GiB data, 266 GiB used, 807 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 908 MiB/s wr, 1.39k op/s 2024-02-18T11:46:32.993 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:46:32 smithi158 ceph-mon[36626]: pgmap v519: 97 pgs: 97 active+clean; 10 GiB data, 266 GiB used, 807 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 908 MiB/s wr, 1.39k op/s 2024-02-18T11:46:34.993 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:46:34 smithi158 ceph-mon[36626]: pgmap v520: 97 pgs: 97 active+clean; 10 GiB data, 269 GiB used, 804 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 795 MiB/s wr, 1.25k op/s 2024-02-18T11:46:35.172 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:46:34 smithi012 ceph-mon[32460]: pgmap v520: 97 pgs: 97 active+clean; 10 GiB data, 269 GiB used, 804 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 795 MiB/s wr, 1.25k op/s 2024-02-18T11:46:35.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:46:34 smithi188 ceph-mon[36925]: pgmap v520: 97 pgs: 97 active+clean; 10 GiB data, 269 GiB used, 804 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 795 MiB/s wr, 1.25k op/s 2024-02-18T11:46:37.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:46:36 smithi158 ceph-mon[36626]: pgmap v521: 97 pgs: 97 active+clean; 10 GiB data, 272 GiB used, 801 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 805 MiB/s wr, 1.24k op/s 2024-02-18T11:46:37.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:46:37 smithi188 ceph-mon[36925]: pgmap v521: 97 pgs: 97 active+clean; 10 GiB data, 272 GiB used, 801 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 805 MiB/s wr, 1.24k op/s 2024-02-18T11:46:37.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:46:36 smithi012 ceph-mon[32460]: pgmap v521: 97 pgs: 97 active+clean; 10 GiB data, 272 GiB used, 801 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 805 MiB/s wr, 1.24k op/s 2024-02-18T11:46:38.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:46:38 smithi188 ceph-mon[36925]: pgmap v522: 97 pgs: 97 active+clean; 10 GiB data, 266 GiB used, 807 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 905 MiB/s wr, 1.37k op/s 2024-02-18T11:46:38.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:46:38 smithi012 ceph-mon[32460]: pgmap v522: 97 pgs: 97 active+clean; 10 GiB data, 266 GiB used, 807 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 905 MiB/s wr, 1.37k op/s 2024-02-18T11:46:38.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:46:38 smithi158 ceph-mon[36626]: pgmap v522: 97 pgs: 97 active+clean; 10 GiB data, 266 GiB used, 807 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 905 MiB/s wr, 1.37k op/s 2024-02-18T11:46:40.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:46:40 smithi012 ceph-mon[32460]: pgmap v523: 97 pgs: 97 active+clean; 10 GiB data, 270 GiB used, 803 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 846 MiB/s wr, 1.23k op/s 2024-02-18T11:46:40.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:46:40 smithi158 ceph-mon[36626]: pgmap v523: 97 pgs: 97 active+clean; 10 GiB data, 270 GiB used, 803 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 846 MiB/s wr, 1.23k op/s 2024-02-18T11:46:40.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:46:40 smithi188 ceph-mon[36925]: pgmap v523: 97 pgs: 97 active+clean; 10 GiB data, 270 GiB used, 803 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 846 MiB/s wr, 1.23k op/s 2024-02-18T11:46:42.993 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:46:42 smithi158 ceph-mon[36626]: pgmap v524: 97 pgs: 97 active+clean; 10 GiB data, 273 GiB used, 800 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 890 MiB/s wr, 1.27k op/s 2024-02-18T11:46:43.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:46:42 smithi188 ceph-mon[36925]: pgmap v524: 97 pgs: 97 active+clean; 10 GiB data, 273 GiB used, 800 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 890 MiB/s wr, 1.27k op/s 2024-02-18T11:46:43.172 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:46:42 smithi012 ceph-mon[32460]: pgmap v524: 97 pgs: 97 active+clean; 10 GiB data, 273 GiB used, 800 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 890 MiB/s wr, 1.27k op/s 2024-02-18T11:46:45.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:46:44 smithi188 ceph-mon[36925]: pgmap v525: 97 pgs: 97 active+clean; 10 GiB data, 277 GiB used, 796 GiB / 1.0 TiB avail; 1.2 MiB/s rd, 814 MiB/s wr, 1.14k op/s 2024-02-18T11:46:45.172 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:46:44 smithi012 ceph-mon[32460]: pgmap v525: 97 pgs: 97 active+clean; 10 GiB data, 277 GiB used, 796 GiB / 1.0 TiB avail; 1.2 MiB/s rd, 814 MiB/s wr, 1.14k op/s 2024-02-18T11:46:45.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:46:44 smithi158 ceph-mon[36626]: pgmap v525: 97 pgs: 97 active+clean; 10 GiB data, 277 GiB used, 796 GiB / 1.0 TiB avail; 1.2 MiB/s rd, 814 MiB/s wr, 1.14k op/s 2024-02-18T11:46:47.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:46:47 smithi188 ceph-mon[36925]: pgmap v526: 97 pgs: 97 active+clean; 10 GiB data, 279 GiB used, 794 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 773 MiB/s wr, 1.07k op/s 2024-02-18T11:46:47.423 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:46:47 smithi012 ceph-mon[32460]: pgmap v526: 97 pgs: 97 active+clean; 10 GiB data, 279 GiB used, 794 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 773 MiB/s wr, 1.07k op/s 2024-02-18T11:46:47.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:46:47 smithi158 ceph-mon[36626]: pgmap v526: 97 pgs: 97 active+clean; 10 GiB data, 279 GiB used, 794 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 773 MiB/s wr, 1.07k op/s 2024-02-18T11:46:49.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:46:49 smithi012 ceph-mon[32460]: pgmap v527: 97 pgs: 97 active+clean; 10 GiB data, 265 GiB used, 808 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 847 MiB/s wr, 1.15k op/s 2024-02-18T11:46:49.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:46:49 smithi158 ceph-mon[36626]: pgmap v527: 97 pgs: 97 active+clean; 10 GiB data, 265 GiB used, 808 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 847 MiB/s wr, 1.15k op/s 2024-02-18T11:46:49.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:46:49 smithi188 ceph-mon[36925]: pgmap v527: 97 pgs: 97 active+clean; 10 GiB data, 265 GiB used, 808 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 847 MiB/s wr, 1.15k op/s 2024-02-18T11:46:50.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:46:50 smithi012 ceph-mon[32460]: pgmap v528: 97 pgs: 97 active+clean; 10 GiB data, 270 GiB used, 803 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 819 MiB/s wr, 1.07k op/s 2024-02-18T11:46:50.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:46:50 smithi158 ceph-mon[36626]: pgmap v528: 97 pgs: 97 active+clean; 10 GiB data, 270 GiB used, 803 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 819 MiB/s wr, 1.07k op/s 2024-02-18T11:46:50.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:46:50 smithi188 ceph-mon[36925]: pgmap v528: 97 pgs: 97 active+clean; 10 GiB data, 270 GiB used, 803 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 819 MiB/s wr, 1.07k op/s 2024-02-18T11:46:53.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:46:53 smithi188 ceph-mon[36925]: pgmap v529: 97 pgs: 97 active+clean; 10 GiB data, 256 GiB used, 817 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 906 MiB/s wr, 1.21k op/s 2024-02-18T11:46:53.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:46:53 smithi012 ceph-mon[32460]: pgmap v529: 97 pgs: 97 active+clean; 10 GiB data, 256 GiB used, 817 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 906 MiB/s wr, 1.21k op/s 2024-02-18T11:46:53.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:46:53 smithi158 ceph-mon[36626]: pgmap v529: 97 pgs: 97 active+clean; 10 GiB data, 256 GiB used, 817 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 906 MiB/s wr, 1.21k op/s 2024-02-18T11:46:54.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:46:54 smithi012 ceph-mon[32460]: pgmap v530: 97 pgs: 97 active+clean; 10 GiB data, 259 GiB used, 813 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 813 MiB/s wr, 1.12k op/s 2024-02-18T11:46:54.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:46:54 smithi158 ceph-mon[36626]: pgmap v530: 97 pgs: 97 active+clean; 10 GiB data, 259 GiB used, 813 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 813 MiB/s wr, 1.12k op/s 2024-02-18T11:46:54.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:46:54 smithi188 ceph-mon[36925]: pgmap v530: 97 pgs: 97 active+clean; 10 GiB data, 259 GiB used, 813 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 813 MiB/s wr, 1.12k op/s 2024-02-18T11:46:56.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:46:56 smithi012 ceph-mon[32460]: pgmap v531: 97 pgs: 97 active+clean; 10 GiB data, 263 GiB used, 810 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 771 MiB/s wr, 1.15k op/s 2024-02-18T11:46:56.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:46:56 smithi158 ceph-mon[36626]: pgmap v531: 97 pgs: 97 active+clean; 10 GiB data, 263 GiB used, 810 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 771 MiB/s wr, 1.15k op/s 2024-02-18T11:46:56.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:46:56 smithi188 ceph-mon[36925]: pgmap v531: 97 pgs: 97 active+clean; 10 GiB data, 263 GiB used, 810 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 771 MiB/s wr, 1.15k op/s 2024-02-18T11:46:58.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:46:58 smithi158 ceph-mon[36626]: pgmap v532: 97 pgs: 97 active+clean; 10 GiB data, 265 GiB used, 808 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 890 MiB/s wr, 1.59k op/s 2024-02-18T11:46:58.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:46:58 smithi188 ceph-mon[36925]: pgmap v532: 97 pgs: 97 active+clean; 10 GiB data, 265 GiB used, 808 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 890 MiB/s wr, 1.59k op/s 2024-02-18T11:46:58.921 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:46:58 smithi012 ceph-mon[32460]: pgmap v532: 97 pgs: 97 active+clean; 10 GiB data, 265 GiB used, 808 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 890 MiB/s wr, 1.59k op/s 2024-02-18T11:47:00.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:47:00 smithi188 ceph-mon[36925]: pgmap v533: 97 pgs: 97 active+clean; 10 GiB data, 250 GiB used, 823 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 891 MiB/s wr, 2.19k op/s 2024-02-18T11:47:00.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:47:00 smithi012 ceph-mon[32460]: pgmap v533: 97 pgs: 97 active+clean; 10 GiB data, 250 GiB used, 823 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 891 MiB/s wr, 2.19k op/s 2024-02-18T11:47:00.993 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:47:00 smithi158 ceph-mon[36626]: pgmap v533: 97 pgs: 97 active+clean; 10 GiB data, 250 GiB used, 823 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 891 MiB/s wr, 2.19k op/s 2024-02-18T11:47:03.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:47:02 smithi188 ceph-mon[36925]: pgmap v534: 97 pgs: 97 active+clean; 10 GiB data, 253 GiB used, 820 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 939 MiB/s wr, 2.73k op/s 2024-02-18T11:47:03.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:47:02 smithi012 ceph-mon[32460]: pgmap v534: 97 pgs: 97 active+clean; 10 GiB data, 253 GiB used, 820 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 939 MiB/s wr, 2.73k op/s 2024-02-18T11:47:03.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:47:02 smithi158 ceph-mon[36626]: pgmap v534: 97 pgs: 97 active+clean; 10 GiB data, 253 GiB used, 820 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 939 MiB/s wr, 2.73k op/s 2024-02-18T11:47:05.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:47:05 smithi188 ceph-mon[36925]: pgmap v535: 97 pgs: 97 active+clean; 10 GiB data, 257 GiB used, 816 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 813 MiB/s wr, 2.52k op/s 2024-02-18T11:47:05.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:47:05 smithi012 ceph-mon[32460]: pgmap v535: 97 pgs: 97 active+clean; 10 GiB data, 257 GiB used, 816 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 813 MiB/s wr, 2.52k op/s 2024-02-18T11:47:05.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:47:05 smithi158 ceph-mon[36626]: pgmap v535: 97 pgs: 97 active+clean; 10 GiB data, 257 GiB used, 816 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 813 MiB/s wr, 2.52k op/s 2024-02-18T11:47:06.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:47:06 smithi188 ceph-mon[36925]: pgmap v536: 97 pgs: 97 active+clean; 10 GiB data, 246 GiB used, 826 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 815 MiB/s wr, 2.46k op/s 2024-02-18T11:47:06.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:47:06 smithi012 ceph-mon[32460]: pgmap v536: 97 pgs: 97 active+clean; 10 GiB data, 246 GiB used, 826 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 815 MiB/s wr, 2.46k op/s 2024-02-18T11:47:06.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:47:06 smithi158 ceph-mon[36626]: pgmap v536: 97 pgs: 97 active+clean; 10 GiB data, 246 GiB used, 826 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 815 MiB/s wr, 2.46k op/s 2024-02-18T11:47:08.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:47:08 smithi188 ceph-mon[36925]: pgmap v537: 97 pgs: 97 active+clean; 10 GiB data, 249 GiB used, 824 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 903 MiB/s wr, 2.58k op/s 2024-02-18T11:47:08.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:47:08 smithi012 ceph-mon[32460]: pgmap v537: 97 pgs: 97 active+clean; 10 GiB data, 249 GiB used, 824 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 903 MiB/s wr, 2.58k op/s 2024-02-18T11:47:08.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:47:08 smithi158 ceph-mon[36626]: pgmap v537: 97 pgs: 97 active+clean; 10 GiB data, 249 GiB used, 824 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 903 MiB/s wr, 2.58k op/s 2024-02-18T11:47:10.744 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:47:10 smithi158 ceph-mon[36626]: pgmap v538: 97 pgs: 97 active+clean; 10 GiB data, 253 GiB used, 820 GiB / 1.0 TiB avail; 1.2 MiB/s rd, 865 MiB/s wr, 2.34k op/s 2024-02-18T11:47:10.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:47:10 smithi188 ceph-mon[36925]: pgmap v538: 97 pgs: 97 active+clean; 10 GiB data, 253 GiB used, 820 GiB / 1.0 TiB avail; 1.2 MiB/s rd, 865 MiB/s wr, 2.34k op/s 2024-02-18T11:47:10.929 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:47:10 smithi012 ceph-mon[32460]: pgmap v538: 97 pgs: 97 active+clean; 10 GiB data, 253 GiB used, 820 GiB / 1.0 TiB avail; 1.2 MiB/s rd, 865 MiB/s wr, 2.34k op/s 2024-02-18T11:47:12.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:47:12 smithi012 ceph-mon[32460]: pgmap v539: 97 pgs: 97 active+clean; 10 GiB data, 256 GiB used, 816 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 906 MiB/s wr, 1.81k op/s 2024-02-18T11:47:12.993 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:47:12 smithi158 ceph-mon[36626]: pgmap v539: 97 pgs: 97 active+clean; 10 GiB data, 256 GiB used, 816 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 906 MiB/s wr, 1.81k op/s 2024-02-18T11:47:13.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:47:12 smithi188 ceph-mon[36925]: pgmap v539: 97 pgs: 97 active+clean; 10 GiB data, 256 GiB used, 816 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 906 MiB/s wr, 1.81k op/s 2024-02-18T11:47:14.993 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:47:14 smithi158 ceph-mon[36626]: pgmap v540: 97 pgs: 97 active+clean; 10 GiB data, 259 GiB used, 814 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 763 MiB/s wr, 1.11k op/s 2024-02-18T11:47:15.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:47:14 smithi188 ceph-mon[36925]: pgmap v540: 97 pgs: 97 active+clean; 10 GiB data, 259 GiB used, 814 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 763 MiB/s wr, 1.11k op/s 2024-02-18T11:47:15.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:47:14 smithi012 ceph-mon[32460]: pgmap v540: 97 pgs: 97 active+clean; 10 GiB data, 259 GiB used, 814 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 763 MiB/s wr, 1.11k op/s 2024-02-18T11:47:17.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:47:16 smithi188 ceph-mon[36925]: pgmap v541: 97 pgs: 97 active+clean; 10 GiB data, 262 GiB used, 811 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 769 MiB/s wr, 1.08k op/s 2024-02-18T11:47:17.172 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:47:16 smithi012 ceph-mon[32460]: pgmap v541: 97 pgs: 97 active+clean; 10 GiB data, 262 GiB used, 811 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 769 MiB/s wr, 1.08k op/s 2024-02-18T11:47:17.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:47:16 smithi158 ceph-mon[36626]: pgmap v541: 97 pgs: 97 active+clean; 10 GiB data, 262 GiB used, 811 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 769 MiB/s wr, 1.08k op/s 2024-02-18T11:47:18.177 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:47:17 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:47:18.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:47:17 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:47:18.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:47:17 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:47:19.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:47:18 smithi158 ceph-mon[36626]: pgmap v542: 97 pgs: 97 active+clean; 10 GiB data, 264 GiB used, 809 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 856 MiB/s wr, 1.15k op/s 2024-02-18T11:47:19.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:47:18 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:47:19.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:47:18 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:47:19.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:47:18 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:47:19.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:47:18 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:47:19.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:47:18 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:47:19.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:47:18 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:47:19.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:47:18 smithi188 ceph-mon[36925]: pgmap v542: 97 pgs: 97 active+clean; 10 GiB data, 264 GiB used, 809 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 856 MiB/s wr, 1.15k op/s 2024-02-18T11:47:19.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:47:18 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:47:19.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:47:18 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:47:19.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:47:18 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:47:19.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:47:18 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:47:19.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:47:18 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:47:19.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:47:18 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:47:19.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:47:18 smithi012 ceph-mon[32460]: pgmap v542: 97 pgs: 97 active+clean; 10 GiB data, 264 GiB used, 809 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 856 MiB/s wr, 1.15k op/s 2024-02-18T11:47:19.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:47:18 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:47:19.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:47:18 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:47:19.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:47:18 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:47:19.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:47:18 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:47:19.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:47:18 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:47:19.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:47:18 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:47:20.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:47:20 smithi012 ceph-mon[32460]: Adjusting osd_memory_target on smithi012 to 177.2M 2024-02-18T11:47:20.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:47:20 smithi012 ceph-mon[32460]: Unable to set osd_memory_target on smithi012 to 185895628: error parsing value: Value '185895628' is below minimum 939524096 2024-02-18T11:47:20.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:47:20 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:47:20.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:47:20 smithi158 ceph-mon[36626]: Adjusting osd_memory_target on smithi012 to 177.2M 2024-02-18T11:47:20.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:47:20 smithi158 ceph-mon[36626]: Unable to set osd_memory_target on smithi012 to 185895628: error parsing value: Value '185895628' is below minimum 939524096 2024-02-18T11:47:20.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:47:20 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:47:20.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:47:20 smithi188 ceph-mon[36925]: Adjusting osd_memory_target on smithi012 to 177.2M 2024-02-18T11:47:20.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:47:20 smithi188 ceph-mon[36925]: Unable to set osd_memory_target on smithi012 to 185895628: error parsing value: Value '185895628' is below minimum 939524096 2024-02-18T11:47:20.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:47:20 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:47:21.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:47:21 smithi188 ceph-mon[36925]: pgmap v543: 97 pgs: 97 active+clean; 10 GiB data, 269 GiB used, 804 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 817 MiB/s wr, 992 op/s 2024-02-18T11:47:21.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:47:21 smithi012 ceph-mon[32460]: pgmap v543: 97 pgs: 97 active+clean; 10 GiB data, 269 GiB used, 804 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 817 MiB/s wr, 992 op/s 2024-02-18T11:47:21.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:47:21 smithi158 ceph-mon[36626]: pgmap v543: 97 pgs: 97 active+clean; 10 GiB data, 269 GiB used, 804 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 817 MiB/s wr, 992 op/s 2024-02-18T11:47:22.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:47:22 smithi158 ceph-mon[36626]: pgmap v544: 97 pgs: 97 active+clean; 10 GiB data, 273 GiB used, 800 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 921 MiB/s wr, 1.04k op/s 2024-02-18T11:47:22.926 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:47:22 smithi012 ceph-mon[32460]: pgmap v544: 97 pgs: 97 active+clean; 10 GiB data, 273 GiB used, 800 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 921 MiB/s wr, 1.04k op/s 2024-02-18T11:47:23.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:47:22 smithi188 ceph-mon[36925]: pgmap v544: 97 pgs: 97 active+clean; 10 GiB data, 273 GiB used, 800 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 921 MiB/s wr, 1.04k op/s 2024-02-18T11:47:24.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:47:24 smithi012 ceph-mon[32460]: pgmap v545: 97 pgs: 97 active+clean; 10 GiB data, 276 GiB used, 797 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 793 MiB/s wr, 882 op/s 2024-02-18T11:47:24.993 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:47:24 smithi158 ceph-mon[36626]: pgmap v545: 97 pgs: 97 active+clean; 10 GiB data, 276 GiB used, 797 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 793 MiB/s wr, 882 op/s 2024-02-18T11:47:25.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:47:24 smithi188 ceph-mon[36925]: pgmap v545: 97 pgs: 97 active+clean; 10 GiB data, 276 GiB used, 797 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 793 MiB/s wr, 882 op/s 2024-02-18T11:47:27.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:47:26 smithi188 ceph-mon[36925]: pgmap v546: 97 pgs: 97 active+clean; 10 GiB data, 279 GiB used, 793 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 825 MiB/s wr, 961 op/s 2024-02-18T11:47:27.172 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:47:26 smithi012 ceph-mon[32460]: pgmap v546: 97 pgs: 97 active+clean; 10 GiB data, 279 GiB used, 793 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 825 MiB/s wr, 961 op/s 2024-02-18T11:47:27.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:47:26 smithi158 ceph-mon[36626]: pgmap v546: 97 pgs: 97 active+clean; 10 GiB data, 279 GiB used, 793 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 825 MiB/s wr, 961 op/s 2024-02-18T11:47:29.176 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:47:28 smithi012 ceph-mon[32460]: pgmap v547: 97 pgs: 97 active+clean; 10 GiB data, 282 GiB used, 791 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 925 MiB/s wr, 1.36k op/s 2024-02-18T11:47:29.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:47:28 smithi158 ceph-mon[36626]: pgmap v547: 97 pgs: 97 active+clean; 10 GiB data, 282 GiB used, 791 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 925 MiB/s wr, 1.36k op/s 2024-02-18T11:47:29.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:47:28 smithi188 ceph-mon[36925]: pgmap v547: 97 pgs: 97 active+clean; 10 GiB data, 282 GiB used, 791 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 925 MiB/s wr, 1.36k op/s 2024-02-18T11:47:31.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:47:31 smithi158 ceph-mon[36626]: pgmap v548: 97 pgs: 97 active+clean; 10 GiB data, 286 GiB used, 787 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 877 MiB/s wr, 1.57k op/s 2024-02-18T11:47:31.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:47:31 smithi188 ceph-mon[36925]: pgmap v548: 97 pgs: 97 active+clean; 10 GiB data, 286 GiB used, 787 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 877 MiB/s wr, 1.57k op/s 2024-02-18T11:47:31.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:47:31 smithi012 ceph-mon[32460]: pgmap v548: 97 pgs: 97 active+clean; 10 GiB data, 286 GiB used, 787 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 877 MiB/s wr, 1.57k op/s 2024-02-18T11:47:32.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:47:32 smithi188 ceph-mon[36925]: pgmap v549: 97 pgs: 97 active+clean; 10 GiB data, 290 GiB used, 783 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 982 MiB/s wr, 2.15k op/s 2024-02-18T11:47:32.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:47:32 smithi012 ceph-mon[32460]: pgmap v549: 97 pgs: 97 active+clean; 10 GiB data, 290 GiB used, 783 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 982 MiB/s wr, 2.15k op/s 2024-02-18T11:47:32.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:47:32 smithi158 ceph-mon[36626]: pgmap v549: 97 pgs: 97 active+clean; 10 GiB data, 290 GiB used, 783 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 982 MiB/s wr, 2.15k op/s 2024-02-18T11:47:34.675 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:47:34 smithi012 ceph-mon[32460]: pgmap v550: 97 pgs: 97 active+clean; 10 GiB data, 293 GiB used, 779 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 837 MiB/s wr, 2.10k op/s 2024-02-18T11:47:34.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:47:34 smithi158 ceph-mon[36626]: pgmap v550: 97 pgs: 97 active+clean; 10 GiB data, 293 GiB used, 779 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 837 MiB/s wr, 2.10k op/s 2024-02-18T11:47:34.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:47:34 smithi188 ceph-mon[36925]: pgmap v550: 97 pgs: 97 active+clean; 10 GiB data, 293 GiB used, 779 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 837 MiB/s wr, 2.10k op/s 2024-02-18T11:47:36.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:47:36 smithi188 ceph-mon[36925]: pgmap v551: 97 pgs: 97 active+clean; 10 GiB data, 297 GiB used, 776 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 845 MiB/s wr, 2.17k op/s 2024-02-18T11:47:36.929 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:47:36 smithi012 ceph-mon[32460]: pgmap v551: 97 pgs: 97 active+clean; 10 GiB data, 297 GiB used, 776 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 845 MiB/s wr, 2.17k op/s 2024-02-18T11:47:36.993 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:47:36 smithi158 ceph-mon[36626]: pgmap v551: 97 pgs: 97 active+clean; 10 GiB data, 297 GiB used, 776 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 845 MiB/s wr, 2.17k op/s 2024-02-18T11:47:38.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:47:38 smithi012 ceph-mon[32460]: pgmap v552: 97 pgs: 97 active+clean; 10 GiB data, 299 GiB used, 774 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 909 MiB/s wr, 2.37k op/s 2024-02-18T11:47:38.993 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:47:38 smithi158 ceph-mon[36626]: pgmap v552: 97 pgs: 97 active+clean; 10 GiB data, 299 GiB used, 774 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 909 MiB/s wr, 2.37k op/s 2024-02-18T11:47:39.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:47:38 smithi188 ceph-mon[36925]: pgmap v552: 97 pgs: 97 active+clean; 10 GiB data, 299 GiB used, 774 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 909 MiB/s wr, 2.37k op/s 2024-02-18T11:47:41.172 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:47:40 smithi012 ceph-mon[32460]: pgmap v553: 97 pgs: 97 active+clean; 10 GiB data, 302 GiB used, 770 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 836 MiB/s wr, 2.02k op/s 2024-02-18T11:47:41.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:47:40 smithi158 ceph-mon[36626]: pgmap v553: 97 pgs: 97 active+clean; 10 GiB data, 302 GiB used, 770 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 836 MiB/s wr, 2.02k op/s 2024-02-18T11:47:41.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:47:40 smithi188 ceph-mon[36925]: pgmap v553: 97 pgs: 97 active+clean; 10 GiB data, 302 GiB used, 770 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 836 MiB/s wr, 2.02k op/s 2024-02-18T11:47:43.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:47:42 smithi158 ceph-mon[36626]: pgmap v554: 97 pgs: 97 active+clean; 10 GiB data, 305 GiB used, 768 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 902 MiB/s wr, 2.00k op/s 2024-02-18T11:47:43.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:47:42 smithi188 ceph-mon[36925]: pgmap v554: 97 pgs: 97 active+clean; 10 GiB data, 305 GiB used, 768 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 902 MiB/s wr, 2.00k op/s 2024-02-18T11:47:43.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:47:42 smithi012 ceph-mon[32460]: pgmap v554: 97 pgs: 97 active+clean; 10 GiB data, 305 GiB used, 768 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 902 MiB/s wr, 2.00k op/s 2024-02-18T11:47:45.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:47:45 smithi188 ceph-mon[36925]: pgmap v555: 97 pgs: 97 active+clean; 10 GiB data, 287 GiB used, 786 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 785 MiB/s wr, 1.63k op/s 2024-02-18T11:47:45.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:47:45 smithi012 ceph-mon[32460]: pgmap v555: 97 pgs: 97 active+clean; 10 GiB data, 287 GiB used, 786 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 785 MiB/s wr, 1.63k op/s 2024-02-18T11:47:45.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:47:45 smithi158 ceph-mon[36626]: pgmap v555: 97 pgs: 97 active+clean; 10 GiB data, 287 GiB used, 786 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 785 MiB/s wr, 1.63k op/s 2024-02-18T11:47:46.678 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:47:46 smithi012 ceph-mon[32460]: pgmap v556: 97 pgs: 97 active+clean; 10 GiB data, 290 GiB used, 782 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 786 MiB/s wr, 1.73k op/s 2024-02-18T11:47:46.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:47:46 smithi158 ceph-mon[36626]: pgmap v556: 97 pgs: 97 active+clean; 10 GiB data, 290 GiB used, 782 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 786 MiB/s wr, 1.73k op/s 2024-02-18T11:47:46.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:47:46 smithi188 ceph-mon[36925]: pgmap v556: 97 pgs: 97 active+clean; 10 GiB data, 290 GiB used, 782 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 786 MiB/s wr, 1.73k op/s 2024-02-18T11:47:48.993 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:47:48 smithi158 ceph-mon[36626]: pgmap v557: 97 pgs: 97 active+clean; 10 GiB data, 293 GiB used, 780 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 867 MiB/s wr, 2.36k op/s 2024-02-18T11:47:49.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:47:48 smithi188 ceph-mon[36925]: pgmap v557: 97 pgs: 97 active+clean; 10 GiB data, 293 GiB used, 780 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 867 MiB/s wr, 2.36k op/s 2024-02-18T11:47:49.172 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:47:48 smithi012 ceph-mon[32460]: pgmap v557: 97 pgs: 97 active+clean; 10 GiB data, 293 GiB used, 780 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 867 MiB/s wr, 2.36k op/s 2024-02-18T11:47:51.172 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:47:50 smithi012 ceph-mon[32460]: pgmap v558: 97 pgs: 97 active+clean; 10 GiB data, 296 GiB used, 777 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 823 MiB/s wr, 2.55k op/s 2024-02-18T11:47:51.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:47:50 smithi158 ceph-mon[36626]: pgmap v558: 97 pgs: 97 active+clean; 10 GiB data, 296 GiB used, 777 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 823 MiB/s wr, 2.55k op/s 2024-02-18T11:47:51.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:47:50 smithi188 ceph-mon[36925]: pgmap v558: 97 pgs: 97 active+clean; 10 GiB data, 296 GiB used, 777 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 823 MiB/s wr, 2.55k op/s 2024-02-18T11:47:53.423 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:47:53 smithi012 ceph-mon[32460]: pgmap v559: 97 pgs: 97 active+clean; 10 GiB data, 299 GiB used, 774 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 889 MiB/s wr, 2.93k op/s 2024-02-18T11:47:53.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:47:53 smithi158 ceph-mon[36626]: pgmap v559: 97 pgs: 97 active+clean; 10 GiB data, 299 GiB used, 774 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 889 MiB/s wr, 2.93k op/s 2024-02-18T11:47:53.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:47:53 smithi188 ceph-mon[36925]: pgmap v559: 97 pgs: 97 active+clean; 10 GiB data, 299 GiB used, 774 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 889 MiB/s wr, 2.93k op/s 2024-02-18T11:47:54.993 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:47:54 smithi158 ceph-mon[36626]: pgmap v560: 97 pgs: 97 active+clean; 10 GiB data, 304 GiB used, 769 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 819 MiB/s wr, 2.88k op/s 2024-02-18T11:47:55.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:47:54 smithi188 ceph-mon[36925]: pgmap v560: 97 pgs: 97 active+clean; 10 GiB data, 304 GiB used, 769 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 819 MiB/s wr, 2.88k op/s 2024-02-18T11:47:55.172 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:47:54 smithi012 ceph-mon[32460]: pgmap v560: 97 pgs: 97 active+clean; 10 GiB data, 304 GiB used, 769 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 819 MiB/s wr, 2.88k op/s 2024-02-18T11:47:57.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:47:56 smithi158 ceph-mon[36626]: pgmap v561: 97 pgs: 97 active+clean; 10 GiB data, 307 GiB used, 766 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 787 MiB/s wr, 2.75k op/s 2024-02-18T11:47:57.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:47:56 smithi188 ceph-mon[36925]: pgmap v561: 97 pgs: 97 active+clean; 10 GiB data, 307 GiB used, 766 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 787 MiB/s wr, 2.75k op/s 2024-02-18T11:47:57.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:47:56 smithi012 ceph-mon[32460]: pgmap v561: 97 pgs: 97 active+clean; 10 GiB data, 307 GiB used, 766 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 787 MiB/s wr, 2.75k op/s 2024-02-18T11:47:59.423 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:47:59 smithi012 ceph-mon[32460]: pgmap v562: 97 pgs: 97 active+clean; 10 GiB data, 309 GiB used, 764 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 877 MiB/s wr, 2.83k op/s 2024-02-18T11:47:59.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:47:59 smithi158 ceph-mon[36626]: pgmap v562: 97 pgs: 97 active+clean; 10 GiB data, 309 GiB used, 764 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 877 MiB/s wr, 2.83k op/s 2024-02-18T11:47:59.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:47:59 smithi188 ceph-mon[36925]: pgmap v562: 97 pgs: 97 active+clean; 10 GiB data, 309 GiB used, 764 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 877 MiB/s wr, 2.83k op/s 2024-02-18T11:48:01.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:48:01 smithi158 ceph-mon[36626]: pgmap v563: 97 pgs: 97 active+clean; 10 GiB data, 282 GiB used, 791 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 833 MiB/s wr, 2.33k op/s 2024-02-18T11:48:01.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:48:01 smithi012 ceph-mon[32460]: pgmap v563: 97 pgs: 97 active+clean; 10 GiB data, 282 GiB used, 791 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 833 MiB/s wr, 2.33k op/s 2024-02-18T11:48:01.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:48:01 smithi188 ceph-mon[36925]: pgmap v563: 97 pgs: 97 active+clean; 10 GiB data, 282 GiB used, 791 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 833 MiB/s wr, 2.33k op/s 2024-02-18T11:48:03.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:48:02 smithi158 ceph-mon[36626]: pgmap v564: 97 pgs: 97 active+clean; 10 GiB data, 285 GiB used, 788 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 915 MiB/s wr, 2.08k op/s 2024-02-18T11:48:03.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:48:02 smithi188 ceph-mon[36925]: pgmap v564: 97 pgs: 97 active+clean; 10 GiB data, 285 GiB used, 788 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 915 MiB/s wr, 2.08k op/s 2024-02-18T11:48:03.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:48:02 smithi012 ceph-mon[32460]: pgmap v564: 97 pgs: 97 active+clean; 10 GiB data, 285 GiB used, 788 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 915 MiB/s wr, 2.08k op/s 2024-02-18T11:48:05.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:48:05 smithi012 ceph-mon[32460]: pgmap v565: 97 pgs: 97 active+clean; 10 GiB data, 289 GiB used, 784 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 842 MiB/s wr, 1.74k op/s 2024-02-18T11:48:05.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:48:05 smithi158 ceph-mon[36626]: pgmap v565: 97 pgs: 97 active+clean; 10 GiB data, 289 GiB used, 784 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 842 MiB/s wr, 1.74k op/s 2024-02-18T11:48:05.602 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:48:05 smithi188 ceph-mon[36925]: pgmap v565: 97 pgs: 97 active+clean; 10 GiB data, 289 GiB used, 784 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 842 MiB/s wr, 1.74k op/s 2024-02-18T11:48:07.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:48:07 smithi158 ceph-mon[36626]: pgmap v566: 97 pgs: 97 active+clean; 10 GiB data, 292 GiB used, 781 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 805 MiB/s wr, 1.57k op/s 2024-02-18T11:48:07.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:48:07 smithi188 ceph-mon[36925]: pgmap v566: 97 pgs: 97 active+clean; 10 GiB data, 292 GiB used, 781 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 805 MiB/s wr, 1.57k op/s 2024-02-18T11:48:07.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:48:07 smithi012 ceph-mon[32460]: pgmap v566: 97 pgs: 97 active+clean; 10 GiB data, 292 GiB used, 781 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 805 MiB/s wr, 1.57k op/s 2024-02-18T11:48:08.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:48:08 smithi188 ceph-mon[36925]: pgmap v567: 97 pgs: 97 active+clean; 10 GiB data, 294 GiB used, 779 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 895 MiB/s wr, 1.84k op/s 2024-02-18T11:48:08.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:48:08 smithi012 ceph-mon[32460]: pgmap v567: 97 pgs: 97 active+clean; 10 GiB data, 294 GiB used, 779 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 895 MiB/s wr, 1.84k op/s 2024-02-18T11:48:08.993 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:48:08 smithi158 ceph-mon[36626]: pgmap v567: 97 pgs: 97 active+clean; 10 GiB data, 294 GiB used, 779 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 895 MiB/s wr, 1.84k op/s 2024-02-18T11:48:10.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:48:10 smithi188 ceph-mon[36925]: pgmap v568: 97 pgs: 97 active+clean; 10 GiB data, 299 GiB used, 774 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 851 MiB/s wr, 1.77k op/s 2024-02-18T11:48:10.925 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:48:10 smithi012 ceph-mon[32460]: pgmap v568: 97 pgs: 97 active+clean; 10 GiB data, 299 GiB used, 774 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 851 MiB/s wr, 1.77k op/s 2024-02-18T11:48:10.993 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:48:10 smithi158 ceph-mon[36626]: pgmap v568: 97 pgs: 97 active+clean; 10 GiB data, 299 GiB used, 774 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 851 MiB/s wr, 1.77k op/s 2024-02-18T11:48:12.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:48:12 smithi012 ceph-mon[32460]: pgmap v569: 97 pgs: 97 active+clean; 10 GiB data, 302 GiB used, 771 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 918 MiB/s wr, 2.02k op/s 2024-02-18T11:48:12.993 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:48:12 smithi158 ceph-mon[36626]: pgmap v569: 97 pgs: 97 active+clean; 10 GiB data, 302 GiB used, 771 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 918 MiB/s wr, 2.02k op/s 2024-02-18T11:48:13.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:48:12 smithi188 ceph-mon[36925]: pgmap v569: 97 pgs: 97 active+clean; 10 GiB data, 302 GiB used, 771 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 918 MiB/s wr, 2.02k op/s 2024-02-18T11:48:15.181 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:48:14 smithi012 ceph-mon[32460]: pgmap v570: 97 pgs: 97 active+clean; 10 GiB data, 260 GiB used, 813 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 848 MiB/s wr, 2.02k op/s 2024-02-18T11:48:15.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:48:14 smithi158 ceph-mon[36626]: pgmap v570: 97 pgs: 97 active+clean; 10 GiB data, 260 GiB used, 813 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 848 MiB/s wr, 2.02k op/s 2024-02-18T11:48:15.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:48:14 smithi188 ceph-mon[36925]: pgmap v570: 97 pgs: 97 active+clean; 10 GiB data, 260 GiB used, 813 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 848 MiB/s wr, 2.02k op/s 2024-02-18T11:48:16.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:48:16 smithi188 ceph-mon[36925]: pgmap v571: 97 pgs: 97 active+clean; 10 GiB data, 264 GiB used, 809 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 837 MiB/s wr, 1.96k op/s 2024-02-18T11:48:16.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:48:16 smithi012 ceph-mon[32460]: pgmap v571: 97 pgs: 97 active+clean; 10 GiB data, 264 GiB used, 809 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 837 MiB/s wr, 1.96k op/s 2024-02-18T11:48:16.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:48:16 smithi158 ceph-mon[36626]: pgmap v571: 97 pgs: 97 active+clean; 10 GiB data, 264 GiB used, 809 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 837 MiB/s wr, 1.96k op/s 2024-02-18T11:48:18.993 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:48:18 smithi158 ceph-mon[36626]: pgmap v572: 97 pgs: 97 active+clean; 10 GiB data, 266 GiB used, 807 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 923 MiB/s wr, 2.05k op/s 2024-02-18T11:48:19.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:48:18 smithi188 ceph-mon[36925]: pgmap v572: 97 pgs: 97 active+clean; 10 GiB data, 266 GiB used, 807 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 923 MiB/s wr, 2.05k op/s 2024-02-18T11:48:19.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:48:18 smithi012 ceph-mon[32460]: pgmap v572: 97 pgs: 97 active+clean; 10 GiB data, 266 GiB used, 807 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 923 MiB/s wr, 2.05k op/s 2024-02-18T11:48:20.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:48:19 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:48:20.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:48:19 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:48:20.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:48:19 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:48:21.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:48:20 smithi158 ceph-mon[36626]: pgmap v573: 97 pgs: 97 active+clean; 10 GiB data, 270 GiB used, 803 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 872 MiB/s wr, 1.79k op/s 2024-02-18T11:48:21.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:48:20 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:48:21.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:48:20 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:48:21.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:48:20 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:48:21.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:48:20 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:48:21.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:48:20 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:48:21.244 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:48:20 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:48:21.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:48:20 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:48:21.245 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:48:20 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:48:21.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:48:20 smithi188 ceph-mon[36925]: pgmap v573: 97 pgs: 97 active+clean; 10 GiB data, 270 GiB used, 803 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 872 MiB/s wr, 1.79k op/s 2024-02-18T11:48:21.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:48:20 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:48:21.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:48:20 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:48:21.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:48:20 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:48:21.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:48:20 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:48:21.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:48:20 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:48:21.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:48:20 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:48:21.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:48:20 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:48:21.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:48:20 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:48:21.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:48:20 smithi012 ceph-mon[32460]: pgmap v573: 97 pgs: 97 active+clean; 10 GiB data, 270 GiB used, 803 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 872 MiB/s wr, 1.79k op/s 2024-02-18T11:48:21.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:48:20 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:48:21.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:48:20 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:48:21.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:48:20 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:48:21.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:48:20 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:48:21.423 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:48:20 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:48:21.423 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:48:20 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:48:21.423 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:48:20 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:48:21.423 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:48:20 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-18T11:48:22.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:48:22 smithi188 ceph-mon[36925]: Adjusting osd_memory_target on smithi188 to 174.1M 2024-02-18T11:48:22.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:48:22 smithi188 ceph-mon[36925]: Unable to set osd_memory_target on smithi188 to 182616268: error parsing value: Value '182616268' is below minimum 939524096 2024-02-18T11:48:22.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:48:22 smithi188 ceph-mon[36925]: Adjusting osd_memory_target on smithi158 to 174.1M 2024-02-18T11:48:22.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:48:22 smithi188 ceph-mon[36925]: Unable to set osd_memory_target on smithi158 to 182634905: error parsing value: Value '182634905' is below minimum 939524096 2024-02-18T11:48:22.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:48:22 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:48:22.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:48:22 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:48:22.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:48:22 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:48:22.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:48:22 smithi012 ceph-mon[32460]: Adjusting osd_memory_target on smithi188 to 174.1M 2024-02-18T11:48:22.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:48:22 smithi012 ceph-mon[32460]: Unable to set osd_memory_target on smithi188 to 182616268: error parsing value: Value '182616268' is below minimum 939524096 2024-02-18T11:48:22.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:48:22 smithi012 ceph-mon[32460]: Adjusting osd_memory_target on smithi158 to 174.1M 2024-02-18T11:48:22.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:48:22 smithi012 ceph-mon[32460]: Unable to set osd_memory_target on smithi158 to 182634905: error parsing value: Value '182634905' is below minimum 939524096 2024-02-18T11:48:22.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:48:22 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:48:22.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:48:22 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:48:22.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:48:22 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:48:22.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:48:22 smithi158 ceph-mon[36626]: Adjusting osd_memory_target on smithi188 to 174.1M 2024-02-18T11:48:22.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:48:22 smithi158 ceph-mon[36626]: Unable to set osd_memory_target on smithi188 to 182616268: error parsing value: Value '182616268' is below minimum 939524096 2024-02-18T11:48:22.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:48:22 smithi158 ceph-mon[36626]: Adjusting osd_memory_target on smithi158 to 174.1M 2024-02-18T11:48:22.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:48:22 smithi158 ceph-mon[36626]: Unable to set osd_memory_target on smithi158 to 182634905: error parsing value: Value '182634905' is below minimum 939524096 2024-02-18T11:48:22.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:48:22 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:48:22.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:48:22 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:48:22.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:48:22 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:48:23.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:48:23 smithi158 ceph-mon[36626]: pgmap v574: 97 pgs: 97 active+clean; 10 GiB data, 273 GiB used, 800 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 933 MiB/s wr, 1.76k op/s 2024-02-18T11:48:23.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:48:23 smithi188 ceph-mon[36925]: pgmap v574: 97 pgs: 97 active+clean; 10 GiB data, 273 GiB used, 800 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 933 MiB/s wr, 1.76k op/s 2024-02-18T11:48:23.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:48:23 smithi012 ceph-mon[32460]: pgmap v574: 97 pgs: 97 active+clean; 10 GiB data, 273 GiB used, 800 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 933 MiB/s wr, 1.76k op/s 2024-02-18T11:48:25.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:48:24 smithi188 ceph-mon[36925]: pgmap v575: 97 pgs: 97 active+clean; 10 GiB data, 277 GiB used, 796 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 857 MiB/s wr, 1.34k op/s 2024-02-18T11:48:25.172 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:48:24 smithi012 ceph-mon[32460]: pgmap v575: 97 pgs: 97 active+clean; 10 GiB data, 277 GiB used, 796 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 857 MiB/s wr, 1.34k op/s 2024-02-18T11:48:25.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:48:24 smithi158 ceph-mon[36626]: pgmap v575: 97 pgs: 97 active+clean; 10 GiB data, 277 GiB used, 796 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 857 MiB/s wr, 1.34k op/s 2024-02-18T11:48:27.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:48:26 smithi158 ceph-mon[36626]: pgmap v576: 97 pgs: 97 active+clean; 10 GiB data, 280 GiB used, 793 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 816 MiB/s wr, 1.18k op/s 2024-02-18T11:48:27.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:48:26 smithi188 ceph-mon[36925]: pgmap v576: 97 pgs: 97 active+clean; 10 GiB data, 280 GiB used, 793 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 816 MiB/s wr, 1.18k op/s 2024-02-18T11:48:27.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:48:26 smithi012 ceph-mon[32460]: pgmap v576: 97 pgs: 97 active+clean; 10 GiB data, 280 GiB used, 793 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 816 MiB/s wr, 1.18k op/s 2024-02-18T11:48:29.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:48:29 smithi188 ceph-mon[36925]: pgmap v577: 97 pgs: 97 active+clean; 10 GiB data, 283 GiB used, 790 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 896 MiB/s wr, 1.30k op/s 2024-02-18T11:48:29.425 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:48:29 smithi012 ceph-mon[32460]: pgmap v577: 97 pgs: 97 active+clean; 10 GiB data, 283 GiB used, 790 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 896 MiB/s wr, 1.30k op/s 2024-02-18T11:48:29.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:48:29 smithi158 ceph-mon[36626]: pgmap v577: 97 pgs: 97 active+clean; 10 GiB data, 283 GiB used, 790 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 896 MiB/s wr, 1.30k op/s 2024-02-18T11:48:30.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:48:30 smithi188 ceph-mon[36925]: pgmap v578: 97 pgs: 97 active+clean; 10 GiB data, 285 GiB used, 787 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 800 MiB/s wr, 1.18k op/s 2024-02-18T11:48:30.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:48:30 smithi012 ceph-mon[32460]: pgmap v578: 97 pgs: 97 active+clean; 10 GiB data, 285 GiB used, 787 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 800 MiB/s wr, 1.18k op/s 2024-02-18T11:48:30.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:48:30 smithi158 ceph-mon[36626]: pgmap v578: 97 pgs: 97 active+clean; 10 GiB data, 285 GiB used, 787 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 800 MiB/s wr, 1.18k op/s 2024-02-18T11:48:32.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:48:32 smithi158 ceph-mon[36626]: pgmap v579: 97 pgs: 97 active+clean; 10 GiB data, 288 GiB used, 785 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 870 MiB/s wr, 1.42k op/s 2024-02-18T11:48:32.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:48:32 smithi188 ceph-mon[36925]: pgmap v579: 97 pgs: 97 active+clean; 10 GiB data, 288 GiB used, 785 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 870 MiB/s wr, 1.42k op/s 2024-02-18T11:48:32.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:48:32 smithi012 ceph-mon[32460]: pgmap v579: 97 pgs: 97 active+clean; 10 GiB data, 288 GiB used, 785 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 870 MiB/s wr, 1.42k op/s 2024-02-18T11:48:35.176 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:48:34 smithi012 ceph-mon[32460]: pgmap v580: 97 pgs: 97 active+clean; 10 GiB data, 294 GiB used, 779 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 843 MiB/s wr, 1.48k op/s 2024-02-18T11:48:35.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:48:34 smithi158 ceph-mon[36626]: pgmap v580: 97 pgs: 97 active+clean; 10 GiB data, 294 GiB used, 779 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 843 MiB/s wr, 1.48k op/s 2024-02-18T11:48:35.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:48:34 smithi188 ceph-mon[36925]: pgmap v580: 97 pgs: 97 active+clean; 10 GiB data, 294 GiB used, 779 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 843 MiB/s wr, 1.48k op/s 2024-02-18T11:48:37.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:48:37 smithi158 ceph-mon[36626]: pgmap v581: 97 pgs: 97 active+clean; 10 GiB data, 296 GiB used, 777 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 776 MiB/s wr, 1.37k op/s 2024-02-18T11:48:37.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:48:37 smithi188 ceph-mon[36925]: pgmap v581: 97 pgs: 97 active+clean; 10 GiB data, 296 GiB used, 777 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 776 MiB/s wr, 1.37k op/s 2024-02-18T11:48:37.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:48:37 smithi012 ceph-mon[32460]: pgmap v581: 97 pgs: 97 active+clean; 10 GiB data, 296 GiB used, 777 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 776 MiB/s wr, 1.37k op/s 2024-02-18T11:48:38.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:48:38 smithi188 ceph-mon[36925]: pgmap v582: 97 pgs: 97 active+clean; 10 GiB data, 299 GiB used, 774 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 897 MiB/s wr, 1.47k op/s 2024-02-18T11:48:38.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:48:38 smithi012 ceph-mon[32460]: pgmap v582: 97 pgs: 97 active+clean; 10 GiB data, 299 GiB used, 774 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 897 MiB/s wr, 1.47k op/s 2024-02-18T11:48:38.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:48:38 smithi158 ceph-mon[36626]: pgmap v582: 97 pgs: 97 active+clean; 10 GiB data, 299 GiB used, 774 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 897 MiB/s wr, 1.47k op/s 2024-02-18T11:48:40.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:48:40 smithi012 ceph-mon[32460]: pgmap v583: 97 pgs: 97 active+clean; 10 GiB data, 302 GiB used, 771 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 802 MiB/s wr, 1.35k op/s 2024-02-18T11:48:40.993 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:48:40 smithi158 ceph-mon[36626]: pgmap v583: 97 pgs: 97 active+clean; 10 GiB data, 302 GiB used, 771 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 802 MiB/s wr, 1.35k op/s 2024-02-18T11:48:41.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:48:40 smithi188 ceph-mon[36925]: pgmap v583: 97 pgs: 97 active+clean; 10 GiB data, 302 GiB used, 771 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 802 MiB/s wr, 1.35k op/s 2024-02-18T11:48:42.938 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:48:42 smithi012 ceph-mon[32460]: pgmap v584: 97 pgs: 97 active+clean; 10 GiB data, 305 GiB used, 768 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 915 MiB/s wr, 1.55k op/s 2024-02-18T11:48:42.993 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:48:42 smithi158 ceph-mon[36626]: pgmap v584: 97 pgs: 97 active+clean; 10 GiB data, 305 GiB used, 768 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 915 MiB/s wr, 1.55k op/s 2024-02-18T11:48:43.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:48:42 smithi188 ceph-mon[36925]: pgmap v584: 97 pgs: 97 active+clean; 10 GiB data, 305 GiB used, 768 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 915 MiB/s wr, 1.55k op/s 2024-02-18T11:48:45.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:48:45 smithi188 ceph-mon[36925]: pgmap v585: 97 pgs: 97 active+clean; 10 GiB data, 310 GiB used, 763 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 883 MiB/s wr, 1.30k op/s 2024-02-18T11:48:45.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:48:45 smithi012 ceph-mon[32460]: pgmap v585: 97 pgs: 97 active+clean; 10 GiB data, 310 GiB used, 763 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 883 MiB/s wr, 1.30k op/s 2024-02-18T11:48:45.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:48:45 smithi158 ceph-mon[36626]: pgmap v585: 97 pgs: 97 active+clean; 10 GiB data, 310 GiB used, 763 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 883 MiB/s wr, 1.30k op/s 2024-02-18T11:48:47.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:48:47 smithi158 ceph-mon[36626]: pgmap v586: 97 pgs: 97 active+clean; 10 GiB data, 312 GiB used, 761 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 760 MiB/s wr, 963 op/s 2024-02-18T11:48:47.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:48:47 smithi188 ceph-mon[36925]: pgmap v586: 97 pgs: 97 active+clean; 10 GiB data, 312 GiB used, 761 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 760 MiB/s wr, 963 op/s 2024-02-18T11:48:47.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:48:47 smithi012 ceph-mon[32460]: pgmap v586: 97 pgs: 97 active+clean; 10 GiB data, 312 GiB used, 761 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 760 MiB/s wr, 963 op/s 2024-02-18T11:48:48.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:48:48 smithi188 ceph-mon[36925]: pgmap v587: 97 pgs: 97 active+clean; 10 GiB data, 315 GiB used, 757 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 927 MiB/s wr, 1.22k op/s 2024-02-18T11:48:48.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:48:48 smithi012 ceph-mon[32460]: pgmap v587: 97 pgs: 97 active+clean; 10 GiB data, 315 GiB used, 757 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 927 MiB/s wr, 1.22k op/s 2024-02-18T11:48:48.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:48:48 smithi158 ceph-mon[36626]: pgmap v587: 97 pgs: 97 active+clean; 10 GiB data, 315 GiB used, 757 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 927 MiB/s wr, 1.22k op/s 2024-02-18T11:48:50.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:48:50 smithi158 ceph-mon[36626]: pgmap v588: 97 pgs: 97 active+clean; 10 GiB data, 318 GiB used, 754 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 797 MiB/s wr, 1.14k op/s 2024-02-18T11:48:50.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:48:50 smithi188 ceph-mon[36925]: pgmap v588: 97 pgs: 97 active+clean; 10 GiB data, 318 GiB used, 754 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 797 MiB/s wr, 1.14k op/s 2024-02-18T11:48:50.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:48:50 smithi012 ceph-mon[32460]: pgmap v588: 97 pgs: 97 active+clean; 10 GiB data, 318 GiB used, 754 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 797 MiB/s wr, 1.14k op/s 2024-02-18T11:48:52.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:48:52 smithi012 ceph-mon[32460]: pgmap v589: 97 pgs: 97 active+clean; 10 GiB data, 306 GiB used, 767 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 896 MiB/s wr, 1.28k op/s 2024-02-18T11:48:52.993 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:48:52 smithi158 ceph-mon[36626]: pgmap v589: 97 pgs: 97 active+clean; 10 GiB data, 306 GiB used, 767 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 896 MiB/s wr, 1.28k op/s 2024-02-18T11:48:53.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:48:52 smithi188 ceph-mon[36925]: pgmap v589: 97 pgs: 97 active+clean; 10 GiB data, 306 GiB used, 767 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 896 MiB/s wr, 1.28k op/s 2024-02-18T11:48:55.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:48:54 smithi012 ceph-mon[32460]: pgmap v590: 97 pgs: 97 active+clean; 10 GiB data, 279 GiB used, 794 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 869 MiB/s wr, 1.19k op/s 2024-02-18T11:48:55.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:48:54 smithi158 ceph-mon[36626]: pgmap v590: 97 pgs: 97 active+clean; 10 GiB data, 279 GiB used, 794 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 869 MiB/s wr, 1.19k op/s 2024-02-18T11:48:55.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:48:54 smithi188 ceph-mon[36925]: pgmap v590: 97 pgs: 97 active+clean; 10 GiB data, 279 GiB used, 794 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 869 MiB/s wr, 1.19k op/s 2024-02-18T11:48:57.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:48:56 smithi158 ceph-mon[36626]: pgmap v591: 97 pgs: 97 active+clean; 10 GiB data, 281 GiB used, 792 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 754 MiB/s wr, 1.02k op/s 2024-02-18T11:48:57.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:48:56 smithi188 ceph-mon[36925]: pgmap v591: 97 pgs: 97 active+clean; 10 GiB data, 281 GiB used, 792 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 754 MiB/s wr, 1.02k op/s 2024-02-18T11:48:57.423 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:48:56 smithi012 ceph-mon[32460]: pgmap v591: 97 pgs: 97 active+clean; 10 GiB data, 281 GiB used, 792 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 754 MiB/s wr, 1.02k op/s 2024-02-18T11:48:59.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:48:59 smithi158 ceph-mon[36626]: pgmap v592: 97 pgs: 97 active+clean; 10 GiB data, 263 GiB used, 810 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 943 MiB/s wr, 1.28k op/s 2024-02-18T11:48:59.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:48:59 smithi012 ceph-mon[32460]: pgmap v592: 97 pgs: 97 active+clean; 10 GiB data, 263 GiB used, 810 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 943 MiB/s wr, 1.28k op/s 2024-02-18T11:48:59.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:48:59 smithi188 ceph-mon[36925]: pgmap v592: 97 pgs: 97 active+clean; 10 GiB data, 263 GiB used, 810 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 943 MiB/s wr, 1.28k op/s 2024-02-18T11:49:01.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:49:01 smithi158 ceph-mon[36626]: pgmap v593: 97 pgs: 97 active+clean; 10 GiB data, 266 GiB used, 807 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 811 MiB/s wr, 1.12k op/s 2024-02-18T11:49:01.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:49:01 smithi188 ceph-mon[36925]: pgmap v593: 97 pgs: 97 active+clean; 10 GiB data, 266 GiB used, 807 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 811 MiB/s wr, 1.12k op/s 2024-02-18T11:49:01.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:49:01 smithi012 ceph-mon[32460]: pgmap v593: 97 pgs: 97 active+clean; 10 GiB data, 266 GiB used, 807 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 811 MiB/s wr, 1.12k op/s 2024-02-18T11:49:02.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:49:02 smithi188 ceph-mon[36925]: pgmap v594: 97 pgs: 97 active+clean; 10 GiB data, 269 GiB used, 804 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 905 MiB/s wr, 1.28k op/s 2024-02-18T11:49:02.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:49:02 smithi012 ceph-mon[32460]: pgmap v594: 97 pgs: 97 active+clean; 10 GiB data, 269 GiB used, 804 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 905 MiB/s wr, 1.28k op/s 2024-02-18T11:49:02.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:49:02 smithi158 ceph-mon[36626]: pgmap v594: 97 pgs: 97 active+clean; 10 GiB data, 269 GiB used, 804 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 905 MiB/s wr, 1.28k op/s 2024-02-18T11:49:04.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:49:04 smithi158 ceph-mon[36626]: pgmap v595: 97 pgs: 97 active+clean; 10 GiB data, 274 GiB used, 798 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 890 MiB/s wr, 1.28k op/s 2024-02-18T11:49:04.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:49:04 smithi188 ceph-mon[36925]: pgmap v595: 97 pgs: 97 active+clean; 10 GiB data, 274 GiB used, 798 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 890 MiB/s wr, 1.28k op/s 2024-02-18T11:49:04.930 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:49:04 smithi012 ceph-mon[32460]: pgmap v595: 97 pgs: 97 active+clean; 10 GiB data, 274 GiB used, 798 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 890 MiB/s wr, 1.28k op/s 2024-02-18T11:49:06.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:49:06 smithi012 ceph-mon[32460]: pgmap v596: 97 pgs: 97 active+clean; 10 GiB data, 275 GiB used, 797 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 741 MiB/s wr, 1.11k op/s 2024-02-18T11:49:06.993 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:49:06 smithi158 ceph-mon[36626]: pgmap v596: 97 pgs: 97 active+clean; 10 GiB data, 275 GiB used, 797 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 741 MiB/s wr, 1.11k op/s 2024-02-18T11:49:07.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:49:06 smithi188 ceph-mon[36925]: pgmap v596: 97 pgs: 97 active+clean; 10 GiB data, 275 GiB used, 797 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 741 MiB/s wr, 1.11k op/s 2024-02-18T11:49:09.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:49:08 smithi188 ceph-mon[36925]: pgmap v597: 97 pgs: 97 active+clean; 10 GiB data, 280 GiB used, 793 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 963 MiB/s wr, 1.63k op/s 2024-02-18T11:49:09.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:49:08 smithi012 ceph-mon[32460]: pgmap v597: 97 pgs: 97 active+clean; 10 GiB data, 280 GiB used, 793 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 963 MiB/s wr, 1.63k op/s 2024-02-18T11:49:09.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:49:08 smithi158 ceph-mon[36626]: pgmap v597: 97 pgs: 97 active+clean; 10 GiB data, 280 GiB used, 793 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 963 MiB/s wr, 1.63k op/s 2024-02-18T11:49:11.172 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:49:10 smithi012 ceph-mon[32460]: pgmap v598: 97 pgs: 97 active+clean; 10 GiB data, 283 GiB used, 790 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 810 MiB/s wr, 1.55k op/s 2024-02-18T11:49:11.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:49:10 smithi158 ceph-mon[36626]: pgmap v598: 97 pgs: 97 active+clean; 10 GiB data, 283 GiB used, 790 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 810 MiB/s wr, 1.55k op/s 2024-02-18T11:49:11.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:49:10 smithi188 ceph-mon[36925]: pgmap v598: 97 pgs: 97 active+clean; 10 GiB data, 283 GiB used, 790 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 810 MiB/s wr, 1.55k op/s 2024-02-18T11:49:13.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:49:13 smithi012 ceph-mon[32460]: pgmap v599: 97 pgs: 97 active+clean; 10 GiB data, 264 GiB used, 809 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 924 MiB/s wr, 1.93k op/s 2024-02-18T11:49:13.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:49:13 smithi158 ceph-mon[36626]: pgmap v599: 97 pgs: 97 active+clean; 10 GiB data, 264 GiB used, 809 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 924 MiB/s wr, 1.93k op/s 2024-02-18T11:49:13.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:49:13 smithi188 ceph-mon[36925]: pgmap v599: 97 pgs: 97 active+clean; 10 GiB data, 264 GiB used, 809 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 924 MiB/s wr, 1.93k op/s 2024-02-18T11:49:15.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:49:15 smithi188 ceph-mon[36925]: pgmap v600: 97 pgs: 97 active+clean; 10 GiB data, 269 GiB used, 804 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 929 MiB/s wr, 1.92k op/s 2024-02-18T11:49:15.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:49:15 smithi012 ceph-mon[32460]: pgmap v600: 97 pgs: 97 active+clean; 10 GiB data, 269 GiB used, 804 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 929 MiB/s wr, 1.92k op/s 2024-02-18T11:49:15.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:49:15 smithi158 ceph-mon[36626]: pgmap v600: 97 pgs: 97 active+clean; 10 GiB data, 269 GiB used, 804 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 929 MiB/s wr, 1.92k op/s 2024-02-18T11:49:16.993 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:49:16 smithi158 ceph-mon[36626]: pgmap v601: 97 pgs: 97 active+clean; 10 GiB data, 270 GiB used, 802 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 778 MiB/s wr, 1.70k op/s 2024-02-18T11:49:17.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:49:16 smithi188 ceph-mon[36925]: pgmap v601: 97 pgs: 97 active+clean; 10 GiB data, 270 GiB used, 802 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 778 MiB/s wr, 1.70k op/s 2024-02-18T11:49:17.172 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:49:16 smithi012 ceph-mon[32460]: pgmap v601: 97 pgs: 97 active+clean; 10 GiB data, 270 GiB used, 802 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 778 MiB/s wr, 1.70k op/s 2024-02-18T11:49:19.172 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:49:18 smithi012 ceph-mon[32460]: pgmap v602: 97 pgs: 97 active+clean; 10 GiB data, 274 GiB used, 799 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 975 MiB/s wr, 2.24k op/s 2024-02-18T11:49:19.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:49:18 smithi158 ceph-mon[36626]: pgmap v602: 97 pgs: 97 active+clean; 10 GiB data, 274 GiB used, 799 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 975 MiB/s wr, 2.24k op/s 2024-02-18T11:49:19.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:49:18 smithi188 ceph-mon[36925]: pgmap v602: 97 pgs: 97 active+clean; 10 GiB data, 274 GiB used, 799 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 975 MiB/s wr, 2.24k op/s 2024-02-18T11:49:21.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:49:21 smithi158 ceph-mon[36626]: pgmap v603: 97 pgs: 97 active+clean; 10 GiB data, 279 GiB used, 794 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 833 MiB/s wr, 1.95k op/s 2024-02-18T11:49:21.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:49:21 smithi188 ceph-mon[36925]: pgmap v603: 97 pgs: 97 active+clean; 10 GiB data, 279 GiB used, 794 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 833 MiB/s wr, 1.95k op/s 2024-02-18T11:49:21.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:49:21 smithi012 ceph-mon[32460]: pgmap v603: 97 pgs: 97 active+clean; 10 GiB data, 279 GiB used, 794 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 833 MiB/s wr, 1.95k op/s 2024-02-18T11:49:22.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:49:22 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:49:22.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:49:22 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:49:22.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:49:22 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:49:23.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:49:23 smithi188 ceph-mon[36925]: pgmap v604: 97 pgs: 97 active+clean; 10 GiB data, 280 GiB used, 793 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 910 MiB/s wr, 1.88k op/s 2024-02-18T11:49:23.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:49:23 smithi012 ceph-mon[32460]: pgmap v604: 97 pgs: 97 active+clean; 10 GiB data, 280 GiB used, 793 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 910 MiB/s wr, 1.88k op/s 2024-02-18T11:49:23.744 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:49:23 smithi158 ceph-mon[36626]: pgmap v604: 97 pgs: 97 active+clean; 10 GiB data, 280 GiB used, 793 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 910 MiB/s wr, 1.88k op/s 2024-02-18T11:49:24.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:49:24 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:49:24.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:49:24 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:49:24.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:49:24 smithi012 ceph-mon[32460]: pgmap v605: 97 pgs: 97 active+clean; 10 GiB data, 285 GiB used, 788 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 880 MiB/s wr, 1.73k op/s 2024-02-18T11:49:24.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:49:24 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:49:24.993 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:49:24 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:49:24.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:49:24 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:49:24.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:49:24 smithi158 ceph-mon[36626]: pgmap v605: 97 pgs: 97 active+clean; 10 GiB data, 285 GiB used, 788 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 880 MiB/s wr, 1.73k op/s 2024-02-18T11:49:24.994 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:49:24 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:49:25.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:49:24 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:49:25.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:49:24 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:49:25.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:49:24 smithi188 ceph-mon[36925]: pgmap v605: 97 pgs: 97 active+clean; 10 GiB data, 285 GiB used, 788 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 880 MiB/s wr, 1.73k op/s 2024-02-18T11:49:25.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:49:24 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:49:27.423 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:49:27 smithi012 ceph-mon[32460]: pgmap v606: 97 pgs: 97 active+clean; 10 GiB data, 286 GiB used, 787 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 718 MiB/s wr, 1.50k op/s 2024-02-18T11:49:27.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:49:27 smithi158 ceph-mon[36626]: pgmap v606: 97 pgs: 97 active+clean; 10 GiB data, 286 GiB used, 787 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 718 MiB/s wr, 1.50k op/s 2024-02-18T11:49:27.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:49:27 smithi188 ceph-mon[36925]: pgmap v606: 97 pgs: 97 active+clean; 10 GiB data, 286 GiB used, 787 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 718 MiB/s wr, 1.50k op/s 2024-02-18T11:49:29.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:49:29 smithi158 ceph-mon[36626]: pgmap v607: 97 pgs: 97 active+clean; 10 GiB data, 289 GiB used, 784 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 897 MiB/s wr, 2.06k op/s 2024-02-18T11:49:29.602 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:49:29 smithi188 ceph-mon[36925]: pgmap v607: 97 pgs: 97 active+clean; 10 GiB data, 289 GiB used, 784 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 897 MiB/s wr, 2.06k op/s 2024-02-18T11:49:29.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:49:29 smithi012 ceph-mon[32460]: pgmap v607: 97 pgs: 97 active+clean; 10 GiB data, 289 GiB used, 784 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 897 MiB/s wr, 2.06k op/s 2024-02-18T11:49:30.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:49:30 smithi158 ceph-mon[36626]: pgmap v608: 97 pgs: 97 active+clean; 10 GiB data, 294 GiB used, 779 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 808 MiB/s wr, 1.88k op/s 2024-02-18T11:49:30.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:49:30 smithi188 ceph-mon[36925]: pgmap v608: 97 pgs: 97 active+clean; 10 GiB data, 294 GiB used, 779 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 808 MiB/s wr, 1.88k op/s 2024-02-18T11:49:30.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:49:30 smithi012 ceph-mon[32460]: pgmap v608: 97 pgs: 97 active+clean; 10 GiB data, 294 GiB used, 779 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 808 MiB/s wr, 1.88k op/s 2024-02-18T11:49:33.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:49:32 smithi188 ceph-mon[36925]: pgmap v609: 97 pgs: 97 active+clean; 10 GiB data, 296 GiB used, 777 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 834 MiB/s wr, 1.89k op/s 2024-02-18T11:49:33.172 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:49:32 smithi012 ceph-mon[32460]: pgmap v609: 97 pgs: 97 active+clean; 10 GiB data, 296 GiB used, 777 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 834 MiB/s wr, 1.89k op/s 2024-02-18T11:49:33.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:49:32 smithi158 ceph-mon[36626]: pgmap v609: 97 pgs: 97 active+clean; 10 GiB data, 296 GiB used, 777 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 834 MiB/s wr, 1.89k op/s 2024-02-18T11:49:35.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:49:34 smithi188 ceph-mon[36925]: pgmap v610: 97 pgs: 97 active+clean; 10 GiB data, 271 GiB used, 802 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 804 MiB/s wr, 1.79k op/s 2024-02-18T11:49:35.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:49:34 smithi012 ceph-mon[32460]: pgmap v610: 97 pgs: 97 active+clean; 10 GiB data, 271 GiB used, 802 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 804 MiB/s wr, 1.79k op/s 2024-02-18T11:49:35.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:49:34 smithi158 ceph-mon[36626]: pgmap v610: 97 pgs: 97 active+clean; 10 GiB data, 271 GiB used, 802 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 804 MiB/s wr, 1.79k op/s 2024-02-18T11:49:37.172 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:49:36 smithi012 ceph-mon[32460]: pgmap v611: 97 pgs: 97 active+clean; 10 GiB data, 272 GiB used, 801 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 649 MiB/s wr, 1.43k op/s 2024-02-18T11:49:37.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:49:36 smithi158 ceph-mon[36626]: pgmap v611: 97 pgs: 97 active+clean; 10 GiB data, 272 GiB used, 801 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 649 MiB/s wr, 1.43k op/s 2024-02-18T11:49:37.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:49:36 smithi188 ceph-mon[36925]: pgmap v611: 97 pgs: 97 active+clean; 10 GiB data, 272 GiB used, 801 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 649 MiB/s wr, 1.43k op/s 2024-02-18T11:49:39.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:49:39 smithi188 ceph-mon[36925]: pgmap v612: 97 pgs: 97 active+clean; 10 GiB data, 253 GiB used, 820 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 834 MiB/s wr, 1.64k op/s 2024-02-18T11:49:39.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:49:39 smithi012 ceph-mon[32460]: pgmap v612: 97 pgs: 97 active+clean; 10 GiB data, 253 GiB used, 820 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 834 MiB/s wr, 1.64k op/s 2024-02-18T11:49:39.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:49:39 smithi158 ceph-mon[36626]: pgmap v612: 97 pgs: 97 active+clean; 10 GiB data, 253 GiB used, 820 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 834 MiB/s wr, 1.64k op/s 2024-02-18T11:49:40.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:49:40 smithi012 ceph-mon[32460]: pgmap v613: 97 pgs: 97 active+clean; 10 GiB data, 258 GiB used, 814 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 789 MiB/s wr, 1.29k op/s 2024-02-18T11:49:40.993 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:49:40 smithi158 ceph-mon[36626]: pgmap v613: 97 pgs: 97 active+clean; 10 GiB data, 258 GiB used, 814 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 789 MiB/s wr, 1.29k op/s 2024-02-18T11:49:41.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:49:40 smithi188 ceph-mon[36925]: pgmap v613: 97 pgs: 97 active+clean; 10 GiB data, 258 GiB used, 814 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 789 MiB/s wr, 1.29k op/s 2024-02-18T11:49:43.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:49:42 smithi188 ceph-mon[36925]: pgmap v614: 97 pgs: 97 active+clean; 10 GiB data, 260 GiB used, 813 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 809 MiB/s wr, 1.32k op/s 2024-02-18T11:49:43.172 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:49:42 smithi012 ceph-mon[32460]: pgmap v614: 97 pgs: 97 active+clean; 10 GiB data, 260 GiB used, 813 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 809 MiB/s wr, 1.32k op/s 2024-02-18T11:49:43.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:49:42 smithi158 ceph-mon[36626]: pgmap v614: 97 pgs: 97 active+clean; 10 GiB data, 260 GiB used, 813 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 809 MiB/s wr, 1.32k op/s 2024-02-18T11:49:45.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:49:44 smithi158 ceph-mon[36626]: pgmap v615: 97 pgs: 97 active+clean; 10 GiB data, 264 GiB used, 809 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 762 MiB/s wr, 1.28k op/s 2024-02-18T11:49:45.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:49:44 smithi188 ceph-mon[36925]: pgmap v615: 97 pgs: 97 active+clean; 10 GiB data, 264 GiB used, 809 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 762 MiB/s wr, 1.28k op/s 2024-02-18T11:49:45.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:49:44 smithi012 ceph-mon[32460]: pgmap v615: 97 pgs: 97 active+clean; 10 GiB data, 264 GiB used, 809 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 762 MiB/s wr, 1.28k op/s 2024-02-18T11:49:47.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:49:46 smithi158 ceph-mon[36626]: pgmap v616: 97 pgs: 97 active+clean; 10 GiB data, 266 GiB used, 807 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 684 MiB/s wr, 1.28k op/s 2024-02-18T11:49:47.352 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:49:46 smithi188 ceph-mon[36925]: pgmap v616: 97 pgs: 97 active+clean; 10 GiB data, 266 GiB used, 807 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 684 MiB/s wr, 1.28k op/s 2024-02-18T11:49:47.425 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:49:46 smithi012 ceph-mon[32460]: pgmap v616: 97 pgs: 97 active+clean; 10 GiB data, 266 GiB used, 807 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 684 MiB/s wr, 1.28k op/s 2024-02-18T11:49:49.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:49:49 smithi158 ceph-mon[36626]: pgmap v617: 97 pgs: 97 active+clean; 10 GiB data, 243 GiB used, 830 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 884 MiB/s wr, 1.50k op/s 2024-02-18T11:49:49.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:49:49 smithi188 ceph-mon[36925]: pgmap v617: 97 pgs: 97 active+clean; 10 GiB data, 243 GiB used, 830 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 884 MiB/s wr, 1.50k op/s 2024-02-18T11:49:49.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:49:49 smithi012 ceph-mon[32460]: pgmap v617: 97 pgs: 97 active+clean; 10 GiB data, 243 GiB used, 830 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 884 MiB/s wr, 1.50k op/s 2024-02-18T11:49:50.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:49:50 smithi188 ceph-mon[36925]: pgmap v618: 97 pgs: 97 active+clean; 10 GiB data, 248 GiB used, 825 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 833 MiB/s wr, 1.61k op/s 2024-02-18T11:49:50.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:49:50 smithi012 ceph-mon[32460]: pgmap v618: 97 pgs: 97 active+clean; 10 GiB data, 248 GiB used, 825 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 833 MiB/s wr, 1.61k op/s 2024-02-18T11:49:50.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:49:50 smithi158 ceph-mon[36626]: pgmap v618: 97 pgs: 97 active+clean; 10 GiB data, 248 GiB used, 825 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 833 MiB/s wr, 1.61k op/s 2024-02-18T11:49:53.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:49:52 smithi188 ceph-mon[36925]: pgmap v619: 97 pgs: 97 active+clean; 10 GiB data, 250 GiB used, 823 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 854 MiB/s wr, 1.80k op/s 2024-02-18T11:49:53.172 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:49:52 smithi012 ceph-mon[32460]: pgmap v619: 97 pgs: 97 active+clean; 10 GiB data, 250 GiB used, 823 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 854 MiB/s wr, 1.80k op/s 2024-02-18T11:49:53.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:49:52 smithi158 ceph-mon[36626]: pgmap v619: 97 pgs: 97 active+clean; 10 GiB data, 250 GiB used, 823 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 854 MiB/s wr, 1.80k op/s 2024-02-18T11:49:55.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:49:54 smithi158 ceph-mon[36626]: pgmap v620: 97 pgs: 97 active+clean; 10 GiB data, 254 GiB used, 819 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 821 MiB/s wr, 1.53k op/s 2024-02-18T11:49:55.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:49:54 smithi188 ceph-mon[36925]: pgmap v620: 97 pgs: 97 active+clean; 10 GiB data, 254 GiB used, 819 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 821 MiB/s wr, 1.53k op/s 2024-02-18T11:49:55.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:49:54 smithi012 ceph-mon[32460]: pgmap v620: 97 pgs: 97 active+clean; 10 GiB data, 254 GiB used, 819 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 821 MiB/s wr, 1.53k op/s 2024-02-18T11:49:57.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:49:57 smithi188 ceph-mon[36925]: pgmap v621: 97 pgs: 97 active+clean; 10 GiB data, 256 GiB used, 817 GiB / 1.0 TiB avail; 1.2 MiB/s rd, 760 MiB/s wr, 1.32k op/s 2024-02-18T11:49:57.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:49:57 smithi012 ceph-mon[32460]: pgmap v621: 97 pgs: 97 active+clean; 10 GiB data, 256 GiB used, 817 GiB / 1.0 TiB avail; 1.2 MiB/s rd, 760 MiB/s wr, 1.32k op/s 2024-02-18T11:49:57.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:49:57 smithi158 ceph-mon[36626]: pgmap v621: 97 pgs: 97 active+clean; 10 GiB data, 256 GiB used, 817 GiB / 1.0 TiB avail; 1.2 MiB/s rd, 760 MiB/s wr, 1.32k op/s 2024-02-18T11:49:59.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:49:59 smithi012 ceph-mon[32460]: pgmap v622: 97 pgs: 97 active+clean; 10 GiB data, 259 GiB used, 814 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 907 MiB/s wr, 1.53k op/s 2024-02-18T11:49:59.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:49:59 smithi158 ceph-mon[36626]: pgmap v622: 97 pgs: 97 active+clean; 10 GiB data, 259 GiB used, 814 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 907 MiB/s wr, 1.53k op/s 2024-02-18T11:49:59.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:49:59 smithi188 ceph-mon[36925]: pgmap v622: 97 pgs: 97 active+clean; 10 GiB data, 259 GiB used, 814 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 907 MiB/s wr, 1.53k op/s 2024-02-18T11:50:00.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:50:00 smithi012 ceph-mon[32460]: pgmap v623: 97 pgs: 97 active+clean; 10 GiB data, 247 GiB used, 826 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 839 MiB/s wr, 1.63k op/s 2024-02-18T11:50:00.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:50:00 smithi012 ceph-mon[32460]: overall HEALTH_OK 2024-02-18T11:50:00.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:50:00 smithi158 ceph-mon[36626]: pgmap v623: 97 pgs: 97 active+clean; 10 GiB data, 247 GiB used, 826 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 839 MiB/s wr, 1.63k op/s 2024-02-18T11:50:00.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:50:00 smithi158 ceph-mon[36626]: overall HEALTH_OK 2024-02-18T11:50:00.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:50:00 smithi188 ceph-mon[36925]: pgmap v623: 97 pgs: 97 active+clean; 10 GiB data, 247 GiB used, 826 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 839 MiB/s wr, 1.63k op/s 2024-02-18T11:50:00.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:50:00 smithi188 ceph-mon[36925]: overall HEALTH_OK 2024-02-18T11:50:03.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:50:02 smithi158 ceph-mon[36626]: pgmap v624: 97 pgs: 97 active+clean; 10 GiB data, 249 GiB used, 824 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 860 MiB/s wr, 1.74k op/s 2024-02-18T11:50:03.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:50:02 smithi188 ceph-mon[36925]: pgmap v624: 97 pgs: 97 active+clean; 10 GiB data, 249 GiB used, 824 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 860 MiB/s wr, 1.74k op/s 2024-02-18T11:50:03.421 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:50:02 smithi012 ceph-mon[32460]: pgmap v624: 97 pgs: 97 active+clean; 10 GiB data, 249 GiB used, 824 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 860 MiB/s wr, 1.74k op/s 2024-02-18T11:50:05.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:50:05 smithi158 ceph-mon[36626]: pgmap v625: 97 pgs: 97 active+clean; 10 GiB data, 253 GiB used, 820 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 822 MiB/s wr, 1.83k op/s 2024-02-18T11:50:05.602 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:50:05 smithi188 ceph-mon[36925]: pgmap v625: 97 pgs: 97 active+clean; 10 GiB data, 253 GiB used, 820 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 822 MiB/s wr, 1.83k op/s 2024-02-18T11:50:05.671 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:50:05 smithi012 ceph-mon[32460]: pgmap v625: 97 pgs: 97 active+clean; 10 GiB data, 253 GiB used, 820 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 822 MiB/s wr, 1.83k op/s 2024-02-18T11:50:06.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:50:06 smithi012 ceph-mon[32460]: pgmap v626: 97 pgs: 97 active+clean; 10 GiB data, 255 GiB used, 818 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 763 MiB/s wr, 1.84k op/s 2024-02-18T11:50:06.744 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:50:06 smithi158 ceph-mon[36626]: pgmap v626: 97 pgs: 97 active+clean; 10 GiB data, 255 GiB used, 818 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 763 MiB/s wr, 1.84k op/s 2024-02-18T11:50:06.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:50:06 smithi188 ceph-mon[36925]: pgmap v626: 97 pgs: 97 active+clean; 10 GiB data, 255 GiB used, 818 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 763 MiB/s wr, 1.84k op/s 2024-02-18T11:50:08.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:50:08 smithi012 ceph-mon[32460]: pgmap v627: 97 pgs: 97 active+clean; 10 GiB data, 258 GiB used, 815 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 910 MiB/s wr, 2.18k op/s 2024-02-18T11:50:08.993 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:50:08 smithi158 ceph-mon[36626]: pgmap v627: 97 pgs: 97 active+clean; 10 GiB data, 258 GiB used, 815 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 910 MiB/s wr, 2.18k op/s 2024-02-18T11:50:09.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:50:08 smithi188 ceph-mon[36925]: pgmap v627: 97 pgs: 97 active+clean; 10 GiB data, 258 GiB used, 815 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 910 MiB/s wr, 2.18k op/s 2024-02-18T11:50:11.178 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:50:10 smithi012 ceph-mon[32460]: pgmap v628: 97 pgs: 97 active+clean; 10 GiB data, 212 GiB used, 861 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 842 MiB/s wr, 2.01k op/s 2024-02-18T11:50:11.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:50:10 smithi158 ceph-mon[36626]: pgmap v628: 97 pgs: 97 active+clean; 10 GiB data, 212 GiB used, 861 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 842 MiB/s wr, 2.01k op/s 2024-02-18T11:50:11.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:50:10 smithi188 ceph-mon[36925]: pgmap v628: 97 pgs: 97 active+clean; 10 GiB data, 212 GiB used, 861 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 842 MiB/s wr, 2.01k op/s 2024-02-18T11:50:13.172 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:50:12 smithi012 ceph-mon[32460]: pgmap v629: 97 pgs: 97 active+clean; 10 GiB data, 214 GiB used, 859 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 872 MiB/s wr, 1.97k op/s 2024-02-18T11:50:13.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:50:12 smithi158 ceph-mon[36626]: pgmap v629: 97 pgs: 97 active+clean; 10 GiB data, 214 GiB used, 859 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 872 MiB/s wr, 1.97k op/s 2024-02-18T11:50:13.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:50:12 smithi188 ceph-mon[36925]: pgmap v629: 97 pgs: 97 active+clean; 10 GiB data, 214 GiB used, 859 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 872 MiB/s wr, 1.97k op/s 2024-02-18T11:50:15.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:50:15 smithi188 ceph-mon[36925]: pgmap v630: 97 pgs: 97 active+clean; 10 GiB data, 219 GiB used, 854 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 850 MiB/s wr, 1.77k op/s 2024-02-18T11:50:15.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:50:15 smithi012 ceph-mon[32460]: pgmap v630: 97 pgs: 97 active+clean; 10 GiB data, 219 GiB used, 854 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 850 MiB/s wr, 1.77k op/s 2024-02-18T11:50:15.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:50:15 smithi158 ceph-mon[36626]: pgmap v630: 97 pgs: 97 active+clean; 10 GiB data, 219 GiB used, 854 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 850 MiB/s wr, 1.77k op/s 2024-02-18T11:50:17.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:50:17 smithi012 ceph-mon[32460]: pgmap v631: 97 pgs: 97 active+clean; 10 GiB data, 221 GiB used, 852 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 775 MiB/s wr, 1.41k op/s 2024-02-18T11:50:17.993 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:50:17 smithi158 ceph-mon[36626]: pgmap v631: 97 pgs: 97 active+clean; 10 GiB data, 221 GiB used, 852 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 775 MiB/s wr, 1.41k op/s 2024-02-18T11:50:18.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:50:17 smithi188 ceph-mon[36925]: pgmap v631: 97 pgs: 97 active+clean; 10 GiB data, 221 GiB used, 852 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 775 MiB/s wr, 1.41k op/s 2024-02-18T11:50:18.993 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:50:18 smithi158 ceph-mon[36626]: pgmap v632: 97 pgs: 97 active+clean; 10 GiB data, 223 GiB used, 850 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 897 MiB/s wr, 1.73k op/s 2024-02-18T11:50:19.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:50:18 smithi188 ceph-mon[36925]: pgmap v632: 97 pgs: 97 active+clean; 10 GiB data, 223 GiB used, 850 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 897 MiB/s wr, 1.73k op/s 2024-02-18T11:50:19.171 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:50:18 smithi012 ceph-mon[32460]: pgmap v632: 97 pgs: 97 active+clean; 10 GiB data, 223 GiB used, 850 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 897 MiB/s wr, 1.73k op/s 2024-02-18T11:50:21.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:50:20 smithi188 ceph-mon[36925]: pgmap v633: 97 pgs: 97 active+clean; 10 GiB data, 229 GiB used, 843 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 888 MiB/s wr, 1.76k op/s 2024-02-18T11:50:21.172 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:50:20 smithi012 ceph-mon[32460]: pgmap v633: 97 pgs: 97 active+clean; 10 GiB data, 229 GiB used, 843 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 888 MiB/s wr, 1.76k op/s 2024-02-18T11:50:21.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:50:20 smithi158 ceph-mon[36626]: pgmap v633: 97 pgs: 97 active+clean; 10 GiB data, 229 GiB used, 843 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 888 MiB/s wr, 1.76k op/s 2024-02-18T11:50:23.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:50:22 smithi012 ceph-mon[32460]: pgmap v634: 97 pgs: 97 active+clean; 10 GiB data, 223 GiB used, 850 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 922 MiB/s wr, 1.89k op/s 2024-02-18T11:50:23.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:50:22 smithi158 ceph-mon[36626]: pgmap v634: 97 pgs: 97 active+clean; 10 GiB data, 223 GiB used, 850 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 922 MiB/s wr, 1.89k op/s 2024-02-18T11:50:23.352 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:50:22 smithi188 ceph-mon[36925]: pgmap v634: 97 pgs: 97 active+clean; 10 GiB data, 223 GiB used, 850 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 922 MiB/s wr, 1.89k op/s 2024-02-18T11:50:24.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:50:24 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:50:24.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:50:24 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:50:24.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:50:24 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:50:25.352 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:50:25 smithi188 ceph-mon[36925]: pgmap v635: 97 pgs: 97 active+clean; 10 GiB data, 226 GiB used, 847 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 871 MiB/s wr, 1.86k op/s 2024-02-18T11:50:25.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:50:25 smithi012 ceph-mon[32460]: pgmap v635: 97 pgs: 97 active+clean; 10 GiB data, 226 GiB used, 847 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 871 MiB/s wr, 1.86k op/s 2024-02-18T11:50:25.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:50:25 smithi158 ceph-mon[36626]: pgmap v635: 97 pgs: 97 active+clean; 10 GiB data, 226 GiB used, 847 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 871 MiB/s wr, 1.86k op/s 2024-02-18T11:50:26.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:50:26 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:50:26.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:50:26 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:50:26.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:50:26 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:50:26.421 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:50:26 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:50:26.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:50:26 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:50:26.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:50:26 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:50:26.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:50:26 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:50:26.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:50:26 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:50:26.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:50:26 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:50:27.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:50:27 smithi188 ceph-mon[36925]: pgmap v636: 97 pgs: 97 active+clean; 10 GiB data, 229 GiB used, 844 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 793 MiB/s wr, 1.75k op/s 2024-02-18T11:50:27.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:50:27 smithi012 ceph-mon[32460]: pgmap v636: 97 pgs: 97 active+clean; 10 GiB data, 229 GiB used, 844 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 793 MiB/s wr, 1.75k op/s 2024-02-18T11:50:27.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:50:27 smithi158 ceph-mon[36626]: pgmap v636: 97 pgs: 97 active+clean; 10 GiB data, 229 GiB used, 844 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 793 MiB/s wr, 1.75k op/s 2024-02-18T11:50:29.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:50:29 smithi188 ceph-mon[36925]: pgmap v637: 97 pgs: 97 active+clean; 10 GiB data, 232 GiB used, 841 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 908 MiB/s wr, 2.16k op/s 2024-02-18T11:50:29.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:50:29 smithi012 ceph-mon[32460]: pgmap v637: 97 pgs: 97 active+clean; 10 GiB data, 232 GiB used, 841 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 908 MiB/s wr, 2.16k op/s 2024-02-18T11:50:29.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:50:29 smithi158 ceph-mon[36626]: pgmap v637: 97 pgs: 97 active+clean; 10 GiB data, 232 GiB used, 841 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 908 MiB/s wr, 2.16k op/s 2024-02-18T11:50:31.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:50:31 smithi012 ceph-mon[32460]: pgmap v638: 97 pgs: 97 active+clean; 10 GiB data, 216 GiB used, 857 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 885 MiB/s wr, 2.12k op/s 2024-02-18T11:50:31.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:50:31 smithi158 ceph-mon[36626]: pgmap v638: 97 pgs: 97 active+clean; 10 GiB data, 216 GiB used, 857 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 885 MiB/s wr, 2.12k op/s 2024-02-18T11:50:31.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:50:31 smithi188 ceph-mon[36925]: pgmap v638: 97 pgs: 97 active+clean; 10 GiB data, 216 GiB used, 857 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 885 MiB/s wr, 2.12k op/s 2024-02-18T11:50:32.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:50:32 smithi158 ceph-mon[36626]: pgmap v639: 97 pgs: 97 active+clean; 10 GiB data, 218 GiB used, 855 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 870 MiB/s wr, 1.93k op/s 2024-02-18T11:50:32.852 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:50:32 smithi188 ceph-mon[36925]: pgmap v639: 97 pgs: 97 active+clean; 10 GiB data, 218 GiB used, 855 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 870 MiB/s wr, 1.93k op/s 2024-02-18T11:50:32.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:50:32 smithi012 ceph-mon[32460]: pgmap v639: 97 pgs: 97 active+clean; 10 GiB data, 218 GiB used, 855 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 870 MiB/s wr, 1.93k op/s 2024-02-18T11:50:34.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:50:34 smithi188 ceph-mon[36925]: pgmap v640: 97 pgs: 97 active+clean; 10 GiB data, 222 GiB used, 851 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 823 MiB/s wr, 1.83k op/s 2024-02-18T11:50:34.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:50:34 smithi012 ceph-mon[32460]: pgmap v640: 97 pgs: 97 active+clean; 10 GiB data, 222 GiB used, 851 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 823 MiB/s wr, 1.83k op/s 2024-02-18T11:50:34.993 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:50:34 smithi158 ceph-mon[36626]: pgmap v640: 97 pgs: 97 active+clean; 10 GiB data, 222 GiB used, 851 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 823 MiB/s wr, 1.83k op/s 2024-02-18T11:50:37.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:50:37 smithi188 ceph-mon[36925]: pgmap v641: 97 pgs: 97 active+clean; 10 GiB data, 225 GiB used, 848 GiB / 1.0 TiB avail; 919 KiB/s rd, 778 MiB/s wr, 1.76k op/s 2024-02-18T11:50:37.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:50:37 smithi012 ceph-mon[32460]: pgmap v641: 97 pgs: 97 active+clean; 10 GiB data, 225 GiB used, 848 GiB / 1.0 TiB avail; 919 KiB/s rd, 778 MiB/s wr, 1.76k op/s 2024-02-18T11:50:37.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:50:37 smithi158 ceph-mon[36626]: pgmap v641: 97 pgs: 97 active+clean; 10 GiB data, 225 GiB used, 848 GiB / 1.0 TiB avail; 919 KiB/s rd, 778 MiB/s wr, 1.76k op/s 2024-02-18T11:50:39.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:50:39 smithi188 ceph-mon[36925]: pgmap v642: 97 pgs: 97 active+clean; 10 GiB data, 228 GiB used, 845 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 889 MiB/s wr, 2.06k op/s 2024-02-18T11:50:39.423 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:50:39 smithi012 ceph-mon[32460]: pgmap v642: 97 pgs: 97 active+clean; 10 GiB data, 228 GiB used, 845 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 889 MiB/s wr, 2.06k op/s 2024-02-18T11:50:39.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:50:39 smithi158 ceph-mon[36626]: pgmap v642: 97 pgs: 97 active+clean; 10 GiB data, 228 GiB used, 845 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 889 MiB/s wr, 2.06k op/s 2024-02-18T11:50:41.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:50:41 smithi158 ceph-mon[36626]: pgmap v643: 97 pgs: 97 active+clean; 10 GiB data, 232 GiB used, 840 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 856 MiB/s wr, 1.73k op/s 2024-02-18T11:50:41.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:50:41 smithi188 ceph-mon[36925]: pgmap v643: 97 pgs: 97 active+clean; 10 GiB data, 232 GiB used, 840 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 856 MiB/s wr, 1.73k op/s 2024-02-18T11:50:41.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:50:41 smithi012 ceph-mon[32460]: pgmap v643: 97 pgs: 97 active+clean; 10 GiB data, 232 GiB used, 840 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 856 MiB/s wr, 1.73k op/s 2024-02-18T11:50:43.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:50:42 smithi188 ceph-mon[36925]: pgmap v644: 97 pgs: 97 active+clean; 10 GiB data, 234 GiB used, 838 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 869 MiB/s wr, 1.54k op/s 2024-02-18T11:50:43.172 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:50:42 smithi012 ceph-mon[32460]: pgmap v644: 97 pgs: 97 active+clean; 10 GiB data, 234 GiB used, 838 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 869 MiB/s wr, 1.54k op/s 2024-02-18T11:50:43.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:50:42 smithi158 ceph-mon[36626]: pgmap v644: 97 pgs: 97 active+clean; 10 GiB data, 234 GiB used, 838 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 869 MiB/s wr, 1.54k op/s 2024-02-18T11:50:45.172 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:50:44 smithi012 ceph-mon[32460]: pgmap v645: 97 pgs: 97 active+clean; 10 GiB data, 220 GiB used, 853 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 811 MiB/s wr, 1.47k op/s 2024-02-18T11:50:45.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:50:44 smithi158 ceph-mon[36626]: pgmap v645: 97 pgs: 97 active+clean; 10 GiB data, 220 GiB used, 853 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 811 MiB/s wr, 1.47k op/s 2024-02-18T11:50:45.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:50:44 smithi188 ceph-mon[36925]: pgmap v645: 97 pgs: 97 active+clean; 10 GiB data, 220 GiB used, 853 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 811 MiB/s wr, 1.47k op/s 2024-02-18T11:50:47.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:50:47 smithi188 ceph-mon[36925]: pgmap v646: 97 pgs: 97 active+clean; 10 GiB data, 223 GiB used, 850 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 757 MiB/s wr, 1.47k op/s 2024-02-18T11:50:47.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:50:47 smithi012 ceph-mon[32460]: pgmap v646: 97 pgs: 97 active+clean; 10 GiB data, 223 GiB used, 850 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 757 MiB/s wr, 1.47k op/s 2024-02-18T11:50:47.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:50:47 smithi158 ceph-mon[36626]: pgmap v646: 97 pgs: 97 active+clean; 10 GiB data, 223 GiB used, 850 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 757 MiB/s wr, 1.47k op/s 2024-02-18T11:50:49.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:50:49 smithi158 ceph-mon[36626]: pgmap v647: 97 pgs: 97 active+clean; 10 GiB data, 205 GiB used, 868 GiB / 1.0 TiB avail; 2.5 MiB/s rd, 847 MiB/s wr, 1.77k op/s 2024-02-18T11:50:49.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:50:49 smithi188 ceph-mon[36925]: pgmap v647: 97 pgs: 97 active+clean; 10 GiB data, 205 GiB used, 868 GiB / 1.0 TiB avail; 2.5 MiB/s rd, 847 MiB/s wr, 1.77k op/s 2024-02-18T11:50:49.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:50:49 smithi012 ceph-mon[32460]: pgmap v647: 97 pgs: 97 active+clean; 10 GiB data, 205 GiB used, 868 GiB / 1.0 TiB avail; 2.5 MiB/s rd, 847 MiB/s wr, 1.77k op/s 2024-02-18T11:50:50.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:50:50 smithi188 ceph-mon[36925]: pgmap v648: 97 pgs: 97 active+clean; 10 GiB data, 209 GiB used, 864 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 778 MiB/s wr, 1.55k op/s 2024-02-18T11:50:50.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:50:50 smithi012 ceph-mon[32460]: pgmap v648: 97 pgs: 97 active+clean; 10 GiB data, 209 GiB used, 864 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 778 MiB/s wr, 1.55k op/s 2024-02-18T11:50:50.993 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:50:50 smithi158 ceph-mon[36626]: pgmap v648: 97 pgs: 97 active+clean; 10 GiB data, 209 GiB used, 864 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 778 MiB/s wr, 1.55k op/s 2024-02-18T11:50:53.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:50:53 smithi188 ceph-mon[36925]: pgmap v649: 97 pgs: 97 active+clean; 10 GiB data, 212 GiB used, 861 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 824 MiB/s wr, 1.73k op/s 2024-02-18T11:50:53.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:50:53 smithi012 ceph-mon[32460]: pgmap v649: 97 pgs: 97 active+clean; 10 GiB data, 212 GiB used, 861 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 824 MiB/s wr, 1.73k op/s 2024-02-18T11:50:53.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:50:53 smithi158 ceph-mon[36626]: pgmap v649: 97 pgs: 97 active+clean; 10 GiB data, 212 GiB used, 861 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 824 MiB/s wr, 1.73k op/s 2024-02-18T11:50:54.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:50:54 smithi188 ceph-mon[36925]: pgmap v650: 97 pgs: 97 active+clean; 10 GiB data, 215 GiB used, 858 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 781 MiB/s wr, 1.78k op/s 2024-02-18T11:50:54.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:50:54 smithi012 ceph-mon[32460]: pgmap v650: 97 pgs: 97 active+clean; 10 GiB data, 215 GiB used, 858 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 781 MiB/s wr, 1.78k op/s 2024-02-18T11:50:54.993 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:50:54 smithi158 ceph-mon[36626]: pgmap v650: 97 pgs: 97 active+clean; 10 GiB data, 215 GiB used, 858 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 781 MiB/s wr, 1.78k op/s 2024-02-18T11:50:57.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:50:57 smithi012 ceph-mon[32460]: pgmap v651: 97 pgs: 97 active+clean; 10 GiB data, 218 GiB used, 855 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 734 MiB/s wr, 1.77k op/s 2024-02-18T11:50:57.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:50:57 smithi158 ceph-mon[36626]: pgmap v651: 97 pgs: 97 active+clean; 10 GiB data, 218 GiB used, 855 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 734 MiB/s wr, 1.77k op/s 2024-02-18T11:50:57.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:50:57 smithi188 ceph-mon[36925]: pgmap v651: 97 pgs: 97 active+clean; 10 GiB data, 218 GiB used, 855 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 734 MiB/s wr, 1.77k op/s 2024-02-18T11:50:59.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:50:59 smithi188 ceph-mon[36925]: pgmap v652: 97 pgs: 97 active+clean; 10 GiB data, 220 GiB used, 852 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 850 MiB/s wr, 1.81k op/s 2024-02-18T11:50:59.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:50:59 smithi012 ceph-mon[32460]: pgmap v652: 97 pgs: 97 active+clean; 10 GiB data, 220 GiB used, 852 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 850 MiB/s wr, 1.81k op/s 2024-02-18T11:50:59.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:50:59 smithi158 ceph-mon[36626]: pgmap v652: 97 pgs: 97 active+clean; 10 GiB data, 220 GiB used, 852 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 850 MiB/s wr, 1.81k op/s 2024-02-18T11:51:00.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:51:00 smithi012 ceph-mon[32460]: pgmap v653: 97 pgs: 97 active+clean; 10 GiB data, 224 GiB used, 849 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 782 MiB/s wr, 1.48k op/s 2024-02-18T11:51:00.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:51:00 smithi158 ceph-mon[36626]: pgmap v653: 97 pgs: 97 active+clean; 10 GiB data, 224 GiB used, 849 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 782 MiB/s wr, 1.48k op/s 2024-02-18T11:51:00.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:51:00 smithi188 ceph-mon[36925]: pgmap v653: 97 pgs: 97 active+clean; 10 GiB data, 224 GiB used, 849 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 782 MiB/s wr, 1.48k op/s 2024-02-18T11:51:02.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:51:02 smithi012 ceph-mon[32460]: pgmap v654: 97 pgs: 97 active+clean; 10 GiB data, 226 GiB used, 847 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 827 MiB/s wr, 1.53k op/s 2024-02-18T11:51:02.993 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:51:02 smithi158 ceph-mon[36626]: pgmap v654: 97 pgs: 97 active+clean; 10 GiB data, 226 GiB used, 847 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 827 MiB/s wr, 1.53k op/s 2024-02-18T11:51:03.102 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:51:02 smithi188 ceph-mon[36925]: pgmap v654: 97 pgs: 97 active+clean; 10 GiB data, 226 GiB used, 847 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 827 MiB/s wr, 1.53k op/s 2024-02-18T11:51:04.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:51:04 smithi012 ceph-mon[32460]: pgmap v655: 97 pgs: 97 active+clean; 10 GiB data, 230 GiB used, 842 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 798 MiB/s wr, 1.40k op/s 2024-02-18T11:51:04.993 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:51:04 smithi158 ceph-mon[36626]: pgmap v655: 97 pgs: 97 active+clean; 10 GiB data, 230 GiB used, 842 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 798 MiB/s wr, 1.40k op/s 2024-02-18T11:51:05.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:51:04 smithi188 ceph-mon[36925]: pgmap v655: 97 pgs: 97 active+clean; 10 GiB data, 230 GiB used, 842 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 798 MiB/s wr, 1.40k op/s 2024-02-18T11:51:07.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:51:06 smithi188 ceph-mon[36925]: pgmap v656: 97 pgs: 97 active+clean; 10 GiB data, 198 GiB used, 875 GiB / 1.0 TiB avail; 1.2 MiB/s rd, 747 MiB/s wr, 1.21k op/s 2024-02-18T11:51:07.172 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:51:06 smithi012 ceph-mon[32460]: pgmap v656: 97 pgs: 97 active+clean; 10 GiB data, 198 GiB used, 875 GiB / 1.0 TiB avail; 1.2 MiB/s rd, 747 MiB/s wr, 1.21k op/s 2024-02-18T11:51:07.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:51:06 smithi158 ceph-mon[36626]: pgmap v656: 97 pgs: 97 active+clean; 10 GiB data, 198 GiB used, 875 GiB / 1.0 TiB avail; 1.2 MiB/s rd, 747 MiB/s wr, 1.21k op/s 2024-02-18T11:51:09.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:51:09 smithi188 ceph-mon[36925]: pgmap v657: 97 pgs: 97 active+clean; 10 GiB data, 194 GiB used, 879 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 882 MiB/s wr, 1.37k op/s 2024-02-18T11:51:09.429 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:51:09 smithi012 ceph-mon[32460]: pgmap v657: 97 pgs: 97 active+clean; 10 GiB data, 194 GiB used, 879 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 882 MiB/s wr, 1.37k op/s 2024-02-18T11:51:09.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:51:09 smithi158 ceph-mon[36626]: pgmap v657: 97 pgs: 97 active+clean; 10 GiB data, 194 GiB used, 879 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 882 MiB/s wr, 1.37k op/s 2024-02-18T11:51:11.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:51:11 smithi188 ceph-mon[36925]: pgmap v658: 97 pgs: 97 active+clean; 10 GiB data, 198 GiB used, 875 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 820 MiB/s wr, 1.36k op/s 2024-02-18T11:51:11.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:51:11 smithi012 ceph-mon[32460]: pgmap v658: 97 pgs: 97 active+clean; 10 GiB data, 198 GiB used, 875 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 820 MiB/s wr, 1.36k op/s 2024-02-18T11:51:11.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:51:11 smithi158 ceph-mon[36626]: pgmap v658: 97 pgs: 97 active+clean; 10 GiB data, 198 GiB used, 875 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 820 MiB/s wr, 1.36k op/s 2024-02-18T11:51:13.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:51:13 smithi188 ceph-mon[36925]: pgmap v659: 97 pgs: 97 active+clean; 10 GiB data, 200 GiB used, 873 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 870 MiB/s wr, 1.47k op/s 2024-02-18T11:51:13.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:51:13 smithi012 ceph-mon[32460]: pgmap v659: 97 pgs: 97 active+clean; 10 GiB data, 200 GiB used, 873 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 870 MiB/s wr, 1.47k op/s 2024-02-18T11:51:13.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:51:13 smithi158 ceph-mon[36626]: pgmap v659: 97 pgs: 97 active+clean; 10 GiB data, 200 GiB used, 873 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 870 MiB/s wr, 1.47k op/s 2024-02-18T11:51:14.921 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:51:14 smithi012 ceph-mon[32460]: pgmap v660: 97 pgs: 97 active+clean; 10 GiB data, 193 GiB used, 880 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 856 MiB/s wr, 1.43k op/s 2024-02-18T11:51:14.993 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:51:14 smithi158 ceph-mon[36626]: pgmap v660: 97 pgs: 97 active+clean; 10 GiB data, 193 GiB used, 880 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 856 MiB/s wr, 1.43k op/s 2024-02-18T11:51:15.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:51:14 smithi188 ceph-mon[36925]: pgmap v660: 97 pgs: 97 active+clean; 10 GiB data, 193 GiB used, 880 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 856 MiB/s wr, 1.43k op/s 2024-02-18T11:51:17.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:51:16 smithi188 ceph-mon[36925]: pgmap v661: 97 pgs: 97 active+clean; 10 GiB data, 195 GiB used, 877 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 770 MiB/s wr, 1.24k op/s 2024-02-18T11:51:17.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:51:16 smithi012 ceph-mon[32460]: pgmap v661: 97 pgs: 97 active+clean; 10 GiB data, 195 GiB used, 877 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 770 MiB/s wr, 1.24k op/s 2024-02-18T11:51:17.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:51:16 smithi158 ceph-mon[36626]: pgmap v661: 97 pgs: 97 active+clean; 10 GiB data, 195 GiB used, 877 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 770 MiB/s wr, 1.24k op/s 2024-02-18T11:51:19.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:51:19 smithi158 ceph-mon[36626]: pgmap v662: 97 pgs: 97 active+clean; 10 GiB data, 177 GiB used, 896 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 905 MiB/s wr, 1.46k op/s 2024-02-18T11:51:19.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:51:19 smithi188 ceph-mon[36925]: pgmap v662: 97 pgs: 97 active+clean; 10 GiB data, 177 GiB used, 896 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 905 MiB/s wr, 1.46k op/s 2024-02-18T11:51:19.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:51:19 smithi012 ceph-mon[32460]: pgmap v662: 97 pgs: 97 active+clean; 10 GiB data, 177 GiB used, 896 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 905 MiB/s wr, 1.46k op/s 2024-02-18T11:51:21.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:51:21 smithi188 ceph-mon[36925]: pgmap v663: 97 pgs: 97 active+clean; 10 GiB data, 181 GiB used, 892 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 816 MiB/s wr, 1.43k op/s 2024-02-18T11:51:21.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:51:21 smithi012 ceph-mon[32460]: pgmap v663: 97 pgs: 97 active+clean; 10 GiB data, 181 GiB used, 892 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 816 MiB/s wr, 1.43k op/s 2024-02-18T11:51:21.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:51:21 smithi158 ceph-mon[36626]: pgmap v663: 97 pgs: 97 active+clean; 10 GiB data, 181 GiB used, 892 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 816 MiB/s wr, 1.43k op/s 2024-02-18T11:51:22.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:51:22 smithi158 ceph-mon[36626]: pgmap v664: 97 pgs: 97 active+clean; 10 GiB data, 183 GiB used, 890 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 812 MiB/s wr, 1.57k op/s 2024-02-18T11:51:22.852 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:51:22 smithi188 ceph-mon[36925]: pgmap v664: 97 pgs: 97 active+clean; 10 GiB data, 183 GiB used, 890 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 812 MiB/s wr, 1.57k op/s 2024-02-18T11:51:22.926 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:51:22 smithi012 ceph-mon[32460]: pgmap v664: 97 pgs: 97 active+clean; 10 GiB data, 183 GiB used, 890 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 812 MiB/s wr, 1.57k op/s 2024-02-18T11:51:24.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:51:24 smithi012 ceph-mon[32460]: pgmap v665: 97 pgs: 97 active+clean; 10 GiB data, 172 GiB used, 901 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 840 MiB/s wr, 1.98k op/s 2024-02-18T11:51:24.993 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:51:24 smithi158 ceph-mon[36626]: pgmap v665: 97 pgs: 97 active+clean; 10 GiB data, 172 GiB used, 901 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 840 MiB/s wr, 1.98k op/s 2024-02-18T11:51:25.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:51:24 smithi188 ceph-mon[36925]: pgmap v665: 97 pgs: 97 active+clean; 10 GiB data, 172 GiB used, 901 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 840 MiB/s wr, 1.98k op/s 2024-02-18T11:51:27.172 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:51:26 smithi012 ceph-mon[32460]: pgmap v666: 97 pgs: 97 active+clean; 10 GiB data, 160 GiB used, 913 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 769 MiB/s wr, 1.98k op/s 2024-02-18T11:51:27.172 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:51:26 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:51:27.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:51:26 smithi158 ceph-mon[36626]: pgmap v666: 97 pgs: 97 active+clean; 10 GiB data, 160 GiB used, 913 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 769 MiB/s wr, 1.98k op/s 2024-02-18T11:51:27.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:51:26 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:51:27.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:51:26 smithi188 ceph-mon[36925]: pgmap v666: 97 pgs: 97 active+clean; 10 GiB data, 160 GiB used, 913 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 769 MiB/s wr, 1.98k op/s 2024-02-18T11:51:27.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:51:26 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:51:28.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:51:28 smithi158 ceph-mon[36626]: pgmap v667: 97 pgs: 97 active+clean; 10 GiB data, 163 GiB used, 909 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 888 MiB/s wr, 2.32k op/s 2024-02-18T11:51:28.852 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:51:28 smithi188 ceph-mon[36925]: pgmap v667: 97 pgs: 97 active+clean; 10 GiB data, 163 GiB used, 909 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 888 MiB/s wr, 2.32k op/s 2024-02-18T11:51:28.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:51:28 smithi012 ceph-mon[32460]: pgmap v667: 97 pgs: 97 active+clean; 10 GiB data, 163 GiB used, 909 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 888 MiB/s wr, 2.32k op/s 2024-02-18T11:51:30.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:51:30 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:51:30.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:51:30 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:51:30.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:51:30 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:51:30.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:51:30 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:51:30.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:51:30 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:51:30.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:51:30 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:51:30.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:51:30 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:51:30.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:51:30 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:51:30.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:51:30 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:51:30.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:51:30 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:51:30.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:51:30 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:51:30.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:51:30 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:51:31.679 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:51:31 smithi012 ceph-mon[32460]: pgmap v668: 97 pgs: 97 active+clean; 10 GiB data, 167 GiB used, 905 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 793 MiB/s wr, 2.21k op/s 2024-02-18T11:51:31.679 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:51:31 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:51:31.679 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:51:31 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:51:31.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:51:31 smithi158 ceph-mon[36626]: pgmap v668: 97 pgs: 97 active+clean; 10 GiB data, 167 GiB used, 905 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 793 MiB/s wr, 2.21k op/s 2024-02-18T11:51:31.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:51:31 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:51:31.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:51:31 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:51:31.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:51:31 smithi188 ceph-mon[36925]: pgmap v668: 97 pgs: 97 active+clean; 10 GiB data, 167 GiB used, 905 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 793 MiB/s wr, 2.21k op/s 2024-02-18T11:51:31.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:51:31 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:51:31.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:51:31 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:51:32.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:51:32 smithi012 ceph-mon[32460]: pgmap v669: 97 pgs: 97 active+clean; 10 GiB data, 169 GiB used, 903 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 801 MiB/s wr, 2.09k op/s 2024-02-18T11:51:32.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:51:32 smithi158 ceph-mon[36626]: pgmap v669: 97 pgs: 97 active+clean; 10 GiB data, 169 GiB used, 903 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 801 MiB/s wr, 2.09k op/s 2024-02-18T11:51:32.852 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:51:32 smithi188 ceph-mon[36925]: pgmap v669: 97 pgs: 97 active+clean; 10 GiB data, 169 GiB used, 903 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 801 MiB/s wr, 2.09k op/s 2024-02-18T11:51:33.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:51:33 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:51:33.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:51:33 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:51:33.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:51:33 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:51:33.852 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:51:33 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:51:33.852 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:51:33 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:51:33.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:51:33 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:51:33.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:51:33 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:51:33.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:51:33 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:51:33.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:51:33 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:51:34.993 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:51:34 smithi158 ceph-mon[36626]: pgmap v670: 97 pgs: 97 active+clean; 10 GiB data, 159 GiB used, 914 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 885 MiB/s wr, 2.13k op/s 2024-02-18T11:51:35.102 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:51:34 smithi188 ceph-mon[36925]: pgmap v670: 97 pgs: 97 active+clean; 10 GiB data, 159 GiB used, 914 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 885 MiB/s wr, 2.13k op/s 2024-02-18T11:51:35.176 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:51:34 smithi012 ceph-mon[32460]: pgmap v670: 97 pgs: 97 active+clean; 10 GiB data, 159 GiB used, 914 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 885 MiB/s wr, 2.13k op/s 2024-02-18T11:51:37.172 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:51:36 smithi012 ceph-mon[32460]: pgmap v671: 97 pgs: 97 active+clean; 10 GiB data, 162 GiB used, 911 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 786 MiB/s wr, 1.65k op/s 2024-02-18T11:51:37.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:51:36 smithi158 ceph-mon[36626]: pgmap v671: 97 pgs: 97 active+clean; 10 GiB data, 162 GiB used, 911 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 786 MiB/s wr, 1.65k op/s 2024-02-18T11:51:37.352 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:51:36 smithi188 ceph-mon[36925]: pgmap v671: 97 pgs: 97 active+clean; 10 GiB data, 162 GiB used, 911 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 786 MiB/s wr, 1.65k op/s 2024-02-18T11:51:39.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:51:39 smithi158 ceph-mon[36626]: pgmap v672: 97 pgs: 97 active+clean; 10 GiB data, 165 GiB used, 908 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 920 MiB/s wr, 1.84k op/s 2024-02-18T11:51:39.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:51:39 smithi188 ceph-mon[36925]: pgmap v672: 97 pgs: 97 active+clean; 10 GiB data, 165 GiB used, 908 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 920 MiB/s wr, 1.84k op/s 2024-02-18T11:51:39.671 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:51:39 smithi012 ceph-mon[32460]: pgmap v672: 97 pgs: 97 active+clean; 10 GiB data, 165 GiB used, 908 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 920 MiB/s wr, 1.84k op/s 2024-02-18T11:51:40.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:51:40 smithi012 ceph-mon[32460]: pgmap v673: 97 pgs: 97 active+clean; 10 GiB data, 169 GiB used, 903 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 848 MiB/s wr, 1.58k op/s 2024-02-18T11:51:40.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:51:40 smithi158 ceph-mon[36626]: pgmap v673: 97 pgs: 97 active+clean; 10 GiB data, 169 GiB used, 903 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 848 MiB/s wr, 1.58k op/s 2024-02-18T11:51:40.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:51:40 smithi188 ceph-mon[36925]: pgmap v673: 97 pgs: 97 active+clean; 10 GiB data, 169 GiB used, 903 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 848 MiB/s wr, 1.58k op/s 2024-02-18T11:51:42.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:51:42 smithi188 ceph-mon[36925]: pgmap v674: 97 pgs: 97 active+clean; 10 GiB data, 171 GiB used, 901 GiB / 1.0 TiB avail; 1.2 MiB/s rd, 843 MiB/s wr, 1.70k op/s 2024-02-18T11:51:42.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:51:42 smithi012 ceph-mon[32460]: pgmap v674: 97 pgs: 97 active+clean; 10 GiB data, 171 GiB used, 901 GiB / 1.0 TiB avail; 1.2 MiB/s rd, 843 MiB/s wr, 1.70k op/s 2024-02-18T11:51:42.993 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:51:42 smithi158 ceph-mon[36626]: pgmap v674: 97 pgs: 97 active+clean; 10 GiB data, 171 GiB used, 901 GiB / 1.0 TiB avail; 1.2 MiB/s rd, 843 MiB/s wr, 1.70k op/s 2024-02-18T11:51:44.826 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:51:44 smithi012 ceph-mon[32460]: pgmap v675: 97 pgs: 97 active+clean; 10 GiB data, 176 GiB used, 897 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 925 MiB/s wr, 2.22k op/s 2024-02-18T11:51:44.852 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:51:44 smithi188 ceph-mon[36925]: pgmap v675: 97 pgs: 97 active+clean; 10 GiB data, 176 GiB used, 897 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 925 MiB/s wr, 2.22k op/s 2024-02-18T11:51:44.993 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:51:44 smithi158 ceph-mon[36626]: pgmap v675: 97 pgs: 97 active+clean; 10 GiB data, 176 GiB used, 897 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 925 MiB/s wr, 2.22k op/s 2024-02-18T11:51:47.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:51:46 smithi188 ceph-mon[36925]: pgmap v676: 97 pgs: 97 active+clean; 10 GiB data, 178 GiB used, 894 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 797 MiB/s wr, 2.06k op/s 2024-02-18T11:51:47.172 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:51:46 smithi012 ceph-mon[32460]: pgmap v676: 97 pgs: 97 active+clean; 10 GiB data, 178 GiB used, 894 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 797 MiB/s wr, 2.06k op/s 2024-02-18T11:51:47.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:51:46 smithi158 ceph-mon[36626]: pgmap v676: 97 pgs: 97 active+clean; 10 GiB data, 178 GiB used, 894 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 797 MiB/s wr, 2.06k op/s 2024-02-18T11:51:49.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:51:49 smithi158 ceph-mon[36626]: pgmap v677: 97 pgs: 97 active+clean; 10 GiB data, 181 GiB used, 892 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 872 MiB/s wr, 2.46k op/s 2024-02-18T11:51:49.426 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:51:49 smithi012 ceph-mon[32460]: pgmap v677: 97 pgs: 97 active+clean; 10 GiB data, 181 GiB used, 892 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 872 MiB/s wr, 2.46k op/s 2024-02-18T11:51:49.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:51:49 smithi188 ceph-mon[36925]: pgmap v677: 97 pgs: 97 active+clean; 10 GiB data, 181 GiB used, 892 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 872 MiB/s wr, 2.46k op/s 2024-02-18T11:51:50.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:51:50 smithi012 ceph-mon[32460]: pgmap v678: 97 pgs: 97 active+clean; 10 GiB data, 186 GiB used, 887 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 821 MiB/s wr, 2.43k op/s 2024-02-18T11:51:50.993 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:51:50 smithi158 ceph-mon[36626]: pgmap v678: 97 pgs: 97 active+clean; 10 GiB data, 186 GiB used, 887 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 821 MiB/s wr, 2.43k op/s 2024-02-18T11:51:51.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:51:50 smithi188 ceph-mon[36925]: pgmap v678: 97 pgs: 97 active+clean; 10 GiB data, 186 GiB used, 887 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 821 MiB/s wr, 2.43k op/s 2024-02-18T11:51:53.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:51:52 smithi012 ceph-mon[32460]: pgmap v679: 97 pgs: 97 active+clean; 10 GiB data, 188 GiB used, 885 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 815 MiB/s wr, 2.36k op/s 2024-02-18T11:51:53.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:51:52 smithi158 ceph-mon[36626]: pgmap v679: 97 pgs: 97 active+clean; 10 GiB data, 188 GiB used, 885 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 815 MiB/s wr, 2.36k op/s 2024-02-18T11:51:53.352 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:51:52 smithi188 ceph-mon[36925]: pgmap v679: 97 pgs: 97 active+clean; 10 GiB data, 188 GiB used, 885 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 815 MiB/s wr, 2.36k op/s 2024-02-18T11:51:55.172 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:51:54 smithi012 ceph-mon[32460]: pgmap v680: 97 pgs: 97 active+clean; 10 GiB data, 192 GiB used, 881 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 900 MiB/s wr, 2.23k op/s 2024-02-18T11:51:55.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:51:54 smithi158 ceph-mon[36626]: pgmap v680: 97 pgs: 97 active+clean; 10 GiB data, 192 GiB used, 881 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 900 MiB/s wr, 2.23k op/s 2024-02-18T11:51:55.352 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:51:54 smithi188 ceph-mon[36925]: pgmap v680: 97 pgs: 97 active+clean; 10 GiB data, 192 GiB used, 881 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 900 MiB/s wr, 2.23k op/s 2024-02-18T11:51:57.352 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:51:57 smithi188 ceph-mon[36925]: pgmap v681: 97 pgs: 97 active+clean; 10 GiB data, 195 GiB used, 878 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 766 MiB/s wr, 1.64k op/s 2024-02-18T11:51:57.377 INFO:tasks.workunit.client.0.smithi012.stdout:stopping iogen 2024-02-18T11:51:57.378 INFO:tasks.workunit.client.0.smithi012.stderr:+ echo 'stopping iogen' 2024-02-18T11:51:57.378 INFO:tasks.workunit.client.0.smithi012.stderr:+ ./iogen -k 2024-02-18T11:51:57.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:51:57 smithi012 ceph-mon[32460]: pgmap v681: 97 pgs: 97 active+clean; 10 GiB data, 195 GiB used, 878 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 766 MiB/s wr, 1.64k op/s 2024-02-18T11:51:57.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:51:57 smithi158 ceph-mon[36626]: pgmap v681: 97 pgs: 97 active+clean; 10 GiB data, 195 GiB used, 878 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 766 MiB/s wr, 1.64k op/s 2024-02-18T11:51:59.602 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:51:59 smithi188 ceph-mon[36925]: pgmap v682: 97 pgs: 97 active+clean; 10 GiB data, 197 GiB used, 876 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 858 MiB/s wr, 1.66k op/s 2024-02-18T11:51:59.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:51:59 smithi012 ceph-mon[32460]: pgmap v682: 97 pgs: 97 active+clean; 10 GiB data, 197 GiB used, 876 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 858 MiB/s wr, 1.66k op/s 2024-02-18T11:51:59.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:51:59 smithi158 ceph-mon[36626]: pgmap v682: 97 pgs: 97 active+clean; 10 GiB data, 197 GiB used, 876 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 858 MiB/s wr, 1.66k op/s 2024-02-18T11:52:00.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:52:00 smithi158 ceph-mon[36626]: pgmap v683: 97 pgs: 97 active+clean; 10 GiB data, 196 GiB used, 877 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 872 MiB/s wr, 1.35k op/s 2024-02-18T11:52:00.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:52:00 smithi188 ceph-mon[36925]: pgmap v683: 97 pgs: 97 active+clean; 10 GiB data, 196 GiB used, 877 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 872 MiB/s wr, 1.35k op/s 2024-02-18T11:52:00.921 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:52:00 smithi012 ceph-mon[32460]: pgmap v683: 97 pgs: 97 active+clean; 10 GiB data, 196 GiB used, 877 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 872 MiB/s wr, 1.35k op/s 2024-02-18T11:52:03.172 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:52:02 smithi012 ceph-mon[32460]: pgmap v684: 97 pgs: 97 active+clean; 10 GiB data, 189 GiB used, 883 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 816 MiB/s wr, 1.07k op/s 2024-02-18T11:52:03.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:52:02 smithi158 ceph-mon[36626]: pgmap v684: 97 pgs: 97 active+clean; 10 GiB data, 189 GiB used, 883 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 816 MiB/s wr, 1.07k op/s 2024-02-18T11:52:03.352 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:52:02 smithi188 ceph-mon[36925]: pgmap v684: 97 pgs: 97 active+clean; 10 GiB data, 189 GiB used, 883 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 816 MiB/s wr, 1.07k op/s 2024-02-18T11:52:05.352 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:52:05 smithi188 ceph-mon[36925]: pgmap v685: 97 pgs: 97 active+clean; 10 GiB data, 193 GiB used, 880 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 873 MiB/s wr, 1.15k op/s 2024-02-18T11:52:05.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:52:05 smithi012 ceph-mon[32460]: pgmap v685: 97 pgs: 97 active+clean; 10 GiB data, 193 GiB used, 880 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 873 MiB/s wr, 1.15k op/s 2024-02-18T11:52:05.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:52:05 smithi158 ceph-mon[36626]: pgmap v685: 97 pgs: 97 active+clean; 10 GiB data, 193 GiB used, 880 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 873 MiB/s wr, 1.15k op/s 2024-02-18T11:52:07.352 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:52:07 smithi188 ceph-mon[36925]: pgmap v686: 97 pgs: 97 active+clean; 10 GiB data, 196 GiB used, 877 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 744 MiB/s wr, 1.05k op/s 2024-02-18T11:52:07.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:52:07 smithi012 ceph-mon[32460]: pgmap v686: 97 pgs: 97 active+clean; 10 GiB data, 196 GiB used, 877 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 744 MiB/s wr, 1.05k op/s 2024-02-18T11:52:07.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:52:07 smithi158 ceph-mon[36626]: pgmap v686: 97 pgs: 97 active+clean; 10 GiB data, 196 GiB used, 877 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 744 MiB/s wr, 1.05k op/s 2024-02-18T11:52:09.602 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:52:09 smithi188 ceph-mon[36925]: pgmap v687: 97 pgs: 97 active+clean; 10 GiB data, 199 GiB used, 874 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 846 MiB/s wr, 1.26k op/s 2024-02-18T11:52:09.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:52:09 smithi012 ceph-mon[32460]: pgmap v687: 97 pgs: 97 active+clean; 10 GiB data, 199 GiB used, 874 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 846 MiB/s wr, 1.26k op/s 2024-02-18T11:52:09.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:52:09 smithi158 ceph-mon[36626]: pgmap v687: 97 pgs: 97 active+clean; 10 GiB data, 199 GiB used, 874 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 846 MiB/s wr, 1.26k op/s 2024-02-18T11:52:10.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:52:10 smithi188 ceph-mon[36925]: pgmap v688: 97 pgs: 97 active+clean; 10 GiB data, 193 GiB used, 880 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 835 MiB/s wr, 1.44k op/s 2024-02-18T11:52:10.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:52:10 smithi012 ceph-mon[32460]: pgmap v688: 97 pgs: 97 active+clean; 10 GiB data, 193 GiB used, 880 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 835 MiB/s wr, 1.44k op/s 2024-02-18T11:52:10.993 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:52:10 smithi158 ceph-mon[36626]: pgmap v688: 97 pgs: 97 active+clean; 10 GiB data, 193 GiB used, 880 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 835 MiB/s wr, 1.44k op/s 2024-02-18T11:52:12.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:52:12 smithi012 ceph-mon[32460]: pgmap v689: 97 pgs: 97 active+clean; 10 GiB data, 196 GiB used, 877 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 775 MiB/s wr, 1.48k op/s 2024-02-18T11:52:12.992 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:52:12 smithi158 ceph-mon[36626]: pgmap v689: 97 pgs: 97 active+clean; 10 GiB data, 196 GiB used, 877 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 775 MiB/s wr, 1.48k op/s 2024-02-18T11:52:13.102 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:52:12 smithi188 ceph-mon[36925]: pgmap v689: 97 pgs: 97 active+clean; 10 GiB data, 196 GiB used, 877 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 775 MiB/s wr, 1.48k op/s 2024-02-18T11:52:15.172 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:52:14 smithi012 ceph-mon[32460]: pgmap v690: 97 pgs: 97 active+clean; 10 GiB data, 201 GiB used, 872 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 892 MiB/s wr, 1.66k op/s 2024-02-18T11:52:15.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:52:14 smithi158 ceph-mon[36626]: pgmap v690: 97 pgs: 97 active+clean; 10 GiB data, 201 GiB used, 872 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 892 MiB/s wr, 1.66k op/s 2024-02-18T11:52:15.352 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:52:14 smithi188 ceph-mon[36925]: pgmap v690: 97 pgs: 97 active+clean; 10 GiB data, 201 GiB used, 872 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 892 MiB/s wr, 1.66k op/s 2024-02-18T11:52:17.427 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:52:17 smithi012 ceph-mon[32460]: pgmap v691: 97 pgs: 97 active+clean; 10 GiB data, 202 GiB used, 871 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 741 MiB/s wr, 1.48k op/s 2024-02-18T11:52:17.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:52:17 smithi158 ceph-mon[36626]: pgmap v691: 97 pgs: 97 active+clean; 10 GiB data, 202 GiB used, 871 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 741 MiB/s wr, 1.48k op/s 2024-02-18T11:52:17.602 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:52:17 smithi188 ceph-mon[36925]: pgmap v691: 97 pgs: 97 active+clean; 10 GiB data, 202 GiB used, 871 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 741 MiB/s wr, 1.48k op/s 2024-02-18T11:52:19.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:52:19 smithi158 ceph-mon[36626]: pgmap v692: 97 pgs: 97 active+clean; 10 GiB data, 193 GiB used, 879 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 889 MiB/s wr, 1.75k op/s 2024-02-18T11:52:19.602 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:52:19 smithi188 ceph-mon[36925]: pgmap v692: 97 pgs: 97 active+clean; 10 GiB data, 193 GiB used, 879 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 889 MiB/s wr, 1.75k op/s 2024-02-18T11:52:19.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:52:19 smithi012 ceph-mon[32460]: pgmap v692: 97 pgs: 97 active+clean; 10 GiB data, 193 GiB used, 879 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 889 MiB/s wr, 1.75k op/s 2024-02-18T11:52:20.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:52:20 smithi158 ceph-mon[36626]: pgmap v693: 97 pgs: 97 active+clean; 10 GiB data, 187 GiB used, 886 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 866 MiB/s wr, 1.75k op/s 2024-02-18T11:52:20.852 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:52:20 smithi188 ceph-mon[36925]: pgmap v693: 97 pgs: 97 active+clean; 10 GiB data, 187 GiB used, 886 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 866 MiB/s wr, 1.75k op/s 2024-02-18T11:52:20.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:52:20 smithi012 ceph-mon[32460]: pgmap v693: 97 pgs: 97 active+clean; 10 GiB data, 187 GiB used, 886 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 866 MiB/s wr, 1.75k op/s 2024-02-18T11:52:22.993 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:52:22 smithi158 ceph-mon[36626]: pgmap v694: 97 pgs: 97 active+clean; 10 GiB data, 189 GiB used, 884 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 837 MiB/s wr, 1.54k op/s 2024-02-18T11:52:23.102 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:52:22 smithi188 ceph-mon[36925]: pgmap v694: 97 pgs: 97 active+clean; 10 GiB data, 189 GiB used, 884 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 837 MiB/s wr, 1.54k op/s 2024-02-18T11:52:23.171 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:52:22 smithi012 ceph-mon[32460]: pgmap v694: 97 pgs: 97 active+clean; 10 GiB data, 189 GiB used, 884 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 837 MiB/s wr, 1.54k op/s 2024-02-18T11:52:25.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:52:25 smithi012 ceph-mon[32460]: pgmap v695: 97 pgs: 97 active+clean; 10 GiB data, 194 GiB used, 879 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 940 MiB/s wr, 1.53k op/s 2024-02-18T11:52:25.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:52:25 smithi158 ceph-mon[36626]: pgmap v695: 97 pgs: 97 active+clean; 10 GiB data, 194 GiB used, 879 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 940 MiB/s wr, 1.53k op/s 2024-02-18T11:52:25.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:52:25 smithi188 ceph-mon[36925]: pgmap v695: 97 pgs: 97 active+clean; 10 GiB data, 194 GiB used, 879 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 940 MiB/s wr, 1.53k op/s 2024-02-18T11:52:27.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:52:27 smithi158 ceph-mon[36626]: pgmap v696: 97 pgs: 97 active+clean; 10 GiB data, 195 GiB used, 878 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 739 MiB/s wr, 1.26k op/s 2024-02-18T11:52:27.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:52:27 smithi188 ceph-mon[36925]: pgmap v696: 97 pgs: 97 active+clean; 10 GiB data, 195 GiB used, 878 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 739 MiB/s wr, 1.26k op/s 2024-02-18T11:52:27.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:52:27 smithi012 ceph-mon[32460]: pgmap v696: 97 pgs: 97 active+clean; 10 GiB data, 195 GiB used, 878 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 739 MiB/s wr, 1.26k op/s 2024-02-18T11:52:29.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:52:29 smithi158 ceph-mon[36626]: pgmap v697: 97 pgs: 97 active+clean; 10 GiB data, 199 GiB used, 874 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 923 MiB/s wr, 1.58k op/s 2024-02-18T11:52:29.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:52:29 smithi188 ceph-mon[36925]: pgmap v697: 97 pgs: 97 active+clean; 10 GiB data, 199 GiB used, 874 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 923 MiB/s wr, 1.58k op/s 2024-02-18T11:52:29.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:52:29 smithi012 ceph-mon[32460]: pgmap v697: 97 pgs: 97 active+clean; 10 GiB data, 199 GiB used, 874 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 923 MiB/s wr, 1.58k op/s 2024-02-18T11:52:31.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:52:31 smithi012 ceph-mon[32460]: pgmap v698: 97 pgs: 97 active+clean; 10 GiB data, 203 GiB used, 870 GiB / 1.0 TiB avail; 1.2 MiB/s rd, 864 MiB/s wr, 1.46k op/s 2024-02-18T11:52:31.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:52:31 smithi158 ceph-mon[36626]: pgmap v698: 97 pgs: 97 active+clean; 10 GiB data, 203 GiB used, 870 GiB / 1.0 TiB avail; 1.2 MiB/s rd, 864 MiB/s wr, 1.46k op/s 2024-02-18T11:52:31.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:52:31 smithi188 ceph-mon[36925]: pgmap v698: 97 pgs: 97 active+clean; 10 GiB data, 203 GiB used, 870 GiB / 1.0 TiB avail; 1.2 MiB/s rd, 864 MiB/s wr, 1.46k op/s 2024-02-18T11:52:33.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:52:32 smithi188 ceph-mon[36925]: pgmap v699: 97 pgs: 97 active+clean; 10 GiB data, 206 GiB used, 867 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 835 MiB/s wr, 1.48k op/s 2024-02-18T11:52:33.171 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:52:32 smithi012 ceph-mon[32460]: pgmap v699: 97 pgs: 97 active+clean; 10 GiB data, 206 GiB used, 867 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 835 MiB/s wr, 1.48k op/s 2024-02-18T11:52:33.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:52:32 smithi158 ceph-mon[36626]: pgmap v699: 97 pgs: 97 active+clean; 10 GiB data, 206 GiB used, 867 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 835 MiB/s wr, 1.48k op/s 2024-02-18T11:52:34.352 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:52:34 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:52:34.421 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:52:34 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:52:34.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:52:34 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:52:35.423 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:52:35 smithi012 ceph-mon[32460]: pgmap v700: 97 pgs: 97 active+clean; 10 GiB data, 211 GiB used, 862 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 934 MiB/s wr, 1.90k op/s 2024-02-18T11:52:35.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:52:35 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:52:35.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:52:35 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:52:35.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:52:35 smithi158 ceph-mon[36626]: pgmap v700: 97 pgs: 97 active+clean; 10 GiB data, 211 GiB used, 862 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 934 MiB/s wr, 1.90k op/s 2024-02-18T11:52:35.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:52:35 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:52:35.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:52:35 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:52:35.602 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:52:35 smithi188 ceph-mon[36925]: pgmap v700: 97 pgs: 97 active+clean; 10 GiB data, 211 GiB used, 862 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 934 MiB/s wr, 1.90k op/s 2024-02-18T11:52:35.602 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:52:35 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:52:35.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:52:35 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:52:36.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:52:36 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:52:36.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:52:36 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:52:36.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:52:36 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:52:36.602 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:52:36 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:52:36.602 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:52:36 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:52:36.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:52:36 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:52:36.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:52:36 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:52:36.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:52:36 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:52:36.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:52:36 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:52:37.602 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:52:37 smithi188 ceph-mon[36925]: pgmap v701: 97 pgs: 97 active+clean; 10 GiB data, 212 GiB used, 861 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 737 MiB/s wr, 1.62k op/s 2024-02-18T11:52:37.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:52:37 smithi012 ceph-mon[32460]: pgmap v701: 97 pgs: 97 active+clean; 10 GiB data, 212 GiB used, 861 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 737 MiB/s wr, 1.62k op/s 2024-02-18T11:52:37.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:52:37 smithi158 ceph-mon[36626]: pgmap v701: 97 pgs: 97 active+clean; 10 GiB data, 212 GiB used, 861 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 737 MiB/s wr, 1.62k op/s 2024-02-18T11:52:39.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:52:39 smithi012 ceph-mon[32460]: pgmap v702: 97 pgs: 97 active+clean; 10 GiB data, 215 GiB used, 857 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 926 MiB/s wr, 1.84k op/s 2024-02-18T11:52:39.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:52:39 smithi158 ceph-mon[36626]: pgmap v702: 97 pgs: 97 active+clean; 10 GiB data, 215 GiB used, 857 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 926 MiB/s wr, 1.84k op/s 2024-02-18T11:52:39.852 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:52:39 smithi188 ceph-mon[36925]: pgmap v702: 97 pgs: 97 active+clean; 10 GiB data, 215 GiB used, 857 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 926 MiB/s wr, 1.84k op/s 2024-02-18T11:52:40.993 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:52:40 smithi158 ceph-mon[36626]: pgmap v703: 97 pgs: 97 active+clean; 10 GiB data, 220 GiB used, 853 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 873 MiB/s wr, 1.75k op/s 2024-02-18T11:52:41.102 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:52:40 smithi188 ceph-mon[36925]: pgmap v703: 97 pgs: 97 active+clean; 10 GiB data, 220 GiB used, 853 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 873 MiB/s wr, 1.75k op/s 2024-02-18T11:52:41.172 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:52:40 smithi012 ceph-mon[32460]: pgmap v703: 97 pgs: 97 active+clean; 10 GiB data, 220 GiB used, 853 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 873 MiB/s wr, 1.75k op/s 2024-02-18T11:52:43.352 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:52:43 smithi188 ceph-mon[36925]: pgmap v704: 97 pgs: 97 active+clean; 10 GiB data, 222 GiB used, 851 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 844 MiB/s wr, 1.78k op/s 2024-02-18T11:52:43.428 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:52:43 smithi012 ceph-mon[32460]: pgmap v704: 97 pgs: 97 active+clean; 10 GiB data, 222 GiB used, 851 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 844 MiB/s wr, 1.78k op/s 2024-02-18T11:52:43.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:52:43 smithi158 ceph-mon[36626]: pgmap v704: 97 pgs: 97 active+clean; 10 GiB data, 222 GiB used, 851 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 844 MiB/s wr, 1.78k op/s 2024-02-18T11:52:45.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:52:45 smithi012 ceph-mon[32460]: pgmap v705: 97 pgs: 97 active+clean; 10 GiB data, 226 GiB used, 847 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 897 MiB/s wr, 2.02k op/s 2024-02-18T11:52:45.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:52:45 smithi158 ceph-mon[36626]: pgmap v705: 97 pgs: 97 active+clean; 10 GiB data, 226 GiB used, 847 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 897 MiB/s wr, 2.02k op/s 2024-02-18T11:52:45.602 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:52:45 smithi188 ceph-mon[36925]: pgmap v705: 97 pgs: 97 active+clean; 10 GiB data, 226 GiB used, 847 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 897 MiB/s wr, 2.02k op/s 2024-02-18T11:52:47.001 INFO:tasks.workunit.client.0.smithi012.stdout:OK 2024-02-18T11:52:47.001 INFO:tasks.workunit.client.0.smithi012.stderr:+ echo OK 2024-02-18T11:52:47.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:52:47 smithi012 ceph-mon[32460]: pgmap v706: 97 pgs: 97 active+clean; 10 GiB data, 228 GiB used, 845 GiB / 1.0 TiB avail; 1006 KiB/s rd, 736 MiB/s wr, 1.56k op/s 2024-02-18T11:52:47.742 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:52:47 smithi158 ceph-mon[36626]: pgmap v706: 97 pgs: 97 active+clean; 10 GiB data, 228 GiB used, 845 GiB / 1.0 TiB avail; 1006 KiB/s rd, 736 MiB/s wr, 1.56k op/s 2024-02-18T11:52:47.852 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:52:47 smithi188 ceph-mon[36925]: pgmap v706: 97 pgs: 97 active+clean; 10 GiB data, 228 GiB used, 845 GiB / 1.0 TiB avail; 1006 KiB/s rd, 736 MiB/s wr, 1.56k op/s 2024-02-18T11:52:48.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:52:48 smithi012 ceph-mon[32460]: pgmap v707: 97 pgs: 97 active+clean; 10 GiB data, 232 GiB used, 841 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 917 MiB/s wr, 1.91k op/s 2024-02-18T11:52:48.993 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:52:48 smithi158 ceph-mon[36626]: pgmap v707: 97 pgs: 97 active+clean; 10 GiB data, 232 GiB used, 841 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 917 MiB/s wr, 1.91k op/s 2024-02-18T11:52:49.102 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:52:48 smithi188 ceph-mon[36925]: pgmap v707: 97 pgs: 97 active+clean; 10 GiB data, 232 GiB used, 841 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 917 MiB/s wr, 1.91k op/s 2024-02-18T11:52:49.339 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-02-18T11:52:49.339 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-02-18T11:52:50.852 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:52:50 smithi188 ceph-mon[36925]: pgmap v708: 97 pgs: 97 active+clean; 10 GiB data, 228 GiB used, 845 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 845 MiB/s wr, 1.80k op/s 2024-02-18T11:52:50.921 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:52:50 smithi012 ceph-mon[32460]: pgmap v708: 97 pgs: 97 active+clean; 10 GiB data, 228 GiB used, 845 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 845 MiB/s wr, 1.80k op/s 2024-02-18T11:52:50.992 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:52:50 smithi158 ceph-mon[36626]: pgmap v708: 97 pgs: 97 active+clean; 10 GiB data, 228 GiB used, 845 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 845 MiB/s wr, 1.80k op/s 2024-02-18T11:52:53.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:52:53 smithi188 ceph-mon[36925]: pgmap v709: 97 pgs: 97 active+clean; 10 GiB data, 230 GiB used, 843 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 772 MiB/s wr, 1.64k op/s 2024-02-18T11:52:53.671 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:52:53 smithi012 ceph-mon[32460]: pgmap v709: 97 pgs: 97 active+clean; 10 GiB data, 230 GiB used, 843 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 772 MiB/s wr, 1.64k op/s 2024-02-18T11:52:53.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:52:53 smithi158 ceph-mon[36626]: pgmap v709: 97 pgs: 97 active+clean; 10 GiB data, 230 GiB used, 843 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 772 MiB/s wr, 1.64k op/s 2024-02-18T11:52:54.742 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:52:54 smithi158 ceph-mon[36626]: pgmap v710: 97 pgs: 97 active+clean; 10 GiB data, 213 GiB used, 860 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 720 MiB/s wr, 1.47k op/s 2024-02-18T11:52:54.852 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:52:54 smithi188 ceph-mon[36925]: pgmap v710: 97 pgs: 97 active+clean; 10 GiB data, 213 GiB used, 860 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 720 MiB/s wr, 1.47k op/s 2024-02-18T11:52:54.921 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:52:54 smithi012 ceph-mon[32460]: pgmap v710: 97 pgs: 97 active+clean; 10 GiB data, 213 GiB used, 860 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 720 MiB/s wr, 1.47k op/s 2024-02-18T11:52:56.852 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:52:56 smithi188 ceph-mon[36925]: pgmap v711: 97 pgs: 97 active+clean; 10 GiB data, 200 GiB used, 872 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 553 MiB/s wr, 920 op/s 2024-02-18T11:52:56.921 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:52:56 smithi012 ceph-mon[32460]: pgmap v711: 97 pgs: 97 active+clean; 10 GiB data, 200 GiB used, 872 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 553 MiB/s wr, 920 op/s 2024-02-18T11:52:56.992 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:52:56 smithi158 ceph-mon[36626]: pgmap v711: 97 pgs: 97 active+clean; 10 GiB data, 200 GiB used, 872 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 553 MiB/s wr, 920 op/s 2024-02-18T11:52:58.852 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:52:58 smithi188 ceph-mon[36925]: pgmap v712: 97 pgs: 97 active+clean; 10 GiB data, 175 GiB used, 897 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 516 MiB/s wr, 799 op/s 2024-02-18T11:52:58.921 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:52:58 smithi012 ceph-mon[32460]: pgmap v712: 97 pgs: 97 active+clean; 10 GiB data, 175 GiB used, 897 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 516 MiB/s wr, 799 op/s 2024-02-18T11:52:58.992 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:52:58 smithi158 ceph-mon[36626]: pgmap v712: 97 pgs: 97 active+clean; 10 GiB data, 175 GiB used, 897 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 516 MiB/s wr, 799 op/s 2024-02-18T11:53:00.921 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:53:00 smithi012 ceph-mon[32460]: pgmap v713: 97 pgs: 97 active+clean; 10 GiB data, 164 GiB used, 909 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 328 MiB/s wr, 442 op/s 2024-02-18T11:53:00.992 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:53:00 smithi158 ceph-mon[36626]: pgmap v713: 97 pgs: 97 active+clean; 10 GiB data, 164 GiB used, 909 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 328 MiB/s wr, 442 op/s 2024-02-18T11:53:01.102 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:53:00 smithi188 ceph-mon[36925]: pgmap v713: 97 pgs: 97 active+clean; 10 GiB data, 164 GiB used, 909 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 328 MiB/s wr, 442 op/s 2024-02-18T11:53:02.921 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:53:02 smithi012 ceph-mon[32460]: pgmap v714: 97 pgs: 97 active+clean; 10 GiB data, 164 GiB used, 908 GiB / 1.0 TiB avail; 2.5 MiB/s rd, 203 MiB/s wr, 333 op/s 2024-02-18T11:53:02.992 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:53:02 smithi158 ceph-mon[36626]: pgmap v714: 97 pgs: 97 active+clean; 10 GiB data, 164 GiB used, 908 GiB / 1.0 TiB avail; 2.5 MiB/s rd, 203 MiB/s wr, 333 op/s 2024-02-18T11:53:03.102 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:53:02 smithi188 ceph-mon[36925]: pgmap v714: 97 pgs: 97 active+clean; 10 GiB data, 164 GiB used, 908 GiB / 1.0 TiB avail; 2.5 MiB/s rd, 203 MiB/s wr, 333 op/s 2024-02-18T11:53:05.102 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:53:04 smithi188 ceph-mon[36925]: pgmap v715: 97 pgs: 97 active+clean; 10 GiB data, 138 GiB used, 935 GiB / 1.0 TiB avail; 2.8 MiB/s rd, 147 MiB/s wr, 293 op/s 2024-02-18T11:53:05.171 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:53:04 smithi012 ceph-mon[32460]: pgmap v715: 97 pgs: 97 active+clean; 10 GiB data, 138 GiB used, 935 GiB / 1.0 TiB avail; 2.8 MiB/s rd, 147 MiB/s wr, 293 op/s 2024-02-18T11:53:05.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:53:04 smithi158 ceph-mon[36626]: pgmap v715: 97 pgs: 97 active+clean; 10 GiB data, 138 GiB used, 935 GiB / 1.0 TiB avail; 2.8 MiB/s rd, 147 MiB/s wr, 293 op/s 2024-02-18T11:53:07.102 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:53:06 smithi188 ceph-mon[36925]: pgmap v716: 97 pgs: 97 active+clean; 10 GiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 2.4 MiB/s rd, 80 MiB/s wr, 192 op/s 2024-02-18T11:53:07.172 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:53:06 smithi012 ceph-mon[32460]: pgmap v716: 97 pgs: 97 active+clean; 10 GiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 2.4 MiB/s rd, 80 MiB/s wr, 192 op/s 2024-02-18T11:53:07.242 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:53:06 smithi158 ceph-mon[36626]: pgmap v716: 97 pgs: 97 active+clean; 10 GiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 2.4 MiB/s rd, 80 MiB/s wr, 192 op/s 2024-02-18T11:53:09.102 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:53:08 smithi188 ceph-mon[36925]: pgmap v717: 97 pgs: 97 active+clean; 10 GiB data, 94 GiB used, 979 GiB / 1.0 TiB avail; 3.7 MiB/s rd, 91 MiB/s wr, 228 op/s 2024-02-18T11:53:09.171 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:53:08 smithi012 ceph-mon[32460]: pgmap v717: 97 pgs: 97 active+clean; 10 GiB data, 94 GiB used, 979 GiB / 1.0 TiB avail; 3.7 MiB/s rd, 91 MiB/s wr, 228 op/s 2024-02-18T11:53:09.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:53:08 smithi158 ceph-mon[36626]: pgmap v717: 97 pgs: 97 active+clean; 10 GiB data, 94 GiB used, 979 GiB / 1.0 TiB avail; 3.7 MiB/s rd, 91 MiB/s wr, 228 op/s 2024-02-18T11:53:11.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:53:10 smithi158 ceph-mon[36626]: pgmap v718: 97 pgs: 97 active+clean; 10 GiB data, 61 GiB used, 1012 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 84 MiB/s wr, 210 op/s 2024-02-18T11:53:11.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:53:10 smithi188 ceph-mon[36925]: pgmap v718: 97 pgs: 97 active+clean; 10 GiB data, 61 GiB used, 1012 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 84 MiB/s wr, 210 op/s 2024-02-18T11:53:11.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:53:10 smithi012 ceph-mon[32460]: pgmap v718: 97 pgs: 97 active+clean; 10 GiB data, 61 GiB used, 1012 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 84 MiB/s wr, 210 op/s 2024-02-18T11:53:13.242 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:53:12 smithi158 ceph-mon[36626]: pgmap v719: 97 pgs: 97 active+clean; 10 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 89 MiB/s wr, 245 op/s 2024-02-18T11:53:13.352 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:53:12 smithi188 ceph-mon[36925]: pgmap v719: 97 pgs: 97 active+clean; 10 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 89 MiB/s wr, 245 op/s 2024-02-18T11:53:13.421 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:53:12 smithi012 ceph-mon[32460]: pgmap v719: 97 pgs: 97 active+clean; 10 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 89 MiB/s wr, 245 op/s 2024-02-18T11:53:15.352 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:53:15 smithi188 ceph-mon[36925]: pgmap v720: 97 pgs: 97 active+clean; 10 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 100 MiB/s wr, 290 op/s 2024-02-18T11:53:15.421 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:53:15 smithi012 ceph-mon[32460]: pgmap v720: 97 pgs: 97 active+clean; 10 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 100 MiB/s wr, 290 op/s 2024-02-18T11:53:15.492 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:53:15 smithi158 ceph-mon[36626]: pgmap v720: 97 pgs: 97 active+clean; 10 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 100 MiB/s wr, 290 op/s 2024-02-18T11:53:17.295 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:53:17 smithi158 ceph-mon[36626]: pgmap v721: 97 pgs: 97 active+clean; 10 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 85 MiB/s wr, 260 op/s 2024-02-18T11:53:17.352 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:53:17 smithi188 ceph-mon[36925]: pgmap v721: 97 pgs: 97 active+clean; 10 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 85 MiB/s wr, 260 op/s 2024-02-18T11:53:17.421 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:53:17 smithi012 ceph-mon[32460]: pgmap v721: 97 pgs: 97 active+clean; 10 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 85 MiB/s wr, 260 op/s 2024-02-18T11:53:19.352 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:53:19 smithi188 ceph-mon[36925]: pgmap v722: 97 pgs: 97 active+clean; 10 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 101 MiB/s wr, 326 op/s 2024-02-18T11:53:19.421 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:53:19 smithi012 ceph-mon[32460]: pgmap v722: 97 pgs: 97 active+clean; 10 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 101 MiB/s wr, 326 op/s 2024-02-18T11:53:19.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:53:19 smithi158 ceph-mon[36626]: pgmap v722: 97 pgs: 97 active+clean; 10 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 101 MiB/s wr, 326 op/s 2024-02-18T11:53:20.947 INFO:tasks.workunit:Stopping ['suites/iogen.sh'] on client.0... 2024-02-18T11:53:20.947 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-02-18T11:53:21.226 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:53:21 smithi012 ceph-mon[32460]: pgmap v723: 97 pgs: 97 active+clean; 10 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 96 MiB/s wr, 332 op/s 2024-02-18T11:53:21.227 DEBUG:teuthology.parallel:result is None 2024-02-18T11:53:21.227 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-02-18T11:53:21.260 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-02-18T11:53:21.261 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-02-18T11:53:21.278 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-18T11:53:21.352 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:53:21 smithi188 ceph-mon[36925]: pgmap v723: 97 pgs: 97 active+clean; 10 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 96 MiB/s wr, 332 op/s 2024-02-18T11:53:21.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.412+0000 7faf90956640 1 -- 172.21.15.12:0/2146147890 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf8c104630 msgr2=0x7faf8c106a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:53:21.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.412+0000 7faf90956640 1 --2- 172.21.15.12:0/2146147890 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf8c104630 0x7faf8c106a20 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7faf74009900 tx=0x7faf7402bc30 comp rx=0 tx=0).stop 2024-02-18T11:53:21.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.412+0000 7faf90956640 1 -- 172.21.15.12:0/2146147890 shutdown_connections 2024-02-18T11:53:21.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.412+0000 7faf90956640 1 --2- 172.21.15.12:0/2146147890 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7faf8c106f60 0x7faf8c109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:53:21.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.412+0000 7faf90956640 1 --2- 172.21.15.12:0/2146147890 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf8c104630 0x7faf8c106a20 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:53:21.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.412+0000 7faf90956640 1 --2- 172.21.15.12:0/2146147890 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf8c101d00 0x7faf8c1040f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:53:21.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.412+0000 7faf90956640 1 -- 172.21.15.12:0/2146147890 >> 172.21.15.12:0/2146147890 conn(0x7faf8c0fb650 msgr2=0x7faf8c0fda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:53:21.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.412+0000 7faf90956640 1 -- 172.21.15.12:0/2146147890 shutdown_connections 2024-02-18T11:53:21.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.412+0000 7faf90956640 1 -- 172.21.15.12:0/2146147890 wait complete. 2024-02-18T11:53:21.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.412+0000 7faf90956640 1 Processor -- start 2024-02-18T11:53:21.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.413+0000 7faf90956640 1 -- start start 2024-02-18T11:53:21.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.413+0000 7faf90956640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7faf8c101d00 0x7faf8c197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:53:21.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.413+0000 7faf90956640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf8c104630 0x7faf8c197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:53:21.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.413+0000 7faf90956640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf8c106f60 0x7faf8c19e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:53:21.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.413+0000 7faf90956640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf8c19ee20 con 0x7faf8c104630 2024-02-18T11:53:21.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.413+0000 7faf90956640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf8c19ef90 con 0x7faf8c101d00 2024-02-18T11:53:21.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.413+0000 7faf90956640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf8c19f150 con 0x7faf8c106f60 2024-02-18T11:53:21.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.413+0000 7faf89d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf8c104630 0x7faf8c197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:53:21.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.413+0000 7faf89d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf8c104630 0x7faf8c197870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60524/0 (socket says 172.21.15.12:60524) 2024-02-18T11:53:21.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.413+0000 7faf89d9b640 1 -- 172.21.15.12:0/1218827753 learned_addr learned my addr 172.21.15.12:0/1218827753 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:53:21.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.413+0000 7faf8ad9d640 1 --2- 172.21.15.12:0/1218827753 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf8c106f60 0x7faf8c19e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:53:21.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.413+0000 7faf8a59c640 1 --2- 172.21.15.12:0/1218827753 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7faf8c101d00 0x7faf8c197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:53:21.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.413+0000 7faf89d9b640 1 -- 172.21.15.12:0/1218827753 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf8c106f60 msgr2=0x7faf8c19e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:53:21.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.413+0000 7faf89d9b640 1 --2- 172.21.15.12:0/1218827753 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf8c106f60 0x7faf8c19e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:53:21.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.413+0000 7faf89d9b640 1 -- 172.21.15.12:0/1218827753 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7faf8c101d00 msgr2=0x7faf8c197330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:53:21.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.413+0000 7faf89d9b640 1 --2- 172.21.15.12:0/1218827753 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7faf8c101d00 0x7faf8c197330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:53:21.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.413+0000 7faf89d9b640 1 -- 172.21.15.12:0/1218827753 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf74009560 con 0x7faf8c104630 2024-02-18T11:53:21.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.413+0000 7faf8a59c640 1 --2- 172.21.15.12:0/1218827753 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7faf8c101d00 0x7faf8c197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:53:21.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.413+0000 7faf89d9b640 1 --2- 172.21.15.12:0/1218827753 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf8c104630 0x7faf8c197870 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7faf74000c00 tx=0x7faf74004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:53:21.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.413+0000 7faf837fe640 1 -- 172.21.15.12:0/1218827753 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf74004df0 con 0x7faf8c104630 2024-02-18T11:53:21.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.413+0000 7faf837fe640 1 -- 172.21.15.12:0/1218827753 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faf74005430 con 0x7faf8c104630 2024-02-18T11:53:21.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.413+0000 7faf837fe640 1 -- 172.21.15.12:0/1218827753 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf7403e450 con 0x7faf8c104630 2024-02-18T11:53:21.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.414+0000 7faf90956640 1 -- 172.21.15.12:0/1218827753 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf8c19f3d0 con 0x7faf8c104630 2024-02-18T11:53:21.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.414+0000 7faf90956640 1 -- 172.21.15.12:0/1218827753 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf8c19f960 con 0x7faf8c104630 2024-02-18T11:53:21.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.414+0000 7faf90956640 1 -- 172.21.15.12:0/1218827753 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf8c0fd140 con 0x7faf8c104630 2024-02-18T11:53:21.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.416+0000 7faf837fe640 1 -- 172.21.15.12:0/1218827753 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 20) v1 ==== 137245+0+0 (secure 0 0 0) 0x7faf740055a0 con 0x7faf8c104630 2024-02-18T11:53:21.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.416+0000 7faf837fe640 1 --2- 172.21.15.12:0/1218827753 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7faf780a2750 0x7faf780a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:53:21.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.416+0000 7faf8a59c640 1 --2- 172.21.15.12:0/1218827753 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7faf780a2750 0x7faf780a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:53:21.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.416+0000 7faf837fe640 1 -- 172.21.15.12:0/1218827753 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8211+0+0 (secure 0 0 0) 0x7faf740f0fe0 con 0x7faf8c104630 2024-02-18T11:53:21.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.417+0000 7faf8a59c640 1 --2- 172.21.15.12:0/1218827753 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7faf780a2750 0x7faf780a4c10 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7faf84009b40 tx=0x7faf84005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:53:21.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.420+0000 7faf837fe640 1 -- 172.21.15.12:0/1218827753 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7faf740b9e20 con 0x7faf8c104630 2024-02-18T11:53:21.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:53:21 smithi158 ceph-mon[36626]: pgmap v723: 97 pgs: 97 active+clean; 10 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 96 MiB/s wr, 332 op/s 2024-02-18T11:53:21.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.604+0000 7faf90956640 1 -- 172.21.15.12:0/1218827753 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faf8c10cea0 con 0x7faf8c104630 2024-02-18T11:53:21.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.605+0000 7faf837fe640 1 -- 172.21.15.12:0/1218827753 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+22662 (secure 0 0 0) 0x7faf740bd4e0 con 0x7faf8c104630 2024-02-18T11:53:21.598 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:53:21.598 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":89,"fsid":"0c8bb0de-ce51-11ee-95ba-87774f69a715","created":"2024-02-18T11:30:49.110059+0000","modified":"2024-02-18T11:39:52.478480+0000","last_up_change":"2024-02-18T11:37:57.512337+0000","last_in_change":"2024-02-18T11:37:36.567969+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-18T11:34:15.281138+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-18T11:38:44.802134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-18T11:38:46.081188+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"35f15ea6-deb8-48a1-a7dd-ab649ec13533","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":536522426},{"type":"v1","addr":"172.21.15.12:6803","nonce":536522426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":536522426},{"type":"v1","addr":"172.21.15.12:6805","nonce":536522426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":536522426},{"type":"v1","addr":"172.21.15.12:6809","nonce":536522426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":536522426},{"type":"v1","addr":"172.21.15.12:6807","nonce":536522426}]},"public_addr":"172.21.15.12:6803/536522426","cluster_addr":"172.21.15.12:6805/536522426","heartbeat_back_addr":"172.21.15.12:6809/536522426","heartbeat_front_addr":"172.21.15.12:6807/536522426","state":["exists","up"]},{"osd":1,"uuid":"2af937ef-d827-44e6-b805-70cffbfe1126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":4249111488},{"type":"v1","addr":"172.21.15.158:6801","nonce":4249111488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":4249111488},{"type":"v1","addr":"172.21.15.158:6803","nonce":4249111488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":4249111488},{"type":"v1","addr":"172.21.15.158:6807","nonce":4249111488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":4249111488},{"type":"v1","addr":"172.21.15.158:6805","nonce":4249111488}]},"public_addr":"172.21.15.158:6801/4249111488","cluster_addr":"172.21.15.158:6803/4249111488","heartbeat_back_addr":"172.21.15.158:6807/4249111488","heartbeat_front_addr":"172.21.15.158:6805/4249111488","state":["exists","up"]},{"osd":2,"uuid":"17b804dd-31df-45ae-a637-4a5f65bbaf26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":627976150},{"type":"v1","addr":"172.21.15.188:6801","nonce":627976150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":627976150},{"type":"v1","addr":"172.21.15.188:6803","nonce":627976150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":627976150},{"type":"v1","addr":"172.21.15.188:6807","nonce":627976150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":627976150},{"type":"v1","addr":"172.21.15.188:6805","nonce":627976150}]},"public_addr":"172.21.15.188:6801/627976150","cluster_addr":"172.21.15.188:6803/627976150","heartbeat_back_addr":"172.21.15.188:6807/627976150","heartbeat_front_addr":"172.21.15.188:6805/627976150","state":["exists","up"]},{"osd":3,"uuid":"3f9cc423-cdfa-4acf-aaa2-b58b32ccb9cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":3758512503},{"type":"v1","addr":"172.21.15.12:6811","nonce":3758512503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":3758512503},{"type":"v1","addr":"172.21.15.12:6813","nonce":3758512503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3758512503},{"type":"v1","addr":"172.21.15.12:6817","nonce":3758512503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":3758512503},{"type":"v1","addr":"172.21.15.12:6815","nonce":3758512503}]},"public_addr":"172.21.15.12:6811/3758512503","cluster_addr":"172.21.15.12:6813/3758512503","heartbeat_back_addr":"172.21.15.12:6817/3758512503","heartbeat_front_addr":"172.21.15.12:6815/3758512503","state":["exists","up"]},{"osd":4,"uuid":"a92e20ec-af1c-4ef4-a0cb-caba5fcc71aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":3892820997},{"type":"v1","addr":"172.21.15.158:6809","nonce":3892820997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":3892820997},{"type":"v1","addr":"172.21.15.158:6811","nonce":3892820997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":3892820997},{"type":"v1","addr":"172.21.15.158:6815","nonce":3892820997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":3892820997},{"type":"v1","addr":"172.21.15.158:6813","nonce":3892820997}]},"public_addr":"172.21.15.158:6809/3892820997","cluster_addr":"172.21.15.158:6811/3892820997","heartbeat_back_addr":"172.21.15.158:6815/3892820997","heartbeat_front_addr":"172.21.15.158:6813/3892820997","state":["exists","up"]},{"osd":5,"uuid":"d9c2e0c8-85f5-4ae2-bf3e-125c1be3a2db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3939495896},{"type":"v1","addr":"172.21.15.188:6809","nonce":3939495896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3939495896},{"type":"v1","addr":"172.21.15.188:6811","nonce":3939495896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3939495896},{"type":"v1","addr":"172.21.15.188:6815","nonce":3939495896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3939495896},{"type":"v1","addr":"172.21.15.188:6813","nonce":3939495896}]},"public_addr":"172.21.15.188:6809/3939495896","cluster_addr":"172.21.15.188:6811/3939495896","heartbeat_back_addr":"172.21.15.188:6815/3939495896","heartbeat_front_addr":"172.21.15.188:6813/3939495896","state":["exists","up"]},{"osd":6,"uuid":"4a523512-de6b-49f0-be81-00f2f00c1c10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":1206337767},{"type":"v1","addr":"172.21.15.12:6819","nonce":1206337767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":1206337767},{"type":"v1","addr":"172.21.15.12:6821","nonce":1206337767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":1206337767},{"type":"v1","addr":"172.21.15.12:6825","nonce":1206337767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":1206337767},{"type":"v1","addr":"172.21.15.12:6823","nonce":1206337767}]},"public_addr":"172.21.15.12:6819/1206337767","cluster_addr":"172.21.15.12:6821/1206337767","heartbeat_back_addr":"172.21.15.12:6825/1206337767","heartbeat_front_addr":"172.21.15.12:6823/1206337767","state":["exists","up"]},{"osd":7,"uuid":"363271a6-df93-44c9-b0e5-3d29768aa96c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":3283400032},{"type":"v1","addr":"172.21.15.158:6817","nonce":3283400032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":3283400032},{"type":"v1","addr":"172.21.15.158:6819","nonce":3283400032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":3283400032},{"type":"v1","addr":"172.21.15.158:6823","nonce":3283400032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":3283400032},{"type":"v1","addr":"172.21.15.158:6821","nonce":3283400032}]},"public_addr":"172.21.15.158:6817/3283400032","cluster_addr":"172.21.15.158:6819/3283400032","heartbeat_back_addr":"172.21.15.158:6823/3283400032","heartbeat_front_addr":"172.21.15.158:6821/3283400032","state":["exists","up"]},{"osd":8,"uuid":"9a089528-3e8e-45d3-9279-ddb54088b96a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2110170682},{"type":"v1","addr":"172.21.15.188:6817","nonce":2110170682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2110170682},{"type":"v1","addr":"172.21.15.188:6819","nonce":2110170682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":2110170682},{"type":"v1","addr":"172.21.15.188:6823","nonce":2110170682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2110170682},{"type":"v1","addr":"172.21.15.188:6821","nonce":2110170682}]},"public_addr":"172.21.15.188:6817/2110170682","cluster_addr":"172.21.15.188:6819/2110170682","heartbeat_back_addr":"172.21.15.188:6823/2110170682","heartbeat_front_addr":"172.21.15.188:6821/2110170682","state":["exists","up"]},{"osd":9,"uuid":"b12e2727-d649-4464-90e1-fafbc1754832","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1825928986},{"type":"v1","addr":"172.21.15.12:6827","nonce":1825928986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1825928986},{"type":"v1","addr":"172.21.15.12:6829","nonce":1825928986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":1825928986},{"type":"v1","addr":"172.21.15.12:6833","nonce":1825928986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1825928986},{"type":"v1","addr":"172.21.15.12:6831","nonce":1825928986}]},"public_addr":"172.21.15.12:6827/1825928986","cluster_addr":"172.21.15.12:6829/1825928986","heartbeat_back_addr":"172.21.15.12:6833/1825928986","heartbeat_front_addr":"172.21.15.12:6831/1825928986","state":["exists","up"]},{"osd":10,"uuid":"518b4cbf-9d90-4e9a-8ac9-b5cb9b3dee4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":3239052167},{"type":"v1","addr":"172.21.15.158:6825","nonce":3239052167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":3239052167},{"type":"v1","addr":"172.21.15.158:6827","nonce":3239052167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":3239052167},{"type":"v1","addr":"172.21.15.158:6831","nonce":3239052167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":3239052167},{"type":"v1","addr":"172.21.15.158:6829","nonce":3239052167}]},"public_addr":"172.21.15.158:6825/3239052167","cluster_addr":"172.21.15.158:6827/3239052167","heartbeat_back_addr":"172.21.15.158:6831/3239052167","heartbeat_front_addr":"172.21.15.158:6829/3239052167","state":["exists","up"]},{"osd":11,"uuid":"98aa3a0d-4b2f-46c8-9341-375d13794c3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1135773350},{"type":"v1","addr":"172.21.15.188:6825","nonce":1135773350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1135773350},{"type":"v1","addr":"172.21.15.188:6827","nonce":1135773350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1135773350},{"type":"v1","addr":"172.21.15.188:6831","nonce":1135773350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1135773350},{"type":"v1","addr":"172.21.15.188:6829","nonce":1135773350}]},"public_addr":"172.21.15.188:6825/1135773350","cluster_addr":"172.21.15.188:6827/1135773350","heartbeat_back_addr":"172.21.15.188:6831/1135773350","heartbeat_front_addr":"172.21.15.188:6829/1135773350","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:33:24.280879+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:33:47.859461+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:34:11.725283+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:34:35.707629+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:35:00.024197+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:35:24.499803+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:35:48.953870+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:36:13.642456+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:36:38.936457+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:37:04.021466+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:37:29.073178+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:37:54.863919+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.37","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:6801/4135274411":"2024-02-19T11:31:50.936928+0000","172.21.15.12:6800/4135274411":"2024-02-19T11:31:50.936928+0000","172.21.15.12:0/118428584":"2024-02-19T11:31:50.936928+0000","172.21.15.12:0/4276810915":"2024-02-19T11:31:50.936928+0000","172.21.15.12:0/3777545262":"2024-02-19T11:31:24.000830+0000","172.21.15.12:0/114707743":"2024-02-19T11:31:24.000830+0000","172.21.15.12:6800/3925204723":"2024-02-19T11:31:24.000830+0000","172.21.15.12:6801/3925204723":"2024-02-19T11:31:24.000830+0000","172.21.15.12:0/74881618":"2024-02-19T11:31:24.000830+0000","172.21.15.12:0/1614805375":"2024-02-19T11:31:50.936928+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-18T11:53:21.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.607+0000 7faf90956640 1 -- 172.21.15.12:0/1218827753 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7faf780a2750 msgr2=0x7faf780a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:53:21.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.607+0000 7faf90956640 1 --2- 172.21.15.12:0/1218827753 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7faf780a2750 0x7faf780a4c10 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7faf84009b40 tx=0x7faf84005e50 comp rx=0 tx=0).stop 2024-02-18T11:53:21.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.607+0000 7faf90956640 1 -- 172.21.15.12:0/1218827753 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf8c104630 msgr2=0x7faf8c197870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:53:21.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.607+0000 7faf90956640 1 --2- 172.21.15.12:0/1218827753 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf8c104630 0x7faf8c197870 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7faf74000c00 tx=0x7faf74004270 comp rx=0 tx=0).stop 2024-02-18T11:53:21.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.607+0000 7faf90956640 1 -- 172.21.15.12:0/1218827753 shutdown_connections 2024-02-18T11:53:21.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.607+0000 7faf90956640 1 --2- 172.21.15.12:0/1218827753 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7faf780a2750 0x7faf780a4c10 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:53:21.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.607+0000 7faf90956640 1 --2- 172.21.15.12:0/1218827753 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf8c106f60 0x7faf8c19e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:53:21.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.607+0000 7faf90956640 1 --2- 172.21.15.12:0/1218827753 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf8c104630 0x7faf8c197870 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:53:21.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.607+0000 7faf90956640 1 --2- 172.21.15.12:0/1218827753 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7faf8c101d00 0x7faf8c197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:53:21.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.607+0000 7faf90956640 1 -- 172.21.15.12:0/1218827753 >> 172.21.15.12:0/1218827753 conn(0x7faf8c0fb650 msgr2=0x7faf8c102720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:53:21.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.607+0000 7faf90956640 1 -- 172.21.15.12:0/1218827753 shutdown_connections 2024-02-18T11:53:21.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.607+0000 7faf90956640 1 -- 172.21.15.12:0/1218827753 wait complete. 2024-02-18T11:53:21.612 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-18T11:34:15.281138+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-18T11:38:44.802134+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-18T11:38:46.081188+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-18T11:53:21.612 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-18T11:53:21.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.726+0000 7fdcb0348640 1 -- 172.21.15.12:0/2125958711 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdca8101c80 msgr2=0x7fdca8108510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:53:21.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.726+0000 7fdcb0348640 1 --2- 172.21.15.12:0/2125958711 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdca8101c80 0x7fdca8108510 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fdca0009ca0 tx=0x7fdca002c0f0 comp rx=0 tx=0).stop 2024-02-18T11:53:21.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.726+0000 7fdcb0348640 1 -- 172.21.15.12:0/2125958711 shutdown_connections 2024-02-18T11:53:21.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.726+0000 7fdcb0348640 1 --2- 172.21.15.12:0/2125958711 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdca8101c80 0x7fdca8108510 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:53:21.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.726+0000 7fdcb0348640 1 --2- 172.21.15.12:0/2125958711 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdca81012c0 0x7fdca8101740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:53:21.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.726+0000 7fdcb0348640 1 --2- 172.21.15.12:0/2125958711 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdca81000d0 0x7fdca81004f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:53:21.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.726+0000 7fdcb0348640 1 -- 172.21.15.12:0/2125958711 >> 172.21.15.12:0/2125958711 conn(0x7fdca80fb660 msgr2=0x7fdca80fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:53:21.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.726+0000 7fdcb0348640 1 -- 172.21.15.12:0/2125958711 shutdown_connections 2024-02-18T11:53:21.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.726+0000 7fdcb0348640 1 -- 172.21.15.12:0/2125958711 wait complete. 2024-02-18T11:53:21.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.727+0000 7fdcb0348640 1 Processor -- start 2024-02-18T11:53:21.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.727+0000 7fdcb0348640 1 -- start start 2024-02-18T11:53:21.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.727+0000 7fdcb0348640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdca81000d0 0x7fdca81972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:53:21.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.727+0000 7fdcb0348640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdca81012c0 0x7fdca81977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:53:21.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.727+0000 7fdcb0348640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdca8101c80 0x7fdca819e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:53:21.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.727+0000 7fdcb0348640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca819eda0 con 0x7fdca81000d0 2024-02-18T11:53:21.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.727+0000 7fdcb0348640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca819ef10 con 0x7fdca81012c0 2024-02-18T11:53:21.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.727+0000 7fdcb0348640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca819f0d0 con 0x7fdca8101c80 2024-02-18T11:53:21.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.727+0000 7fdcae0e4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdca81000d0 0x7fdca81972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:53:21.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.727+0000 7fdcae0e4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdca81000d0 0x7fdca81972b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48976/0 (socket says 172.21.15.12:48976) 2024-02-18T11:53:21.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.727+0000 7fdcae0e4640 1 -- 172.21.15.12:0/1093617378 learned_addr learned my addr 172.21.15.12:0/1093617378 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:53:21.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.727+0000 7fdcad8e3640 1 --2- 172.21.15.12:0/1093617378 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdca81012c0 0x7fdca81977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:53:21.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.727+0000 7fdcae8e5640 1 --2- 172.21.15.12:0/1093617378 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdca8101c80 0x7fdca819e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:53:21.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.727+0000 7fdcae0e4640 1 -- 172.21.15.12:0/1093617378 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdca8101c80 msgr2=0x7fdca819e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:53:21.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.727+0000 7fdcae0e4640 1 --2- 172.21.15.12:0/1093617378 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdca8101c80 0x7fdca819e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:53:21.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.727+0000 7fdcae0e4640 1 -- 172.21.15.12:0/1093617378 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdca81012c0 msgr2=0x7fdca81977f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:53:21.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.727+0000 7fdcae0e4640 1 --2- 172.21.15.12:0/1093617378 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdca81012c0 0x7fdca81977f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:53:21.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.727+0000 7fdcae0e4640 1 -- 172.21.15.12:0/1093617378 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdca0009900 con 0x7fdca81000d0 2024-02-18T11:53:21.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.727+0000 7fdcae8e5640 1 --2- 172.21.15.12:0/1093617378 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdca8101c80 0x7fdca819e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:53:21.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.727+0000 7fdcae0e4640 1 --2- 172.21.15.12:0/1093617378 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdca81000d0 0x7fdca81972b0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fdca400b450 tx=0x7fdca400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:53:21.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.728+0000 7fdc9f7fe640 1 -- 172.21.15.12:0/1093617378 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca4011890 con 0x7fdca81000d0 2024-02-18T11:53:21.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.728+0000 7fdc9f7fe640 1 -- 172.21.15.12:0/1093617378 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdca40103e0 con 0x7fdca81000d0 2024-02-18T11:53:21.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.728+0000 7fdcb0348640 1 -- 172.21.15.12:0/1093617378 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdca819f3b0 con 0x7fdca81000d0 2024-02-18T11:53:21.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.728+0000 7fdc9f7fe640 1 -- 172.21.15.12:0/1093617378 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca400f580 con 0x7fdca81000d0 2024-02-18T11:53:21.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.728+0000 7fdcb0348640 1 -- 172.21.15.12:0/1093617378 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdca819f870 con 0x7fdca81000d0 2024-02-18T11:53:21.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.728+0000 7fdcb0348640 1 -- 172.21.15.12:0/1093617378 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdca8107470 con 0x7fdca81000d0 2024-02-18T11:53:21.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.730+0000 7fdc9f7fe640 1 -- 172.21.15.12:0/1093617378 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 20) v1 ==== 137245+0+0 (secure 0 0 0) 0x7fdca40108e0 con 0x7fdca81000d0 2024-02-18T11:53:21.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.730+0000 7fdc9f7fe640 1 --2- 172.21.15.12:0/1093617378 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fdc940a2750 0x7fdc940a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:53:21.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.730+0000 7fdc9f7fe640 1 -- 172.21.15.12:0/1093617378 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fdca40ce760 con 0x7fdca81000d0 2024-02-18T11:53:21.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.730+0000 7fdcad8e3640 1 --2- 172.21.15.12:0/1093617378 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fdc940a2750 0x7fdc940a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:53:21.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.731+0000 7fdcad8e3640 1 --2- 172.21.15.12:0/1093617378 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fdc940a2750 0x7fdc940a4c10 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fdca81987c0 tx=0x7fdc9800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:53:21.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.734+0000 7fdc9f7fe640 1 -- 172.21.15.12:0/1093617378 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdca4097500 con 0x7fdca81000d0 2024-02-18T11:53:21.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.914+0000 7fdcb0348640 1 -- 172.21.15.12:0/1093617378 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdca810cc40 con 0x7fdca81000d0 2024-02-18T11:53:21.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.915+0000 7fdc9f7fe640 1 -- 172.21.15.12:0/1093617378 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdca409abc0 con 0x7fdca81000d0 2024-02-18T11:53:21.908 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-02-18T11:53:21.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.917+0000 7fdcb0348640 1 -- 172.21.15.12:0/1093617378 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fdc940a2750 msgr2=0x7fdc940a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:53:21.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.917+0000 7fdcb0348640 1 --2- 172.21.15.12:0/1093617378 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fdc940a2750 0x7fdc940a4c10 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fdca81987c0 tx=0x7fdc9800b390 comp rx=0 tx=0).stop 2024-02-18T11:53:21.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.917+0000 7fdcb0348640 1 -- 172.21.15.12:0/1093617378 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdca81000d0 msgr2=0x7fdca81972b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:53:21.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.917+0000 7fdcb0348640 1 --2- 172.21.15.12:0/1093617378 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdca81000d0 0x7fdca81972b0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fdca400b450 tx=0x7fdca400b910 comp rx=0 tx=0).stop 2024-02-18T11:53:21.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.917+0000 7fdcb0348640 1 -- 172.21.15.12:0/1093617378 shutdown_connections 2024-02-18T11:53:21.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.917+0000 7fdcb0348640 1 --2- 172.21.15.12:0/1093617378 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fdc940a2750 0x7fdc940a4c10 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:53:21.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.917+0000 7fdcb0348640 1 --2- 172.21.15.12:0/1093617378 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdca8101c80 0x7fdca819e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:53:21.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.917+0000 7fdcb0348640 1 --2- 172.21.15.12:0/1093617378 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdca81012c0 0x7fdca81977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:53:21.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.917+0000 7fdcb0348640 1 --2- 172.21.15.12:0/1093617378 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdca81000d0 0x7fdca81972b0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:53:21.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.917+0000 7fdcb0348640 1 -- 172.21.15.12:0/1093617378 >> 172.21.15.12:0/1093617378 conn(0x7fdca80fb660 msgr2=0x7fdca8104640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:53:21.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.917+0000 7fdcb0348640 1 -- 172.21.15.12:0/1093617378 shutdown_connections 2024-02-18T11:53:21.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:21.917+0000 7fdcb0348640 1 -- 172.21.15.12:0/1093617378 wait complete. 2024-02-18T11:53:21.921 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-18T11:53:22.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.034+0000 7f1b78681640 1 -- 172.21.15.12:0/3715189702 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1b70101c50 msgr2=0x7f1b70104040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:53:22.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.034+0000 7f1b78681640 1 --2- 172.21.15.12:0/3715189702 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1b70101c50 0x7f1b70104040 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c009900 tx=0x7f1b6c02bc30 comp rx=0 tx=0).stop 2024-02-18T11:53:22.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.034+0000 7f1b78681640 1 -- 172.21.15.12:0/3715189702 shutdown_connections 2024-02-18T11:53:22.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.034+0000 7f1b78681640 1 --2- 172.21.15.12:0/3715189702 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b70104580 0x7f1b70106970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:53:22.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.034+0000 7f1b78681640 1 --2- 172.21.15.12:0/3715189702 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1b70106eb0 0x7f1b70109300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:53:22.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.034+0000 7f1b78681640 1 --2- 172.21.15.12:0/3715189702 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1b70101c50 0x7f1b70104040 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:53:22.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.034+0000 7f1b78681640 1 -- 172.21.15.12:0/3715189702 >> 172.21.15.12:0/3715189702 conn(0x7f1b700fb660 msgr2=0x7f1b700fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:53:22.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.034+0000 7f1b78681640 1 -- 172.21.15.12:0/3715189702 shutdown_connections 2024-02-18T11:53:22.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.034+0000 7f1b78681640 1 -- 172.21.15.12:0/3715189702 wait complete. 2024-02-18T11:53:22.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.035+0000 7f1b78681640 1 Processor -- start 2024-02-18T11:53:22.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.035+0000 7f1b78681640 1 -- start start 2024-02-18T11:53:22.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.035+0000 7f1b78681640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b70101c50 0x7f1b70197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:53:22.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.035+0000 7f1b78681640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1b70104580 0x7f1b70197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:53:22.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.035+0000 7f1b78681640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1b70106eb0 0x7f1b7019e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:53:22.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.035+0000 7f1b78681640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b7019ed30 con 0x7f1b70104580 2024-02-18T11:53:22.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.035+0000 7f1b78681640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b7019eea0 con 0x7f1b70106eb0 2024-02-18T11:53:22.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.035+0000 7f1b78681640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b7019f060 con 0x7f1b70101c50 2024-02-18T11:53:22.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.035+0000 7f1b75c1c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1b70104580 0x7f1b70197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:53:22.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.035+0000 7f1b75c1c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1b70104580 0x7f1b70197780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49000/0 (socket says 172.21.15.12:49000) 2024-02-18T11:53:22.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.035+0000 7f1b75c1c640 1 -- 172.21.15.12:0/1038782812 learned_addr learned my addr 172.21.15.12:0/1038782812 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:53:22.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.035+0000 7f1b7641d640 1 --2- 172.21.15.12:0/1038782812 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b70101c50 0x7f1b70197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:53:22.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.035+0000 7f1b76c1e640 1 --2- 172.21.15.12:0/1038782812 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1b70106eb0 0x7f1b7019e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:53:22.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.035+0000 7f1b75c1c640 1 -- 172.21.15.12:0/1038782812 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b70101c50 msgr2=0x7f1b70197240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:53:22.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.035+0000 7f1b75c1c640 1 --2- 172.21.15.12:0/1038782812 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b70101c50 0x7f1b70197240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:53:22.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.035+0000 7f1b75c1c640 1 -- 172.21.15.12:0/1038782812 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1b70106eb0 msgr2=0x7f1b7019e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:53:22.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.035+0000 7f1b75c1c640 1 --2- 172.21.15.12:0/1038782812 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1b70106eb0 0x7f1b7019e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:53:22.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.035+0000 7f1b75c1c640 1 -- 172.21.15.12:0/1038782812 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b6c009560 con 0x7f1b70104580 2024-02-18T11:53:22.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.035+0000 7f1b7641d640 1 --2- 172.21.15.12:0/1038782812 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b70101c50 0x7f1b70197240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:53:22.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.035+0000 7f1b76c1e640 1 --2- 172.21.15.12:0/1038782812 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1b70106eb0 0x7f1b7019e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:53:22.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.035+0000 7f1b75c1c640 1 --2- 172.21.15.12:0/1038782812 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1b70104580 0x7f1b70197780 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f1b6000c970 tx=0x7f1b6000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:53:22.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.035+0000 7f1b677fe640 1 -- 172.21.15.12:0/1038782812 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b60004440 con 0x7f1b70104580 2024-02-18T11:53:22.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.036+0000 7f1b78681640 1 -- 172.21.15.12:0/1038782812 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b7019f340 con 0x7f1b70104580 2024-02-18T11:53:22.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.036+0000 7f1b677fe640 1 -- 172.21.15.12:0/1038782812 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1b60007d40 con 0x7f1b70104580 2024-02-18T11:53:22.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.036+0000 7f1b677fe640 1 -- 172.21.15.12:0/1038782812 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b6000f450 con 0x7f1b70104580 2024-02-18T11:53:22.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.036+0000 7f1b78681640 1 -- 172.21.15.12:0/1038782812 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b7019f850 con 0x7f1b70104580 2024-02-18T11:53:22.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.037+0000 7f1b78681640 1 -- 172.21.15.12:0/1038782812 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b700fd150 con 0x7f1b70104580 2024-02-18T11:53:22.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.038+0000 7f1b677fe640 1 -- 172.21.15.12:0/1038782812 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 20) v1 ==== 137245+0+0 (secure 0 0 0) 0x7f1b60007860 con 0x7f1b70104580 2024-02-18T11:53:22.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.038+0000 7f1b677fe640 1 --2- 172.21.15.12:0/1038782812 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f1b5c0a2750 0x7f1b5c0a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:53:22.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.038+0000 7f1b677fe640 1 -- 172.21.15.12:0/1038782812 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1b600ce480 con 0x7f1b70104580 2024-02-18T11:53:22.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.038+0000 7f1b7641d640 1 --2- 172.21.15.12:0/1038782812 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f1b5c0a2750 0x7f1b5c0a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:53:22.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.039+0000 7f1b7641d640 1 --2- 172.21.15.12:0/1038782812 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f1b5c0a2750 0x7f1b5c0a4c10 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c000c00 tx=0x7f1b6c02c800 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:53:22.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.042+0000 7f1b677fe640 1 -- 172.21.15.12:0/1038782812 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1b60097220 con 0x7f1b70104580 2024-02-18T11:53:22.171 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:53:22 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/1218827753' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-18T11:53:22.171 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:53:22 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/1093617378' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-18T11:53:22.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.222+0000 7f1b78681640 1 -- 172.21.15.12:0/1038782812 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1b7002ce80 con 0x7f1b70104580 2024-02-18T11:53:22.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.222+0000 7f1b677fe640 1 -- 172.21.15.12:0/1038782812 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1b6009a8e0 con 0x7f1b70104580 2024-02-18T11:53:22.215 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-02-18T11:53:22.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.224+0000 7f1b78681640 1 -- 172.21.15.12:0/1038782812 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f1b5c0a2750 msgr2=0x7f1b5c0a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:53:22.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.224+0000 7f1b78681640 1 --2- 172.21.15.12:0/1038782812 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f1b5c0a2750 0x7f1b5c0a4c10 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c000c00 tx=0x7f1b6c02c800 comp rx=0 tx=0).stop 2024-02-18T11:53:22.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.224+0000 7f1b78681640 1 -- 172.21.15.12:0/1038782812 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1b70104580 msgr2=0x7f1b70197780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:53:22.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.224+0000 7f1b78681640 1 --2- 172.21.15.12:0/1038782812 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1b70104580 0x7f1b70197780 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f1b6000c970 tx=0x7f1b6000ce30 comp rx=0 tx=0).stop 2024-02-18T11:53:22.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.225+0000 7f1b78681640 1 -- 172.21.15.12:0/1038782812 shutdown_connections 2024-02-18T11:53:22.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.225+0000 7f1b78681640 1 --2- 172.21.15.12:0/1038782812 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f1b5c0a2750 0x7f1b5c0a4c10 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:53:22.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.225+0000 7f1b78681640 1 --2- 172.21.15.12:0/1038782812 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1b70106eb0 0x7f1b7019e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:53:22.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.225+0000 7f1b78681640 1 --2- 172.21.15.12:0/1038782812 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1b70104580 0x7f1b70197780 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:53:22.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.225+0000 7f1b78681640 1 --2- 172.21.15.12:0/1038782812 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b70101c50 0x7f1b70197240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:53:22.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.225+0000 7f1b78681640 1 -- 172.21.15.12:0/1038782812 >> 172.21.15.12:0/1038782812 conn(0x7f1b700fb660 msgr2=0x7f1b70102590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:53:22.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.225+0000 7f1b78681640 1 -- 172.21.15.12:0/1038782812 shutdown_connections 2024-02-18T11:53:22.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.225+0000 7f1b78681640 1 -- 172.21.15.12:0/1038782812 wait complete. 2024-02-18T11:53:22.229 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-18T11:53:22.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.341+0000 7f0eb937a640 1 -- 172.21.15.12:0/3494715543 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0eb4104570 msgr2=0x7f0eb4106960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:53:22.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.341+0000 7f0eb937a640 1 --2- 172.21.15.12:0/3494715543 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0eb4104570 0x7f0eb4106960 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c009900 tx=0x7f0e9c02bc30 comp rx=0 tx=0).stop 2024-02-18T11:53:22.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.341+0000 7f0eb937a640 1 -- 172.21.15.12:0/3494715543 shutdown_connections 2024-02-18T11:53:22.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.341+0000 7f0eb937a640 1 --2- 172.21.15.12:0/3494715543 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0eb4106ea0 0x7f0eb41092f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:53:22.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.341+0000 7f0eb937a640 1 --2- 172.21.15.12:0/3494715543 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0eb4104570 0x7f0eb4106960 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:53:22.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.341+0000 7f0eb937a640 1 --2- 172.21.15.12:0/3494715543 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0eb4101c40 0x7f0eb4104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:53:22.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.341+0000 7f0eb937a640 1 -- 172.21.15.12:0/3494715543 >> 172.21.15.12:0/3494715543 conn(0x7f0eb40fb650 msgr2=0x7f0eb40fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:53:22.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.341+0000 7f0eb937a640 1 -- 172.21.15.12:0/3494715543 shutdown_connections 2024-02-18T11:53:22.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.341+0000 7f0eb937a640 1 -- 172.21.15.12:0/3494715543 wait complete. 2024-02-18T11:53:22.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.342+0000 7f0eb937a640 1 Processor -- start 2024-02-18T11:53:22.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.342+0000 7f0eb937a640 1 -- start start 2024-02-18T11:53:22.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.342+0000 7f0eb937a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0eb4101c40 0x7f0eb4197220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:53:22.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.342+0000 7f0eb937a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0eb4104570 0x7f0eb4197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:53:22.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.342+0000 7f0eb937a640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0eb4106ea0 0x7f0eb419e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:53:22.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.342+0000 7f0eb937a640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb419ed10 con 0x7f0eb4101c40 2024-02-18T11:53:22.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.342+0000 7f0eb937a640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb419ee80 con 0x7f0eb4106ea0 2024-02-18T11:53:22.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.342+0000 7f0eb937a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb419f040 con 0x7f0eb4104570 2024-02-18T11:53:22.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.342+0000 7f0eb2ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0eb4101c40 0x7f0eb4197220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:53:22.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.342+0000 7f0eb2ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0eb4101c40 0x7f0eb4197220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49024/0 (socket says 172.21.15.12:49024) 2024-02-18T11:53:22.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.342+0000 7f0eb2ffd640 1 -- 172.21.15.12:0/3252204354 learned_addr learned my addr 172.21.15.12:0/3252204354 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:53:22.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.342+0000 7f0eb27fc640 1 --2- 172.21.15.12:0/3252204354 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0eb4104570 0x7f0eb4197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:53:22.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.343+0000 7f0eb2ffd640 1 -- 172.21.15.12:0/3252204354 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0eb4104570 msgr2=0x7f0eb4197760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:53:22.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.343+0000 7f0eb2ffd640 1 --2- 172.21.15.12:0/3252204354 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0eb4104570 0x7f0eb4197760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:53:22.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.343+0000 7f0eb2ffd640 1 -- 172.21.15.12:0/3252204354 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0eb4106ea0 msgr2=0x7f0eb419e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:53:22.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.343+0000 7f0eb37fe640 1 --2- 172.21.15.12:0/3252204354 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0eb4106ea0 0x7f0eb419e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:53:22.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.343+0000 7f0eb2ffd640 1 --2- 172.21.15.12:0/3252204354 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0eb4106ea0 0x7f0eb419e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:53:22.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.343+0000 7f0eb2ffd640 1 -- 172.21.15.12:0/3252204354 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e9c009560 con 0x7f0eb4101c40 2024-02-18T11:53:22.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.343+0000 7f0eb37fe640 1 --2- 172.21.15.12:0/3252204354 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0eb4106ea0 0x7f0eb419e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:53:22.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.343+0000 7f0eb2ffd640 1 --2- 172.21.15.12:0/3252204354 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0eb4101c40 0x7f0eb4197220 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f0eac00c920 tx=0x7f0eac00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:53:22.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.343+0000 7f0eabfff640 1 -- 172.21.15.12:0/3252204354 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eac007c70 con 0x7f0eb4101c40 2024-02-18T11:53:22.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.343+0000 7f0eabfff640 1 -- 172.21.15.12:0/3252204354 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0eac007dd0 con 0x7f0eb4101c40 2024-02-18T11:53:22.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.343+0000 7f0eb937a640 1 -- 172.21.15.12:0/3252204354 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0eb419f320 con 0x7f0eb4101c40 2024-02-18T11:53:22.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.343+0000 7f0eabfff640 1 -- 172.21.15.12:0/3252204354 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eac018860 con 0x7f0eb4101c40 2024-02-18T11:53:22.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.343+0000 7f0eb937a640 1 -- 172.21.15.12:0/3252204354 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0eb419f800 con 0x7f0eb4101c40 2024-02-18T11:53:22.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.344+0000 7f0eb937a640 1 -- 172.21.15.12:0/3252204354 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0eb40fd140 con 0x7f0eb4101c40 2024-02-18T11:53:22.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.345+0000 7f0eabfff640 1 -- 172.21.15.12:0/3252204354 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 20) v1 ==== 137245+0+0 (secure 0 0 0) 0x7f0eac00fac0 con 0x7f0eb4101c40 2024-02-18T11:53:22.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.346+0000 7f0eabfff640 1 --2- 172.21.15.12:0/3252204354 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f0ea00a2750 0x7f0ea00a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:53:22.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.346+0000 7f0eb27fc640 1 --2- 172.21.15.12:0/3252204354 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f0ea00a2750 0x7f0ea00a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:53:22.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.346+0000 7f0eabfff640 1 -- 172.21.15.12:0/3252204354 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f0eac0ce750 con 0x7f0eb4101c40 2024-02-18T11:53:22.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.346+0000 7f0eb27fc640 1 --2- 172.21.15.12:0/3252204354 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f0ea00a2750 0x7f0ea00a4c10 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c006010 tx=0x7f0e9c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:53:22.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.350+0000 7f0eabfff640 1 -- 172.21.15.12:0/3252204354 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0eac097490 con 0x7f0eb4101c40 2024-02-18T11:53:22.352 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:53:22 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/1218827753' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-18T11:53:22.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:53:22 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/1093617378' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-18T11:53:22.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:53:22 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/1218827753' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-18T11:53:22.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:53:22 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/1093617378' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-18T11:53:22.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.532+0000 7f0eb937a640 1 -- 172.21.15.12:0/3252204354 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0eb402ce80 con 0x7f0eb4101c40 2024-02-18T11:53:22.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.532+0000 7f0eabfff640 1 -- 172.21.15.12:0/3252204354 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0eac09ab50 con 0x7f0eb4101c40 2024-02-18T11:53:22.525 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2024-02-18T11:53:22.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.534+0000 7f0eb937a640 1 -- 172.21.15.12:0/3252204354 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f0ea00a2750 msgr2=0x7f0ea00a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:53:22.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.534+0000 7f0eb937a640 1 --2- 172.21.15.12:0/3252204354 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f0ea00a2750 0x7f0ea00a4c10 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c006010 tx=0x7f0e9c000bc0 comp rx=0 tx=0).stop 2024-02-18T11:53:22.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.534+0000 7f0eb937a640 1 -- 172.21.15.12:0/3252204354 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0eb4101c40 msgr2=0x7f0eb4197220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:53:22.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.534+0000 7f0eb937a640 1 --2- 172.21.15.12:0/3252204354 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0eb4101c40 0x7f0eb4197220 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f0eac00c920 tx=0x7f0eac00cde0 comp rx=0 tx=0).stop 2024-02-18T11:53:22.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.534+0000 7f0eb937a640 1 -- 172.21.15.12:0/3252204354 shutdown_connections 2024-02-18T11:53:22.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.534+0000 7f0eb937a640 1 --2- 172.21.15.12:0/3252204354 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f0ea00a2750 0x7f0ea00a4c10 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:53:22.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.534+0000 7f0eb937a640 1 --2- 172.21.15.12:0/3252204354 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0eb4104570 0x7f0eb4197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:53:22.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.534+0000 7f0eb937a640 1 --2- 172.21.15.12:0/3252204354 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0eb4106ea0 0x7f0eb419e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:53:22.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.534+0000 7f0eb937a640 1 --2- 172.21.15.12:0/3252204354 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0eb4101c40 0x7f0eb4197220 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:53:22.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.534+0000 7f0eb937a640 1 -- 172.21.15.12:0/3252204354 >> 172.21.15.12:0/3252204354 conn(0x7f0eb40fb650 msgr2=0x7f0eb4102580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:53:22.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.534+0000 7f0eb937a640 1 -- 172.21.15.12:0/3252204354 shutdown_connections 2024-02-18T11:53:22.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.534+0000 7f0eb937a640 1 -- 172.21.15.12:0/3252204354 wait complete. 2024-02-18T11:53:22.538 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-02-18T11:53:22.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.650+0000 7f227178a640 1 -- 172.21.15.12:0/1392830787 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f226c105d50 msgr2=0x7f226c108210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:53:22.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.650+0000 7f227178a640 1 --2- 172.21.15.12:0/1392830787 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f226c105d50 0x7f226c108210 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f2264009ca0 tx=0x7f226402c180 comp rx=0 tx=0).stop 2024-02-18T11:53:22.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.650+0000 7f227178a640 1 -- 172.21.15.12:0/1392830787 shutdown_connections 2024-02-18T11:53:22.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.650+0000 7f227178a640 1 --2- 172.21.15.12:0/1392830787 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f226c105d50 0x7f226c108210 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:53:22.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.650+0000 7f227178a640 1 --2- 172.21.15.12:0/1392830787 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f226c103250 0x7f226c1056c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:53:22.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.650+0000 7f227178a640 1 --2- 172.21.15.12:0/1392830787 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f226c100920 0x7f226c102d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:53:22.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.650+0000 7f227178a640 1 -- 172.21.15.12:0/1392830787 >> 172.21.15.12:0/1392830787 conn(0x7f226c0fa9c0 msgr2=0x7f226c0fcde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:53:22.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.650+0000 7f227178a640 1 -- 172.21.15.12:0/1392830787 shutdown_connections 2024-02-18T11:53:22.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.650+0000 7f227178a640 1 -- 172.21.15.12:0/1392830787 wait complete. 2024-02-18T11:53:22.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.650+0000 7f227178a640 1 Processor -- start 2024-02-18T11:53:22.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.650+0000 7f227178a640 1 -- start start 2024-02-18T11:53:22.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.651+0000 7f227178a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f226c100920 0x7f226c1001c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:53:22.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.651+0000 7f227178a640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f226c103250 0x7f226c19bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:53:22.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.651+0000 7f227178a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f226c105d50 0x7f226c19e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:53:22.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.651+0000 7f227178a640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f226c19ed40 con 0x7f226c100920 2024-02-18T11:53:22.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.651+0000 7f227178a640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f226c19eeb0 con 0x7f226c103250 2024-02-18T11:53:22.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.651+0000 7f227178a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f226c19f070 con 0x7f226c105d50 2024-02-18T11:53:22.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.651+0000 7f226affd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f226c100920 0x7f226c1001c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:53:22.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.651+0000 7f226affd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f226c100920 0x7f226c1001c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49044/0 (socket says 172.21.15.12:49044) 2024-02-18T11:53:22.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.651+0000 7f226affd640 1 -- 172.21.15.12:0/564011142 learned_addr learned my addr 172.21.15.12:0/564011142 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:53:22.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.651+0000 7f226a7fc640 1 --2- 172.21.15.12:0/564011142 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f226c103250 0x7f226c19bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:53:22.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.651+0000 7f226b7fe640 1 --2- 172.21.15.12:0/564011142 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f226c105d50 0x7f226c19e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:53:22.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.651+0000 7f226affd640 1 -- 172.21.15.12:0/564011142 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f226c105d50 msgr2=0x7f226c19e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:53:22.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.651+0000 7f226affd640 1 --2- 172.21.15.12:0/564011142 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f226c105d50 0x7f226c19e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:53:22.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.651+0000 7f226affd640 1 -- 172.21.15.12:0/564011142 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f226c103250 msgr2=0x7f226c19bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:53:22.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.651+0000 7f226affd640 1 --2- 172.21.15.12:0/564011142 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f226c103250 0x7f226c19bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:53:22.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.651+0000 7f226affd640 1 -- 172.21.15.12:0/564011142 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2264009900 con 0x7f226c100920 2024-02-18T11:53:22.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.651+0000 7f226b7fe640 1 --2- 172.21.15.12:0/564011142 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f226c105d50 0x7f226c19e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:53:22.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.651+0000 7f226affd640 1 --2- 172.21.15.12:0/564011142 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f226c100920 0x7f226c1001c0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f225c00c970 tx=0x7f225c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:53:22.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.651+0000 7f2253fff640 1 -- 172.21.15.12:0/564011142 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f225c007bb0 con 0x7f226c100920 2024-02-18T11:53:22.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.651+0000 7f2253fff640 1 -- 172.21.15.12:0/564011142 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f225c007d10 con 0x7f226c100920 2024-02-18T11:53:22.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.651+0000 7f227178a640 1 -- 172.21.15.12:0/564011142 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f226c19f350 con 0x7f226c100920 2024-02-18T11:53:22.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.651+0000 7f2253fff640 1 -- 172.21.15.12:0/564011142 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f225c00f450 con 0x7f226c100920 2024-02-18T11:53:22.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.651+0000 7f227178a640 1 -- 172.21.15.12:0/564011142 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f226c19f810 con 0x7f226c100920 2024-02-18T11:53:22.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.652+0000 7f227178a640 1 -- 172.21.15.12:0/564011142 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f226c0fc4b0 con 0x7f226c100920 2024-02-18T11:53:22.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.653+0000 7f2253fff640 1 -- 172.21.15.12:0/564011142 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 20) v1 ==== 137245+0+0 (secure 0 0 0) 0x7f225c004f70 con 0x7f226c100920 2024-02-18T11:53:22.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.654+0000 7f2253fff640 1 --2- 172.21.15.12:0/564011142 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f22540a2750 0x7f22540a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:53:22.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.654+0000 7f2253fff640 1 -- 172.21.15.12:0/564011142 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f225c0ce320 con 0x7f226c100920 2024-02-18T11:53:22.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.654+0000 7f226a7fc640 1 --2- 172.21.15.12:0/564011142 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f22540a2750 0x7f22540a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:53:22.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.654+0000 7f226a7fc640 1 --2- 172.21.15.12:0/564011142 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f22540a2750 0x7f22540a4c10 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f226c0fff30 tx=0x7f226000a4c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:53:22.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.658+0000 7f2253fff640 1 -- 172.21.15.12:0/564011142 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f225c0970c0 con 0x7f226c100920 2024-02-18T11:53:22.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.890+0000 7f227178a640 1 -- 172.21.15.12:0/564011142 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7f226c10baf0 con 0x7f226c100920 2024-02-18T11:53:22.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.898+0000 7f2253fff640 1 -- 172.21.15.12:0/564011142 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v20) v1 ==== 81+0+738341 (secure 0 0 0) 0x7f225c085020 con 0x7f226c100920 2024-02-18T11:53:22.902 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:53:22.902 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-02-18T11:53:22.902 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 20, 2024-02-18T11:53:22.902 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:22.902 INFO:teuthology.orchestra.run.smithi012.stdout: "active_gid": 14152, 2024-02-18T11:53:22.902 INFO:teuthology.orchestra.run.smithi012.stdout: "active_name": "x", 2024-02-18T11:53:22.902 INFO:teuthology.orchestra.run.smithi012.stdout: "active_addrs": { 2024-02-18T11:53:22.902 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-02-18T11:53:22.902 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:22.902 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-02-18T11:53:22.902 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6800", 2024-02-18T11:53:22.902 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1153736685 2024-02-18T11:53:22.902 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.903 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:22.903 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-02-18T11:53:22.903 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6801", 2024-02-18T11:53:22.903 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1153736685 2024-02-18T11:53:22.903 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:22.903 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-02-18T11:53:22.903 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.903 INFO:teuthology.orchestra.run.smithi012.stdout: "active_addr": "172.21.15.12:6801/1153736685", 2024-02-18T11:53:22.903 INFO:teuthology.orchestra.run.smithi012.stdout: "active_change": "2024-02-18T11:31:50.937042+0000", 2024-02-18T11:53:22.903 INFO:teuthology.orchestra.run.smithi012.stdout: "active_mgr_features": 4540701547738038271, 2024-02-18T11:53:22.903 INFO:teuthology.orchestra.run.smithi012.stdout: "available": true, 2024-02-18T11:53:22.903 INFO:teuthology.orchestra.run.smithi012.stdout: "standbys": [ 2024-02-18T11:53:22.903 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:22.903 INFO:teuthology.orchestra.run.smithi012.stdout: "gid": 24106, 2024-02-18T11:53:22.903 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "z", 2024-02-18T11:53:22.903 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr_features": 4540701547738038271, 2024-02-18T11:53:22.904 INFO:teuthology.orchestra.run.smithi012.stdout: "available_modules": [ 2024-02-18T11:53:22.904 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:22.904 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "alerts", 2024-02-18T11:53:22.904 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:22.904 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:22.904 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:22.904 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-02-18T11:53:22.904 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-02-18T11:53:22.904 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-02-18T11:53:22.904 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.904 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:22.904 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-02-18T11:53:22.904 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.904 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.905 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.905 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How frequently to reexamine health status", 2024-02-18T11:53:22.905 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.905 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.905 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.905 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.905 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:22.905 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:22.905 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:22.905 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.905 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:22.905 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:22.905 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.905 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.905 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:22.905 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:22.906 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:22.906 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:22.906 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:22.906 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:22.906 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:22.906 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:22.906 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:22.906 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.906 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.906 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.906 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.906 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:22.906 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:22.906 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:22.907 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.907 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:22.907 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:22.907 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.907 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.907 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.907 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:22.907 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.907 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.907 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.907 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.907 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:22.908 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:22.908 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:22.908 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.908 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:22.908 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:22.908 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.908 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.908 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:22.908 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:22.908 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:22.908 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:22.908 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:22.908 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:22.908 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:22.908 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:22.909 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:22.909 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.909 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.909 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.909 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.909 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:22.909 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:22.909 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:22.909 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.909 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:22.909 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:22.909 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.909 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.909 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.909 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:22.910 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.910 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.910 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.910 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.910 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_destination": { 2024-02-18T11:53:22.910 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_destination", 2024-02-18T11:53:22.910 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:22.910 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.910 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:22.910 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:22.910 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.910 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.910 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.910 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Email address to send alerts to", 2024-02-18T11:53:22.910 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.911 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.911 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.911 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.911 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_from_name": { 2024-02-18T11:53:22.911 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_from_name", 2024-02-18T11:53:22.911 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:22.911 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.911 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:22.911 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "Ceph", 2024-02-18T11:53:22.911 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.911 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.911 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.911 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Email From: name", 2024-02-18T11:53:22.911 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.911 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.911 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.912 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.912 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_host": { 2024-02-18T11:53:22.912 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_host", 2024-02-18T11:53:22.912 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:22.912 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.912 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:22.912 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:22.912 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.912 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.912 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.912 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SMTP server", 2024-02-18T11:53:22.912 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.912 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.912 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.912 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.913 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_password": { 2024-02-18T11:53:22.913 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_password", 2024-02-18T11:53:22.913 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:22.913 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.913 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:22.913 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:22.913 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.913 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.913 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.913 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Password to authenticate with", 2024-02-18T11:53:22.913 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.913 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.913 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.913 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.913 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_port": { 2024-02-18T11:53:22.913 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_port", 2024-02-18T11:53:22.914 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:22.914 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.914 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:22.914 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "465", 2024-02-18T11:53:22.914 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.914 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.914 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.914 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SMTP port", 2024-02-18T11:53:22.914 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.914 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.914 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.914 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.914 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_sender": { 2024-02-18T11:53:22.914 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_sender", 2024-02-18T11:53:22.914 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:22.915 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.915 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:22.915 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:22.915 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.915 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.915 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.915 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SMTP envelope sender", 2024-02-18T11:53:22.915 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.915 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.915 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.915 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.915 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_ssl": { 2024-02-18T11:53:22.915 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_ssl", 2024-02-18T11:53:22.915 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:22.916 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.916 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:22.916 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:22.916 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.916 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.916 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.916 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use SSL to connect to SMTP server", 2024-02-18T11:53:22.916 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.916 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.916 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.916 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.916 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_user": { 2024-02-18T11:53:22.916 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_user", 2024-02-18T11:53:22.916 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:22.917 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.917 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:22.917 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:22.917 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.917 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.917 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.917 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "User to authenticate as", 2024-02-18T11:53:22.917 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.917 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.917 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.917 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:22.917 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:22.917 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.917 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:22.917 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "balancer", 2024-02-18T11:53:22.918 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:22.918 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:22.918 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:22.918 INFO:teuthology.orchestra.run.smithi012.stdout: "active": { 2024-02-18T11:53:22.918 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "active", 2024-02-18T11:53:22.918 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:22.918 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.918 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:22.918 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:22.918 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.918 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.918 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.918 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "automatically balance PGs across cluster", 2024-02-18T11:53:22.918 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.918 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.919 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.919 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.919 INFO:teuthology.orchestra.run.smithi012.stdout: "begin_time": { 2024-02-18T11:53:22.919 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "begin_time", 2024-02-18T11:53:22.919 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:22.919 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.919 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:22.919 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0000", 2024-02-18T11:53:22.919 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.919 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.919 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.919 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "beginning time of day to automatically balance", 2024-02-18T11:53:22.919 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-02-18T11:53:22.919 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.919 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.920 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.920 INFO:teuthology.orchestra.run.smithi012.stdout: "begin_weekday": { 2024-02-18T11:53:22.920 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "begin_weekday", 2024-02-18T11:53:22.920 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-02-18T11:53:22.920 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.920 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:22.920 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-02-18T11:53:22.920 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "0", 2024-02-18T11:53:22.920 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "6", 2024-02-18T11:53:22.920 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.920 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-02-18T11:53:22.920 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-02-18T11:53:22.920 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.920 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.921 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.921 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_compat_max_iterations": { 2024-02-18T11:53:22.921 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crush_compat_max_iterations", 2024-02-18T11:53:22.921 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-02-18T11:53:22.921 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.921 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:22.921 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "25", 2024-02-18T11:53:22.921 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-02-18T11:53:22.921 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "250", 2024-02-18T11:53:22.921 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.921 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-02-18T11:53:22.921 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.921 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.921 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.921 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.922 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_compat_metrics": { 2024-02-18T11:53:22.922 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crush_compat_metrics", 2024-02-18T11:53:22.922 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:22.922 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.922 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:22.922 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "pgs,objects,bytes", 2024-02-18T11:53:22.922 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.922 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.922 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.922 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-02-18T11:53:22.922 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-02-18T11:53:22.922 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.922 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.922 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.922 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_compat_step": { 2024-02-18T11:53:22.922 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crush_compat_step", 2024-02-18T11:53:22.923 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-02-18T11:53:22.923 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.923 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:22.923 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0.5", 2024-02-18T11:53:22.923 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "0.001", 2024-02-18T11:53:22.923 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "0.999", 2024-02-18T11:53:22.923 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.923 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "aggressiveness of optimization", 2024-02-18T11:53:22.923 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-02-18T11:53:22.923 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.923 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.923 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.923 INFO:teuthology.orchestra.run.smithi012.stdout: "end_time": { 2024-02-18T11:53:22.923 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "end_time", 2024-02-18T11:53:22.923 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:22.924 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.924 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:22.924 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "2359", 2024-02-18T11:53:22.924 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.924 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.924 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.924 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "ending time of day to automatically balance", 2024-02-18T11:53:22.924 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-02-18T11:53:22.924 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.924 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.924 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.924 INFO:teuthology.orchestra.run.smithi012.stdout: "end_weekday": { 2024-02-18T11:53:22.924 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "end_weekday", 2024-02-18T11:53:22.924 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-02-18T11:53:22.924 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.924 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:22.925 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-02-18T11:53:22.925 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "0", 2024-02-18T11:53:22.925 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "6", 2024-02-18T11:53:22.925 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.925 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-02-18T11:53:22.925 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-02-18T11:53:22.925 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.925 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.925 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.925 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:22.925 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:22.925 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:22.925 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.925 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:22.925 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:22.926 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.926 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.926 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:22.926 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:22.926 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:22.926 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:22.926 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:22.926 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:22.926 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:22.926 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:22.926 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:22.926 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.926 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.926 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.926 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.927 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:22.927 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:22.927 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:22.927 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.927 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:22.927 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:22.927 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.927 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.927 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.927 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:22.927 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.927 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.928 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.928 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.928 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:22.928 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:22.928 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:22.928 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.928 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:22.928 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:22.928 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.928 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.928 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:22.928 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:22.928 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:22.928 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:22.929 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:22.929 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:22.929 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:22.929 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:22.929 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:22.929 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.929 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.929 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.929 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.929 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:22.929 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:22.929 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:22.929 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.929 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:22.929 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:22.929 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.930 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.930 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.930 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:22.930 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.930 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.930 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.930 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.930 INFO:teuthology.orchestra.run.smithi012.stdout: "min_score": { 2024-02-18T11:53:22.930 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "min_score", 2024-02-18T11:53:22.930 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-02-18T11:53:22.930 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.930 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:22.930 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-02-18T11:53:22.930 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.930 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.931 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.931 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-02-18T11:53:22.931 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.931 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.931 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.931 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.931 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": { 2024-02-18T11:53:22.931 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mode", 2024-02-18T11:53:22.931 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:22.931 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.931 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:22.931 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "upmap", 2024-02-18T11:53:22.931 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.931 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.931 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:22.931 INFO:teuthology.orchestra.run.smithi012.stdout: "crush-compat", 2024-02-18T11:53:22.932 INFO:teuthology.orchestra.run.smithi012.stdout: "none", 2024-02-18T11:53:22.932 INFO:teuthology.orchestra.run.smithi012.stdout: "read", 2024-02-18T11:53:22.932 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap", 2024-02-18T11:53:22.932 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap-read" 2024-02-18T11:53:22.932 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:22.932 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Balancer mode", 2024-02-18T11:53:22.932 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.932 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.932 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.932 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.932 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_ids": { 2024-02-18T11:53:22.932 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pool_ids", 2024-02-18T11:53:22.932 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:22.932 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.933 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:22.933 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:22.933 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.933 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.933 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.933 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-02-18T11:53:22.933 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.933 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.933 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.933 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.933 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-02-18T11:53:22.933 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-02-18T11:53:22.933 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-02-18T11:53:22.933 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.933 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:22.934 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-02-18T11:53:22.934 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.934 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.934 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.934 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-02-18T11:53:22.934 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.934 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.934 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.934 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.934 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap_max_deviation": { 2024-02-18T11:53:22.934 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "upmap_max_deviation", 2024-02-18T11:53:22.934 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:22.934 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.934 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:22.934 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-02-18T11:53:22.934 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-02-18T11:53:22.935 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.935 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.935 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "deviation below which no optimization is attempted", 2024-02-18T11:53:22.935 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-02-18T11:53:22.935 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.935 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.935 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.935 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap_max_optimizations": { 2024-02-18T11:53:22.935 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "upmap_max_optimizations", 2024-02-18T11:53:22.935 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-02-18T11:53:22.935 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.935 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:22.935 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-02-18T11:53:22.935 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.935 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.935 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.936 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-02-18T11:53:22.936 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.936 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.936 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.936 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:22.936 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:22.936 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.936 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:22.936 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cephadm", 2024-02-18T11:53:22.936 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:22.936 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:22.936 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:22.936 INFO:teuthology.orchestra.run.smithi012.stdout: "agent_down_multiplier": { 2024-02-18T11:53:22.936 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "agent_down_multiplier", 2024-02-18T11:53:22.936 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-02-18T11:53:22.937 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.937 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:22.937 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "3.0", 2024-02-18T11:53:22.937 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.937 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.937 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.937 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-02-18T11:53:22.937 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.937 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.937 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.937 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.937 INFO:teuthology.orchestra.run.smithi012.stdout: "agent_refresh_rate": { 2024-02-18T11:53:22.937 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "agent_refresh_rate", 2024-02-18T11:53:22.937 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-02-18T11:53:22.937 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.937 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:22.938 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "20", 2024-02-18T11:53:22.938 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.938 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.938 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.938 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-02-18T11:53:22.938 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.938 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.938 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.938 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.938 INFO:teuthology.orchestra.run.smithi012.stdout: "agent_starting_port": { 2024-02-18T11:53:22.938 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "agent_starting_port", 2024-02-18T11:53:22.938 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:22.938 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.938 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:22.939 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "4721", 2024-02-18T11:53:22.939 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.939 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.939 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.939 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-02-18T11:53:22.939 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.939 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.939 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.939 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.939 INFO:teuthology.orchestra.run.smithi012.stdout: "allow_ptrace": { 2024-02-18T11:53:22.939 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "allow_ptrace", 2024-02-18T11:53:22.939 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:22.939 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.940 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:22.940 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:22.940 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.940 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.940 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.940 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-02-18T11:53:22.940 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-02-18T11:53:22.940 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.940 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.940 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.941 INFO:teuthology.orchestra.run.smithi012.stdout: "autotune_interval": { 2024-02-18T11:53:22.941 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "autotune_interval", 2024-02-18T11:53:22.941 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-02-18T11:53:22.941 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.941 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:22.941 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-02-18T11:53:22.941 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.941 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.941 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.941 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to autotune daemon memory", 2024-02-18T11:53:22.941 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.941 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.941 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.941 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.941 INFO:teuthology.orchestra.run.smithi012.stdout: "autotune_memory_target_ratio": { 2024-02-18T11:53:22.942 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "autotune_memory_target_ratio", 2024-02-18T11:53:22.942 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-02-18T11:53:22.942 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.942 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:22.942 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0.7", 2024-02-18T11:53:22.942 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.942 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.942 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.942 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-02-18T11:53:22.942 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.942 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.943 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.943 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.943 INFO:teuthology.orchestra.run.smithi012.stdout: "cephadm_log_destination": { 2024-02-18T11:53:22.943 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cephadm_log_destination", 2024-02-18T11:53:22.943 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:22.943 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.943 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:22.943 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:22.943 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.944 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.944 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:22.944 INFO:teuthology.orchestra.run.smithi012.stdout: "file", 2024-02-18T11:53:22.944 INFO:teuthology.orchestra.run.smithi012.stdout: "file,syslog", 2024-02-18T11:53:22.944 INFO:teuthology.orchestra.run.smithi012.stdout: "syslog" 2024-02-18T11:53:22.944 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:22.944 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-02-18T11:53:22.944 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.944 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.944 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.944 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.944 INFO:teuthology.orchestra.run.smithi012.stdout: "cgroups_split": { 2024-02-18T11:53:22.944 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cgroups_split", 2024-02-18T11:53:22.944 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:22.944 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.945 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:22.945 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:22.945 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.945 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.945 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.945 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-02-18T11:53:22.945 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.945 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.945 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.945 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.945 INFO:teuthology.orchestra.run.smithi012.stdout: "config_checks_enabled": { 2024-02-18T11:53:22.945 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "config_checks_enabled", 2024-02-18T11:53:22.945 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:22.945 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.945 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:22.946 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:22.946 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.946 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.946 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.946 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-02-18T11:53:22.946 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.946 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.946 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.946 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.946 INFO:teuthology.orchestra.run.smithi012.stdout: "config_dashboard": { 2024-02-18T11:53:22.946 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "config_dashboard", 2024-02-18T11:53:22.946 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:22.946 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.946 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:22.946 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:22.947 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.947 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.947 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.947 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-02-18T11:53:22.947 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.947 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.947 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.947 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.947 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_alertmanager": { 2024-02-18T11:53:22.947 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_alertmanager", 2024-02-18T11:53:22.947 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:22.947 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.947 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:22.948 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-02-18T11:53:22.948 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.948 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.948 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.948 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-02-18T11:53:22.948 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.948 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.948 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.948 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.948 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_base": { 2024-02-18T11:53:22.948 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_base", 2024-02-18T11:53:22.948 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:22.948 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.948 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:22.948 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/ceph", 2024-02-18T11:53:22.949 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.949 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.949 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.949 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Container image name, without the tag", 2024-02-18T11:53:22.949 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.949 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.949 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.949 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.949 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_elasticsearch": { 2024-02-18T11:53:22.949 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_elasticsearch", 2024-02-18T11:53:22.949 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:22.949 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.949 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:22.949 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-02-18T11:53:22.949 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.949 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.950 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.950 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "elasticsearch container image", 2024-02-18T11:53:22.950 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.950 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.950 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.950 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.950 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_grafana": { 2024-02-18T11:53:22.950 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_grafana", 2024-02-18T11:53:22.950 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:22.950 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.950 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:22.950 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-02-18T11:53:22.950 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.950 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.950 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.951 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-02-18T11:53:22.951 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.951 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.951 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.951 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.951 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_haproxy": { 2024-02-18T11:53:22.951 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_haproxy", 2024-02-18T11:53:22.951 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:22.951 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.951 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:22.951 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-02-18T11:53:22.951 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.951 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.952 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.952 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "HAproxy container image", 2024-02-18T11:53:22.952 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.952 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.952 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.952 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.952 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_jaeger_agent": { 2024-02-18T11:53:22.952 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_jaeger_agent", 2024-02-18T11:53:22.952 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:22.952 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.952 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:22.952 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-02-18T11:53:22.952 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.953 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.953 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.953 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Jaeger agent container image", 2024-02-18T11:53:22.953 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.953 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.953 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.953 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.953 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_jaeger_collector": { 2024-02-18T11:53:22.953 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_jaeger_collector", 2024-02-18T11:53:22.953 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:22.953 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.953 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:22.953 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-02-18T11:53:22.953 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.953 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.954 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.954 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Jaeger collector container image", 2024-02-18T11:53:22.954 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.954 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.954 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.954 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.954 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_jaeger_query": { 2024-02-18T11:53:22.954 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_jaeger_query", 2024-02-18T11:53:22.954 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:22.954 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.954 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:22.954 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-02-18T11:53:22.954 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.954 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.954 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.954 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Jaeger query container image", 2024-02-18T11:53:22.955 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.955 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.955 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.955 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.955 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_keepalived": { 2024-02-18T11:53:22.955 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_keepalived", 2024-02-18T11:53:22.955 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:22.955 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.955 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:22.955 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-02-18T11:53:22.955 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.956 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.956 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.956 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Keepalived container image", 2024-02-18T11:53:22.956 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.956 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.956 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.956 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.956 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_loki": { 2024-02-18T11:53:22.956 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_loki", 2024-02-18T11:53:22.956 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:22.957 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.957 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:22.957 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-02-18T11:53:22.957 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.957 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.957 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.957 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Loki container image", 2024-02-18T11:53:22.957 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.957 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.957 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.957 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.957 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_node_exporter": { 2024-02-18T11:53:22.957 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_node_exporter", 2024-02-18T11:53:22.957 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:22.957 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.957 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:22.959 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-02-18T11:53:22.959 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.959 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.960 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.960 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-02-18T11:53:22.960 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.960 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.960 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.960 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.960 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_nvmeof": { 2024-02-18T11:53:22.960 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_nvmeof", 2024-02-18T11:53:22.960 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:22.960 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.960 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:22.960 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2024-02-18T11:53:22.960 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.960 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.960 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.961 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Nvme-of container image", 2024-02-18T11:53:22.961 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.961 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.961 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.961 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.961 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_prometheus": { 2024-02-18T11:53:22.961 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_prometheus", 2024-02-18T11:53:22.961 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:22.961 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.961 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:22.961 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-02-18T11:53:22.961 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.961 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.961 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.961 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-02-18T11:53:22.961 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.962 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.962 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.962 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.962 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_promtail": { 2024-02-18T11:53:22.962 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_promtail", 2024-02-18T11:53:22.962 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:22.962 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.962 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:22.962 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-02-18T11:53:22.962 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.962 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.962 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.962 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Promtail container image", 2024-02-18T11:53:22.962 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.962 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.963 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.963 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.963 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_snmp_gateway": { 2024-02-18T11:53:22.963 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_snmp_gateway", 2024-02-18T11:53:22.963 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:22.963 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.963 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:22.963 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-02-18T11:53:22.963 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.963 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.963 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.963 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SNMP Gateway container image", 2024-02-18T11:53:22.963 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.963 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.963 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.963 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.964 INFO:teuthology.orchestra.run.smithi012.stdout: "container_init": { 2024-02-18T11:53:22.964 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_init", 2024-02-18T11:53:22.964 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:22.964 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.964 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:22.964 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:22.964 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.964 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.964 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.964 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Run podman/docker with `--init`", 2024-02-18T11:53:22.964 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.964 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.964 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.964 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.964 INFO:teuthology.orchestra.run.smithi012.stdout: "daemon_cache_timeout": { 2024-02-18T11:53:22.965 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "daemon_cache_timeout", 2024-02-18T11:53:22.965 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-02-18T11:53:22.965 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.965 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:22.965 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-02-18T11:53:22.965 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.965 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.965 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.965 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-02-18T11:53:22.965 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.965 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.965 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.965 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.966 INFO:teuthology.orchestra.run.smithi012.stdout: "default_cephadm_command_timeout": { 2024-02-18T11:53:22.966 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "default_cephadm_command_timeout", 2024-02-18T11:53:22.966 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:22.966 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.966 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:22.966 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "900", 2024-02-18T11:53:22.966 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.966 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.966 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.966 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-02-18T11:53:22.966 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.966 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.966 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.966 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.966 INFO:teuthology.orchestra.run.smithi012.stdout: "default_registry": { 2024-02-18T11:53:22.967 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "default_registry", 2024-02-18T11:53:22.967 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:22.967 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.967 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:22.967 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io", 2024-02-18T11:53:22.967 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.967 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.967 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.967 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-02-18T11:53:22.967 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.967 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.967 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.967 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.967 INFO:teuthology.orchestra.run.smithi012.stdout: "device_cache_timeout": { 2024-02-18T11:53:22.967 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "device_cache_timeout", 2024-02-18T11:53:22.968 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-02-18T11:53:22.968 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.968 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:22.968 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "1800", 2024-02-18T11:53:22.968 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.968 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.968 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.968 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "seconds to cache device inventory", 2024-02-18T11:53:22.968 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.968 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.968 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.968 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.968 INFO:teuthology.orchestra.run.smithi012.stdout: "device_enhanced_scan": { 2024-02-18T11:53:22.968 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "device_enhanced_scan", 2024-02-18T11:53:22.968 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:22.969 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.969 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:22.969 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:22.969 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.969 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.969 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.969 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use libstoragemgmt during device scans", 2024-02-18T11:53:22.969 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.969 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.969 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.969 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.969 INFO:teuthology.orchestra.run.smithi012.stdout: "facts_cache_timeout": { 2024-02-18T11:53:22.969 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "facts_cache_timeout", 2024-02-18T11:53:22.969 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-02-18T11:53:22.969 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.970 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:22.970 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-02-18T11:53:22.970 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.970 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.970 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.970 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "seconds to cache host facts data", 2024-02-18T11:53:22.970 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.970 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.970 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.970 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.970 INFO:teuthology.orchestra.run.smithi012.stdout: "host_check_interval": { 2024-02-18T11:53:22.970 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "host_check_interval", 2024-02-18T11:53:22.970 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-02-18T11:53:22.970 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.970 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:22.971 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-02-18T11:53:22.971 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.971 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.971 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.971 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to perform a host check", 2024-02-18T11:53:22.971 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.971 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.971 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.971 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.971 INFO:teuthology.orchestra.run.smithi012.stdout: "hw_monitoring": { 2024-02-18T11:53:22.971 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "hw_monitoring", 2024-02-18T11:53:22.971 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:22.971 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.971 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:22.971 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:22.971 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.972 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.972 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.972 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-02-18T11:53:22.972 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.972 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.972 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.972 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.972 INFO:teuthology.orchestra.run.smithi012.stdout: "inventory_list_all": { 2024-02-18T11:53:22.972 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "inventory_list_all", 2024-02-18T11:53:22.972 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:22.972 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.972 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:22.972 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:22.972 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.972 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.973 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.973 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-02-18T11:53:22.973 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.973 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.973 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.973 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.973 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:22.973 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:22.973 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:22.973 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.973 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:22.973 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:22.973 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.973 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.973 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:22.973 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:22.974 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:22.974 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:22.974 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:22.974 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:22.974 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:22.974 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:22.974 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:22.974 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.974 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.974 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.974 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.974 INFO:teuthology.orchestra.run.smithi012.stdout: "log_refresh_metadata": { 2024-02-18T11:53:22.974 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_refresh_metadata", 2024-02-18T11:53:22.974 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:22.974 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.974 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:22.975 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:22.975 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.975 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.975 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.975 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-02-18T11:53:22.975 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.975 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.975 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.975 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.975 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:22.975 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:22.975 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:22.975 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.975 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:22.975 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:22.976 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.976 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.976 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.976 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-02-18T11:53:22.976 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.976 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.976 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.976 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.976 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:22.976 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:22.976 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:22.976 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.976 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:22.976 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:22.977 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.977 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.977 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:22.977 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:22.977 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:22.977 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:22.977 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:22.977 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:22.977 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:22.977 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:22.977 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:22.977 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.977 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.977 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.977 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.978 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:22.978 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:22.978 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:22.978 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.978 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:22.978 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:22.978 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.978 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.978 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.978 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:22.978 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.978 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.978 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.978 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.978 INFO:teuthology.orchestra.run.smithi012.stdout: "manage_etc_ceph_ceph_conf": { 2024-02-18T11:53:22.979 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-02-18T11:53:22.979 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:22.979 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.979 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:22.979 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:22.979 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.979 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.979 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.979 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-02-18T11:53:22.979 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.979 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.979 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.979 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.979 INFO:teuthology.orchestra.run.smithi012.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-02-18T11:53:22.979 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-02-18T11:53:22.979 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:22.980 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.980 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:22.980 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "*", 2024-02-18T11:53:22.980 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.980 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.980 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.980 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-02-18T11:53:22.980 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.980 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.980 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.980 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.980 INFO:teuthology.orchestra.run.smithi012.stdout: "max_count_per_host": { 2024-02-18T11:53:22.980 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_count_per_host", 2024-02-18T11:53:22.980 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:22.981 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.981 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:22.981 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-02-18T11:53:22.981 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.981 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.981 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.981 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "max number of daemons per service per host", 2024-02-18T11:53:22.981 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.981 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.981 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.981 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.981 INFO:teuthology.orchestra.run.smithi012.stdout: "max_osd_draining_count": { 2024-02-18T11:53:22.981 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_osd_draining_count", 2024-02-18T11:53:22.981 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:22.981 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.982 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:22.982 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-02-18T11:53:22.982 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.982 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.982 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.982 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-02-18T11:53:22.982 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.982 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.982 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.982 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.982 INFO:teuthology.orchestra.run.smithi012.stdout: "migration_current": { 2024-02-18T11:53:22.982 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "migration_current", 2024-02-18T11:53:22.982 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:22.982 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.982 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:22.982 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:22.983 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.983 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.983 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.983 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "internal - do not modify", 2024-02-18T11:53:22.983 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.983 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.983 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.983 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.983 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": { 2024-02-18T11:53:22.983 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mode", 2024-02-18T11:53:22.983 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:22.983 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.983 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:22.983 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "root", 2024-02-18T11:53:22.983 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.984 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.984 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:22.984 INFO:teuthology.orchestra.run.smithi012.stdout: "cephadm-package", 2024-02-18T11:53:22.984 INFO:teuthology.orchestra.run.smithi012.stdout: "root" 2024-02-18T11:53:22.984 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:22.984 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "mode for remote execution of cephadm", 2024-02-18T11:53:22.984 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.984 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.984 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.984 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.984 INFO:teuthology.orchestra.run.smithi012.stdout: "oob_default_addr": { 2024-02-18T11:53:22.984 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "oob_default_addr", 2024-02-18T11:53:22.984 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:22.984 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.984 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:22.985 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "169.254.1.1", 2024-02-18T11:53:22.985 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.985 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.985 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.985 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Default address for RedFish API (oob management).", 2024-02-18T11:53:22.985 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.985 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.985 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.985 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.985 INFO:teuthology.orchestra.run.smithi012.stdout: "prometheus_alerts_path": { 2024-02-18T11:53:22.985 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "prometheus_alerts_path", 2024-02-18T11:53:22.985 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:22.985 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.985 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:22.985 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-02-18T11:53:22.985 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.986 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.986 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.986 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-02-18T11:53:22.986 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.986 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.986 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.986 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.986 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_insecure": { 2024-02-18T11:53:22.986 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_insecure", 2024-02-18T11:53:22.986 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:22.986 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.986 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:22.986 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:22.986 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.986 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.987 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.987 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-02-18T11:53:22.987 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.987 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.987 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.987 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.987 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_password": { 2024-02-18T11:53:22.987 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_password", 2024-02-18T11:53:22.987 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:22.987 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.987 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:22.987 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:22.987 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.987 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.987 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.987 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-02-18T11:53:22.988 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.988 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.988 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.988 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.988 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_url": { 2024-02-18T11:53:22.988 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_url", 2024-02-18T11:53:22.988 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:22.988 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.988 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:22.988 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:22.988 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.988 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.988 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.989 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-02-18T11:53:22.989 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.989 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.989 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.989 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.989 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_username": { 2024-02-18T11:53:22.989 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_username", 2024-02-18T11:53:22.989 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:22.989 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.989 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:22.989 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:22.989 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.989 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.989 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.989 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-02-18T11:53:22.990 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.990 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.990 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.990 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.990 INFO:teuthology.orchestra.run.smithi012.stdout: "secure_monitoring_stack": { 2024-02-18T11:53:22.990 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "secure_monitoring_stack", 2024-02-18T11:53:22.990 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:22.990 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.990 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:22.990 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:22.990 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.990 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.990 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.990 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-02-18T11:53:22.990 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.991 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.991 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.991 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.991 INFO:teuthology.orchestra.run.smithi012.stdout: "service_discovery_port": { 2024-02-18T11:53:22.991 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "service_discovery_port", 2024-02-18T11:53:22.991 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:22.991 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.991 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:22.991 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8765", 2024-02-18T11:53:22.991 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.991 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.991 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.991 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "cephadm service discovery port", 2024-02-18T11:53:22.991 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.991 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.991 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.992 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.992 INFO:teuthology.orchestra.run.smithi012.stdout: "ssh_config_file": { 2024-02-18T11:53:22.992 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssh_config_file", 2024-02-18T11:53:22.992 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:22.992 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.992 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:22.992 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:22.992 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.992 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.992 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.992 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-02-18T11:53:22.992 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.992 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.992 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.993 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.993 INFO:teuthology.orchestra.run.smithi012.stdout: "use_agent": { 2024-02-18T11:53:22.993 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "use_agent", 2024-02-18T11:53:22.993 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:22.993 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.993 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:22.993 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:22.993 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.993 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.993 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.993 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-02-18T11:53:22.993 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.993 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.993 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.993 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.994 INFO:teuthology.orchestra.run.smithi012.stdout: "use_repo_digest": { 2024-02-18T11:53:22.994 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "use_repo_digest", 2024-02-18T11:53:22.994 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:22.994 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.994 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:22.994 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:22.994 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.994 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.994 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.994 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-02-18T11:53:22.994 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.994 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.994 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.994 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.994 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_on_failed_host_check": { 2024-02-18T11:53:22.995 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_on_failed_host_check", 2024-02-18T11:53:22.995 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:22.995 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.995 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:22.995 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:22.995 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.995 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.995 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.995 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise a health warning if the host check fails", 2024-02-18T11:53:22.995 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.995 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.995 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.995 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.995 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_on_stray_daemons": { 2024-02-18T11:53:22.995 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_on_stray_daemons", 2024-02-18T11:53:22.995 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:22.996 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.996 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:22.996 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:22.996 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.996 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.996 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.996 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-02-18T11:53:22.996 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.996 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.996 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.996 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.996 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_on_stray_hosts": { 2024-02-18T11:53:22.996 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_on_stray_hosts", 2024-02-18T11:53:22.996 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:22.996 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.997 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:22.997 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:22.997 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.997 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.997 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:22.997 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-02-18T11:53:22.997 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.997 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.997 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.997 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:22.997 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:22.997 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.997 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:22.997 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crash", 2024-02-18T11:53:22.997 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:22.997 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:22.998 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:22.998 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:22.998 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:22.998 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:22.998 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.998 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:22.998 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:22.998 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.998 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.998 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:22.998 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:22.998 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:22.998 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:22.998 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:22.998 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:22.998 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:22.999 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:22.999 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:22.999 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:22.999 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:22.999 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:22.999 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:22.999 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:22.999 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:22.999 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:22.999 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:22.999 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:22.999 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:22.999 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:22.999 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:22.999 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.000 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.000 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.000 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.000 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.000 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.000 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.000 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.000 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.000 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.000 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.000 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.000 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.000 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.000 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.001 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.001 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.001 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.001 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.001 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.001 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.001 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.001 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.001 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.001 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.001 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.002 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.002 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.002 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.002 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.002 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.002 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.002 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.002 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.002 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.002 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.002 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.002 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.002 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.002 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.002 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.003 INFO:teuthology.orchestra.run.smithi012.stdout: "retain_interval": { 2024-02-18T11:53:23.003 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "retain_interval", 2024-02-18T11:53:23.003 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-02-18T11:53:23.003 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.003 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.003 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "31536000", 2024-02-18T11:53:23.003 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.003 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.003 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.003 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how long to retain crashes before pruning them", 2024-02-18T11:53:23.003 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.003 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.003 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.003 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.003 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_recent_interval": { 2024-02-18T11:53:23.004 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_recent_interval", 2024-02-18T11:53:23.004 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-02-18T11:53:23.004 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.004 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.004 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "1209600", 2024-02-18T11:53:23.004 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.004 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.004 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.004 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "time interval in which to warn about recent crashes", 2024-02-18T11:53:23.004 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.004 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.004 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.004 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.004 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.004 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.005 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.005 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "dashboard", 2024-02-18T11:53:23.005 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.005 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.005 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.005 INFO:teuthology.orchestra.run.smithi012.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-02-18T11:53:23.005 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-02-18T11:53:23.005 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.005 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.005 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.005 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-02-18T11:53:23.005 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.005 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.005 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.005 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.006 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.006 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.006 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.006 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.006 INFO:teuthology.orchestra.run.smithi012.stdout: "ALERTMANAGER_API_HOST": { 2024-02-18T11:53:23.006 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ALERTMANAGER_API_HOST", 2024-02-18T11:53:23.006 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.006 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.006 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.006 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.006 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.006 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.006 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.006 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.006 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.006 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.007 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.007 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.007 INFO:teuthology.orchestra.run.smithi012.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-02-18T11:53:23.007 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-02-18T11:53:23.007 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.007 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.007 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.007 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.007 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.007 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.007 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.007 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.007 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.007 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.007 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.008 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.008 INFO:teuthology.orchestra.run.smithi012.stdout: "AUDIT_API_ENABLED": { 2024-02-18T11:53:23.008 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "AUDIT_API_ENABLED", 2024-02-18T11:53:23.008 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.008 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.008 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.008 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.008 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.008 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.008 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.008 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.008 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.008 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.008 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.008 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.009 INFO:teuthology.orchestra.run.smithi012.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-02-18T11:53:23.009 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-02-18T11:53:23.009 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.009 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.009 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.009 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.009 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.009 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.009 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.009 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.009 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.009 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.009 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.009 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.009 INFO:teuthology.orchestra.run.smithi012.stdout: "ENABLE_BROWSABLE_API": { 2024-02-18T11:53:23.009 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ENABLE_BROWSABLE_API", 2024-02-18T11:53:23.010 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.010 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.010 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.010 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.010 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.010 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.010 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.010 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.010 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.010 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.010 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.010 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.010 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-02-18T11:53:23.010 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-02-18T11:53:23.010 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.010 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.011 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.011 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.011 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.011 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.011 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.011 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.011 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.011 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.011 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.011 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.011 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-02-18T11:53:23.011 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-02-18T11:53:23.011 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.011 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.011 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.011 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.012 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.012 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.012 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.012 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.012 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.012 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.012 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.012 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.012 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-02-18T11:53:23.012 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-02-18T11:53:23.012 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.012 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.012 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.013 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.013 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.013 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.013 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.013 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.013 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.013 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.013 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.013 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.013 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-02-18T11:53:23.013 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-02-18T11:53:23.013 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.013 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.013 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.013 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.014 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.014 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.014 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.014 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.014 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.014 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.014 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.014 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.014 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_NFS": { 2024-02-18T11:53:23.014 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-02-18T11:53:23.014 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.014 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.014 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.014 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.015 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.015 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.015 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.015 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.015 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.015 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.015 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.015 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.015 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_RBD": { 2024-02-18T11:53:23.015 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-02-18T11:53:23.015 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.015 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.015 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.015 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.015 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.015 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.016 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.016 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.016 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.016 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.016 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.016 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.016 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_RGW": { 2024-02-18T11:53:23.016 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-02-18T11:53:23.016 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.016 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.016 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.016 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.016 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.016 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.016 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.017 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.017 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.017 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.017 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.017 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.017 INFO:teuthology.orchestra.run.smithi012.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-02-18T11:53:23.017 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-02-18T11:53:23.017 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.017 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.017 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.017 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.017 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.017 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.017 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.017 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.018 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.018 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.018 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.018 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.018 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_PASSWORD": { 2024-02-18T11:53:23.018 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_PASSWORD", 2024-02-18T11:53:23.018 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.018 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.018 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.018 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "admin", 2024-02-18T11:53:23.018 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.018 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.018 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.018 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.018 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.019 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.019 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.019 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.019 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-02-18T11:53:23.019 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-02-18T11:53:23.019 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.019 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.019 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.019 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.019 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.019 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.019 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.019 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.019 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.019 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.020 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.020 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.020 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_URL": { 2024-02-18T11:53:23.020 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_URL", 2024-02-18T11:53:23.020 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.020 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.020 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.020 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.020 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.020 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.020 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.020 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.020 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.020 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.020 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.021 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.021 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_USERNAME": { 2024-02-18T11:53:23.021 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_USERNAME", 2024-02-18T11:53:23.021 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.021 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.021 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.021 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "admin", 2024-02-18T11:53:23.021 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.021 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.021 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.021 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.021 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.021 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.021 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.021 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.021 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-02-18T11:53:23.022 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-02-18T11:53:23.022 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.022 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.022 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.022 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.022 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.022 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.022 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.022 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.022 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.022 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.022 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.022 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.022 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-02-18T11:53:23.022 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-02-18T11:53:23.023 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.023 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.023 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.023 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.023 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.023 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.023 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.023 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.023 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.023 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.023 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.023 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.023 INFO:teuthology.orchestra.run.smithi012.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-02-18T11:53:23.023 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-02-18T11:53:23.023 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.024 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.024 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.024 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.024 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.024 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.024 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.024 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.024 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.024 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.024 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.024 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.024 INFO:teuthology.orchestra.run.smithi012.stdout: "ISSUE_TRACKER_API_KEY": { 2024-02-18T11:53:23.024 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-02-18T11:53:23.024 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.024 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.025 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.025 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.025 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.025 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.025 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.025 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.025 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.025 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.025 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.025 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.025 INFO:teuthology.orchestra.run.smithi012.stdout: "PROMETHEUS_API_HOST": { 2024-02-18T11:53:23.025 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PROMETHEUS_API_HOST", 2024-02-18T11:53:23.025 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.025 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.026 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.026 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.026 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.026 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.026 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.026 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.026 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.026 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.026 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.026 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.026 INFO:teuthology.orchestra.run.smithi012.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-02-18T11:53:23.026 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-02-18T11:53:23.026 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.026 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.026 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.027 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.027 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.027 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.027 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.027 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.027 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.027 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.027 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.027 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.027 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-02-18T11:53:23.027 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-02-18T11:53:23.027 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.027 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.027 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.027 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.028 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.028 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.028 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.028 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.028 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.028 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.028 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.028 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.028 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-02-18T11:53:23.028 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-02-18T11:53:23.028 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.028 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.028 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.028 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.029 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.029 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.029 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.029 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.029 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.029 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.029 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.029 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.029 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-02-18T11:53:23.029 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-02-18T11:53:23.029 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.029 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.029 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.029 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.029 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.029 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.030 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.030 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.030 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.030 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.030 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.030 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.030 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-02-18T11:53:23.030 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-02-18T11:53:23.030 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.030 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.030 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.030 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.030 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.030 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.030 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.031 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.031 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.031 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.031 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.031 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.031 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-02-18T11:53:23.031 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-02-18T11:53:23.031 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.031 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.031 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.031 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.031 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.031 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.031 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.031 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.031 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.032 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.032 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.032 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.032 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-02-18T11:53:23.032 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-02-18T11:53:23.032 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.032 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.032 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.032 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.032 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.032 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.032 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.032 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.032 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.032 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.033 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.033 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.033 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-02-18T11:53:23.033 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-02-18T11:53:23.033 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.033 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.033 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.033 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.033 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.033 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.033 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.033 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.033 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.033 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.033 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.034 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.034 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_ENABLED": { 2024-02-18T11:53:23.034 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_ENABLED", 2024-02-18T11:53:23.034 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.034 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.034 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.034 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.034 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.034 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.034 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.034 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.034 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.034 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.034 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.034 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.034 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-02-18T11:53:23.035 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-02-18T11:53:23.035 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.035 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.035 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.035 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-02-18T11:53:23.035 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.035 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.035 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.035 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.035 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.035 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.035 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.035 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.035 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-02-18T11:53:23.035 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-02-18T11:53:23.036 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.036 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.036 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.036 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-02-18T11:53:23.036 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.036 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.036 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.036 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.036 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.036 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.036 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.036 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.036 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-02-18T11:53:23.036 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-02-18T11:53:23.037 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.037 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.037 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.037 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8", 2024-02-18T11:53:23.037 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.037 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.037 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.037 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.037 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.037 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.037 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.037 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.037 INFO:teuthology.orchestra.run.smithi012.stdout: "REST_REQUESTS_TIMEOUT": { 2024-02-18T11:53:23.037 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-02-18T11:53:23.037 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.038 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.038 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.038 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "45", 2024-02-18T11:53:23.038 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.038 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.038 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.038 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.038 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.038 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.038 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.038 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.038 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_ACCESS_KEY": { 2024-02-18T11:53:23.038 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_ACCESS_KEY", 2024-02-18T11:53:23.038 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.038 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.039 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.039 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.039 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.039 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.039 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.039 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.039 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.039 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.039 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.039 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.039 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-02-18T11:53:23.039 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-02-18T11:53:23.039 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.039 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.039 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.040 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "admin", 2024-02-18T11:53:23.040 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.040 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.040 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.040 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.040 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.040 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.040 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.040 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.040 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_SECRET_KEY": { 2024-02-18T11:53:23.040 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_SECRET_KEY", 2024-02-18T11:53:23.040 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.040 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.040 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.040 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.041 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.041 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.041 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.041 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.041 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.041 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.041 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.041 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.041 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_SSL_VERIFY": { 2024-02-18T11:53:23.041 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_SSL_VERIFY", 2024-02-18T11:53:23.041 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.041 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.041 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.041 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.041 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.042 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.042 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.042 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.042 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.042 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.042 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.042 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.042 INFO:teuthology.orchestra.run.smithi012.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-02-18T11:53:23.042 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-02-18T11:53:23.042 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.042 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.042 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.042 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-02-18T11:53:23.042 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.042 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.042 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.043 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.043 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.043 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.043 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.043 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.043 INFO:teuthology.orchestra.run.smithi012.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-02-18T11:53:23.043 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-02-18T11:53:23.043 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.043 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.043 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.044 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-02-18T11:53:23.044 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.044 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.044 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.044 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.044 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.044 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.044 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.044 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.044 INFO:teuthology.orchestra.run.smithi012.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-02-18T11:53:23.044 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-02-18T11:53:23.044 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.044 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.044 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.044 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-02-18T11:53:23.045 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.045 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.045 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.045 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.045 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.045 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.045 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.045 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.045 INFO:teuthology.orchestra.run.smithi012.stdout: "cross_origin_url": { 2024-02-18T11:53:23.045 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cross_origin_url", 2024-02-18T11:53:23.045 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.045 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.045 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.045 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.045 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.045 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.046 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.046 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.046 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.046 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.046 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.046 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.046 INFO:teuthology.orchestra.run.smithi012.stdout: "crt_file": { 2024-02-18T11:53:23.046 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crt_file", 2024-02-18T11:53:23.046 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.046 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.046 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.046 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.046 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.046 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.046 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.047 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.047 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.047 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.047 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.047 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.047 INFO:teuthology.orchestra.run.smithi012.stdout: "debug": { 2024-02-18T11:53:23.047 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "debug", 2024-02-18T11:53:23.047 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.047 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.047 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.047 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.047 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.047 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.047 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.047 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Enable/disable debug options", 2024-02-18T11:53:23.047 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.048 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.048 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.048 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.048 INFO:teuthology.orchestra.run.smithi012.stdout: "jwt_token_ttl": { 2024-02-18T11:53:23.048 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "jwt_token_ttl", 2024-02-18T11:53:23.048 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.048 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.048 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.048 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "28800", 2024-02-18T11:53:23.048 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.048 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.048 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.048 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.048 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.048 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.049 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.049 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.049 INFO:teuthology.orchestra.run.smithi012.stdout: "key_file": { 2024-02-18T11:53:23.049 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "key_file", 2024-02-18T11:53:23.049 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.049 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.049 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.049 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.049 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.049 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.049 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.049 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.049 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.049 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.050 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.050 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.050 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.050 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.050 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.050 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.050 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.050 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.050 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.050 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.050 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.050 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.050 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.050 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.050 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.051 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.051 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.051 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.051 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.051 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.051 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.051 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.051 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.051 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.051 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.051 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.051 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.051 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.051 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.051 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.052 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.052 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.052 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.052 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.052 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.052 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.052 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.052 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.052 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.052 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.052 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.052 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.052 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.052 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.052 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.053 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.053 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.053 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.053 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.053 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.053 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.053 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.053 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.053 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.053 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.053 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.053 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.053 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.053 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.053 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.053 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.054 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.054 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.054 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.054 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.054 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.054 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.054 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.054 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.054 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.054 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.054 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.054 INFO:teuthology.orchestra.run.smithi012.stdout: "motd": { 2024-02-18T11:53:23.054 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "motd", 2024-02-18T11:53:23.054 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.054 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.055 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.055 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.055 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.055 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.055 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.055 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "The message of the day", 2024-02-18T11:53:23.055 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.055 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.055 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.055 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.055 INFO:teuthology.orchestra.run.smithi012.stdout: "redirect_resolve_ip_addr": { 2024-02-18T11:53:23.055 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "redirect_resolve_ip_addr", 2024-02-18T11:53:23.055 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.055 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.055 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.055 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.056 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.056 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.056 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.056 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.056 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.056 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.056 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.056 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.056 INFO:teuthology.orchestra.run.smithi012.stdout: "server_addr": { 2024-02-18T11:53:23.056 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_addr", 2024-02-18T11:53:23.056 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.056 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.056 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.056 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "::", 2024-02-18T11:53:23.056 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.057 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.057 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.057 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.057 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.057 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.057 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.057 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.057 INFO:teuthology.orchestra.run.smithi012.stdout: "server_port": { 2024-02-18T11:53:23.057 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_port", 2024-02-18T11:53:23.057 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.057 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.057 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.057 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8080", 2024-02-18T11:53:23.057 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.057 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.057 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.058 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.058 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.058 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.058 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.058 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.058 INFO:teuthology.orchestra.run.smithi012.stdout: "ssl": { 2024-02-18T11:53:23.058 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssl", 2024-02-18T11:53:23.058 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.058 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.058 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.058 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.058 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.058 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.058 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.058 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.059 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.059 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.059 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.059 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.059 INFO:teuthology.orchestra.run.smithi012.stdout: "ssl_server_port": { 2024-02-18T11:53:23.059 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssl_server_port", 2024-02-18T11:53:23.059 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.059 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.059 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.059 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8443", 2024-02-18T11:53:23.059 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.059 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.059 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.059 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.059 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.060 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.060 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.060 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.060 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_behaviour": { 2024-02-18T11:53:23.060 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_behaviour", 2024-02-18T11:53:23.060 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.060 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.060 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.060 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "redirect", 2024-02-18T11:53:23.060 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.060 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.060 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.060 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.060 INFO:teuthology.orchestra.run.smithi012.stdout: "redirect" 2024-02-18T11:53:23.060 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.061 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.061 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.061 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.061 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.061 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.061 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_error_status_code": { 2024-02-18T11:53:23.061 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_error_status_code", 2024-02-18T11:53:23.061 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.061 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.061 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.061 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "500", 2024-02-18T11:53:23.061 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "400", 2024-02-18T11:53:23.061 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "599", 2024-02-18T11:53:23.061 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.061 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.061 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.062 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.062 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.062 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.062 INFO:teuthology.orchestra.run.smithi012.stdout: "url_prefix": { 2024-02-18T11:53:23.062 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "url_prefix", 2024-02-18T11:53:23.062 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.062 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.062 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.062 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.062 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.062 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.062 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.062 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.063 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.063 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.063 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.063 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.063 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.063 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.063 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.063 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "devicehealth", 2024-02-18T11:53:23.063 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.063 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.063 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.063 INFO:teuthology.orchestra.run.smithi012.stdout: "enable_monitoring": { 2024-02-18T11:53:23.063 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enable_monitoring", 2024-02-18T11:53:23.063 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.063 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.064 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.064 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.064 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.064 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.064 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.064 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "monitor device health metrics", 2024-02-18T11:53:23.064 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.064 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.064 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.064 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.064 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.064 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.064 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.064 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.064 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.065 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.065 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.065 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.065 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.065 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.065 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.065 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.065 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.065 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.065 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.065 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.065 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.065 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.065 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.065 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.066 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.066 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.066 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.066 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.066 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.066 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.066 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.066 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.066 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.066 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.066 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.066 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.066 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.066 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.066 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.066 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.067 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.067 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.067 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.067 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.067 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.067 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.067 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.067 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.067 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.067 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.067 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.067 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.067 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.067 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.067 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.068 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.068 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.068 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.068 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.068 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.068 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.068 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.068 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.068 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.068 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.068 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.068 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.068 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.068 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.068 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.069 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.069 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.069 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.069 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.069 INFO:teuthology.orchestra.run.smithi012.stdout: "mark_out_threshold": { 2024-02-18T11:53:23.069 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mark_out_threshold", 2024-02-18T11:53:23.069 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-02-18T11:53:23.069 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.069 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.069 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "2419200", 2024-02-18T11:53:23.069 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.069 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.069 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.069 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-02-18T11:53:23.069 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.070 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.070 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.070 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.070 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_name": { 2024-02-18T11:53:23.070 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pool_name", 2024-02-18T11:53:23.070 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.070 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.070 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.070 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "device_health_metrics", 2024-02-18T11:53:23.070 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.070 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.070 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.070 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "name of pool in which to store device health metrics", 2024-02-18T11:53:23.070 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.070 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.071 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.071 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.071 INFO:teuthology.orchestra.run.smithi012.stdout: "retention_period": { 2024-02-18T11:53:23.071 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "retention_period", 2024-02-18T11:53:23.071 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-02-18T11:53:23.071 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.071 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.071 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "15552000", 2024-02-18T11:53:23.071 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.071 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.071 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.071 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how long to retain device health metrics", 2024-02-18T11:53:23.071 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.071 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.071 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.071 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.072 INFO:teuthology.orchestra.run.smithi012.stdout: "scrape_frequency": { 2024-02-18T11:53:23.072 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "scrape_frequency", 2024-02-18T11:53:23.072 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-02-18T11:53:23.072 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.072 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.072 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "86400", 2024-02-18T11:53:23.072 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.072 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.072 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.072 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to scrape device health metrics", 2024-02-18T11:53:23.072 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.072 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.072 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.072 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.072 INFO:teuthology.orchestra.run.smithi012.stdout: "self_heal": { 2024-02-18T11:53:23.073 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "self_heal", 2024-02-18T11:53:23.073 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.073 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.073 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.073 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.073 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.073 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.073 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.073 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-02-18T11:53:23.073 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.073 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.073 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.073 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.073 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-02-18T11:53:23.073 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-02-18T11:53:23.074 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-02-18T11:53:23.074 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.074 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.074 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-02-18T11:53:23.074 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.074 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.074 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.074 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to wake up and check device health", 2024-02-18T11:53:23.074 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.074 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.074 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.074 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.074 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_threshold": { 2024-02-18T11:53:23.074 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_threshold", 2024-02-18T11:53:23.075 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-02-18T11:53:23.075 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.075 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.075 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "7257600", 2024-02-18T11:53:23.075 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.075 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.075 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.075 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-02-18T11:53:23.075 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.075 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.075 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.075 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.075 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.075 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.075 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.076 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "diskprediction_local", 2024-02-18T11:53:23.076 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.076 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.076 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.076 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.076 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.076 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.076 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.076 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.076 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.076 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.076 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.076 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.076 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.076 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.077 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.077 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.077 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.077 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.077 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.077 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.077 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.077 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.077 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.077 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.077 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.077 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.077 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.077 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.077 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.077 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.078 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.078 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.078 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.078 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.078 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.078 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.078 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.078 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.078 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.078 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.078 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.078 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.078 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.078 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.078 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.079 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.079 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.079 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.079 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.079 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.079 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.079 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.079 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.079 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.079 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.079 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.079 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.079 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.079 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.079 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.080 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.080 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.080 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.080 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.080 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.080 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.080 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.080 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.080 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.080 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.080 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.080 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.080 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.080 INFO:teuthology.orchestra.run.smithi012.stdout: "predict_interval": { 2024-02-18T11:53:23.080 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "predict_interval", 2024-02-18T11:53:23.081 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.081 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.081 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.081 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "86400", 2024-02-18T11:53:23.081 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.081 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.081 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.081 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.081 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.081 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.081 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.081 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.081 INFO:teuthology.orchestra.run.smithi012.stdout: "predictor_model": { 2024-02-18T11:53:23.081 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "predictor_model", 2024-02-18T11:53:23.081 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.082 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.082 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.082 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "prophetstor", 2024-02-18T11:53:23.082 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.082 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.082 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.082 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.082 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.082 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.082 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.082 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.082 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-02-18T11:53:23.082 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-02-18T11:53:23.082 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.082 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.082 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.083 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-02-18T11:53:23.083 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.083 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.083 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.083 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.083 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.083 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.083 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.083 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.083 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.083 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.083 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.083 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "influx", 2024-02-18T11:53:23.083 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": false, 2024-02-18T11:53:23.083 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "influxdb python module not found", 2024-02-18T11:53:23.084 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.084 INFO:teuthology.orchestra.run.smithi012.stdout: "batch_size": { 2024-02-18T11:53:23.084 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "batch_size", 2024-02-18T11:53:23.084 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.084 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.084 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.084 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5000", 2024-02-18T11:53:23.084 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.084 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.084 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.084 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-02-18T11:53:23.084 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.084 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.084 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.084 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.085 INFO:teuthology.orchestra.run.smithi012.stdout: "database": { 2024-02-18T11:53:23.085 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "database", 2024-02-18T11:53:23.085 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.085 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.085 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.085 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "ceph", 2024-02-18T11:53:23.085 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.085 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.085 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.085 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-02-18T11:53:23.085 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.085 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.085 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.085 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.085 INFO:teuthology.orchestra.run.smithi012.stdout: "hostname": { 2024-02-18T11:53:23.086 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "hostname", 2024-02-18T11:53:23.086 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.086 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.086 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.086 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.086 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.086 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.086 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.086 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "InfluxDB server hostname", 2024-02-18T11:53:23.086 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.086 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.086 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.086 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.086 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-02-18T11:53:23.087 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-02-18T11:53:23.087 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-02-18T11:53:23.087 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.087 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.087 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "30", 2024-02-18T11:53:23.087 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "5", 2024-02-18T11:53:23.087 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.087 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.087 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-02-18T11:53:23.087 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.087 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.087 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.087 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.087 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.087 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.088 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.088 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.088 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.088 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.088 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.088 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.088 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.088 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.088 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.088 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.088 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.088 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.088 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.088 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.089 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.089 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.089 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.089 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.089 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.089 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.089 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.089 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.089 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.089 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.089 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.089 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.089 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.089 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.089 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.089 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.090 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.090 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.090 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.090 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.090 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.090 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.090 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.090 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.090 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.090 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.090 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.090 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.090 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.090 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.091 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.091 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.091 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.091 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.091 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.091 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.091 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.091 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.091 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.091 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.091 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.091 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.091 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.091 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.091 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.091 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.092 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.092 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.092 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.092 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.092 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.092 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.092 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.092 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.092 INFO:teuthology.orchestra.run.smithi012.stdout: "password": { 2024-02-18T11:53:23.092 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "password", 2024-02-18T11:53:23.092 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.092 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.092 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.092 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.092 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.093 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.093 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.093 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "password of InfluxDB server user", 2024-02-18T11:53:23.093 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.093 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.093 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.093 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.093 INFO:teuthology.orchestra.run.smithi012.stdout: "port": { 2024-02-18T11:53:23.093 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "port", 2024-02-18T11:53:23.093 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.093 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.093 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.093 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8086", 2024-02-18T11:53:23.093 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.093 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.094 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.094 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "InfluxDB server port", 2024-02-18T11:53:23.094 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.094 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.094 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.094 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.094 INFO:teuthology.orchestra.run.smithi012.stdout: "ssl": { 2024-02-18T11:53:23.094 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssl", 2024-02-18T11:53:23.094 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.094 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.094 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.094 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "false", 2024-02-18T11:53:23.094 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.094 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.094 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.094 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-02-18T11:53:23.095 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.095 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.095 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.095 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.095 INFO:teuthology.orchestra.run.smithi012.stdout: "threads": { 2024-02-18T11:53:23.095 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "threads", 2024-02-18T11:53:23.095 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.095 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.095 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.095 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-02-18T11:53:23.095 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-02-18T11:53:23.095 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "32", 2024-02-18T11:53:23.095 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.095 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-02-18T11:53:23.095 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.095 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.096 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.096 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.096 INFO:teuthology.orchestra.run.smithi012.stdout: "username": { 2024-02-18T11:53:23.096 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "username", 2024-02-18T11:53:23.096 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.096 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.096 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.096 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.096 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.096 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.096 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.096 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "username of InfluxDB server user", 2024-02-18T11:53:23.096 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.096 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.096 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.097 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.097 INFO:teuthology.orchestra.run.smithi012.stdout: "verify_ssl": { 2024-02-18T11:53:23.097 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "verify_ssl", 2024-02-18T11:53:23.097 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.097 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.097 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.097 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "true", 2024-02-18T11:53:23.097 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.097 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.097 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.097 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-02-18T11:53:23.097 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.097 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.097 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.097 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.097 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.098 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.098 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.098 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "insights", 2024-02-18T11:53:23.098 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.098 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.098 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.098 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.098 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.098 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.098 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.098 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.098 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.098 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.098 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.098 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.099 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.099 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.099 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.099 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.099 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.099 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.099 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.099 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.099 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.099 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.099 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.099 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.099 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.099 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.100 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.100 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.100 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.100 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.100 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.100 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.100 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.100 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.100 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.100 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.100 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.100 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.100 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.100 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.100 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.101 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.101 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.101 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.101 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.101 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.101 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.101 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.101 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.101 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.101 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.101 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.101 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.101 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.101 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.101 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.102 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.102 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.102 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.102 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.102 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.102 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.102 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.102 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.102 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.102 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.102 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.102 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.102 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.102 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.102 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.102 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.103 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.103 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.103 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.103 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.103 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "iostat", 2024-02-18T11:53:23.103 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.103 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.103 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.103 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.103 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.103 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.103 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.103 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.103 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.103 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.104 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.104 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.104 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.104 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.104 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.104 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.104 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.104 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.104 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.104 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.104 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.104 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.104 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.104 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.104 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.105 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.105 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.105 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.105 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.105 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.105 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.105 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.105 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.105 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.105 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.105 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.105 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.105 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.105 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.105 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.106 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.106 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.106 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.106 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.106 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.106 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.106 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.106 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.106 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.106 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.106 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.106 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.106 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.106 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.106 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.106 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.107 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.107 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.107 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.107 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.107 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.107 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.107 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.107 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.107 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.107 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.107 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.107 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.107 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.107 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.107 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.108 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.108 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.108 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.108 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.108 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.108 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "k8sevents", 2024-02-18T11:53:23.108 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.108 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.108 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.108 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_event_retention_days": { 2024-02-18T11:53:23.108 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ceph_event_retention_days", 2024-02-18T11:53:23.108 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.108 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.108 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.108 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "7", 2024-02-18T11:53:23.109 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.109 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.109 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.109 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Days to hold ceph event information within local cache", 2024-02-18T11:53:23.109 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.109 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.109 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.109 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.109 INFO:teuthology.orchestra.run.smithi012.stdout: "config_check_secs": { 2024-02-18T11:53:23.109 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "config_check_secs", 2024-02-18T11:53:23.109 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.109 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.109 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.109 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-02-18T11:53:23.109 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "10", 2024-02-18T11:53:23.109 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.110 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.110 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-02-18T11:53:23.110 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.110 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.110 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.110 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.110 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.110 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.110 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.110 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.110 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.110 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.110 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.110 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.110 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.111 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.111 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.111 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.111 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.111 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.111 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.111 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.111 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.111 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.111 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.111 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.111 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.111 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.111 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.111 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.111 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.112 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.112 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.112 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.112 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.112 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.112 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.112 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.112 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.112 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.112 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.112 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.112 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.112 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.113 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.113 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.113 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.113 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.113 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.113 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.113 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.113 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.113 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.113 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.113 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.113 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.113 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.113 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.113 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.113 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.114 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.114 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.114 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.114 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.114 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.114 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.114 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.114 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.114 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.114 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.114 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.114 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.114 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.114 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.114 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.114 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.115 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.115 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.115 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.115 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "localpool", 2024-02-18T11:53:23.115 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.115 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.115 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.115 INFO:teuthology.orchestra.run.smithi012.stdout: "failure_domain": { 2024-02-18T11:53:23.115 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "failure_domain", 2024-02-18T11:53:23.115 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.115 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.115 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.115 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "host", 2024-02-18T11:53:23.115 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.115 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.116 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.116 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "failure domain for any created local pool", 2024-02-18T11:53:23.116 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-02-18T11:53:23.116 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.116 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.116 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.116 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.116 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.116 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.116 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.116 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.116 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.116 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.116 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.116 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.117 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.117 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.117 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.117 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.117 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.117 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.117 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.117 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.117 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.117 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.117 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.117 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.117 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.117 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.117 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.118 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.118 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.118 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.118 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.118 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.118 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.118 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.118 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.118 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.118 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.118 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.118 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.118 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.118 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.118 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.118 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.119 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.119 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.119 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.119 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.119 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.119 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.119 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.119 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.119 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.119 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.119 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.119 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.119 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.119 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.119 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.120 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.120 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.120 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.120 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.120 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.120 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.120 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.120 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.120 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.120 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.120 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.120 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.120 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.120 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.120 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.121 INFO:teuthology.orchestra.run.smithi012.stdout: "min_size": { 2024-02-18T11:53:23.121 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "min_size", 2024-02-18T11:53:23.121 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.121 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.121 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.121 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.121 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.121 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.121 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.121 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "default min_size for any created local pool", 2024-02-18T11:53:23.121 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-02-18T11:53:23.121 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.121 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.121 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.121 INFO:teuthology.orchestra.run.smithi012.stdout: "num_rep": { 2024-02-18T11:53:23.121 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "num_rep", 2024-02-18T11:53:23.122 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.122 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.122 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.122 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "3", 2024-02-18T11:53:23.122 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.122 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.122 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.122 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "default replica count for any created local pool", 2024-02-18T11:53:23.122 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.122 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.122 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.122 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.122 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num": { 2024-02-18T11:53:23.122 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pg_num", 2024-02-18T11:53:23.122 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.123 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.123 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.123 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "128", 2024-02-18T11:53:23.123 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.123 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.123 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.123 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "default pg_num for any created local pool", 2024-02-18T11:53:23.123 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.123 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.123 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.123 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.123 INFO:teuthology.orchestra.run.smithi012.stdout: "prefix": { 2024-02-18T11:53:23.123 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "prefix", 2024-02-18T11:53:23.123 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.123 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.123 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.124 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.124 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.124 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.124 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.124 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "name prefix for any created local pool", 2024-02-18T11:53:23.124 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.124 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.124 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.124 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.124 INFO:teuthology.orchestra.run.smithi012.stdout: "subtree": { 2024-02-18T11:53:23.124 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "subtree", 2024-02-18T11:53:23.124 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.124 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.124 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.125 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "rack", 2024-02-18T11:53:23.125 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.125 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.125 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.125 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "CRUSH level for which to create a local pool", 2024-02-18T11:53:23.125 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-02-18T11:53:23.125 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.125 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.125 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.125 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.125 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.125 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.125 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mds_autoscaler", 2024-02-18T11:53:23.125 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.125 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.126 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.126 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.126 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.126 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.126 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.126 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.126 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.126 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.126 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.126 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.126 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.126 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.126 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.126 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.126 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.126 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.127 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.127 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.127 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.127 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.127 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.127 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.127 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.127 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.127 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.127 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.127 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.127 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.127 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.127 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.127 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.128 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.128 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.128 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.128 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.128 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.128 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.128 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.128 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.128 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.128 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.128 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.128 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.128 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.128 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.128 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.129 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.129 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.129 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.129 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.129 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.129 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.129 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.129 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.129 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.129 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.129 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.129 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.129 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.129 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.129 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.130 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.130 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.130 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.130 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.130 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.130 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.130 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.130 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.130 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.130 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.130 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.130 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.130 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.130 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mirroring", 2024-02-18T11:53:23.130 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.130 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.131 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.131 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.131 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.131 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.131 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.131 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.131 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.131 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.131 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.131 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.131 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.131 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.131 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.131 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.131 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.131 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.132 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.132 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.132 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.132 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.132 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.132 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.132 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.132 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.132 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.132 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.132 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.132 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.132 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.132 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.132 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.133 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.133 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.133 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.133 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.133 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.133 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.133 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.133 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.133 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.133 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.133 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.133 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.133 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.133 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.133 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.134 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.134 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.134 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.134 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.134 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.134 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.134 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.134 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.134 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.134 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.134 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.134 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.134 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.134 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.134 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.134 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.135 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.135 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.135 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.135 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.135 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.135 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.135 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.135 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.135 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.135 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.135 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.135 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.135 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "nfs", 2024-02-18T11:53:23.135 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.135 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.135 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.136 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.136 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.136 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.136 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.136 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.136 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.136 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.136 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.136 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.136 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.136 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.136 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.136 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.136 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.136 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.137 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.137 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.137 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.137 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.137 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.137 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.137 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.137 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.137 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.137 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.137 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.137 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.137 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.137 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.138 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.138 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.138 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.138 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.138 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.138 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.138 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.138 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.138 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.138 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.138 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.138 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.138 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.138 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.138 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.138 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.139 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.139 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.139 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.139 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.139 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.139 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.139 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.139 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.139 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.139 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.139 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.139 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.139 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.139 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.139 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.140 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.140 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.140 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.140 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.140 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.140 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.140 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.140 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.140 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.140 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.140 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.140 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.140 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.140 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "orchestrator", 2024-02-18T11:53:23.140 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.141 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.141 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.141 INFO:teuthology.orchestra.run.smithi012.stdout: "fail_fs": { 2024-02-18T11:53:23.141 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "fail_fs", 2024-02-18T11:53:23.141 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.141 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.141 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.141 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.141 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.141 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.141 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.141 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-02-18T11:53:23.141 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.141 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.141 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.141 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.142 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.142 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.142 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.142 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.142 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.142 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.142 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.142 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.142 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.142 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.142 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.142 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.142 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.142 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.142 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.143 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.143 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.143 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.143 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.143 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.143 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.143 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.143 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.143 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.143 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.143 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.143 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.143 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.143 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.143 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.144 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.144 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.144 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.144 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.144 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.144 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.144 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.144 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.144 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.144 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.144 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.144 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.144 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.144 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.144 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.145 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.145 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.145 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.145 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.145 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.145 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.145 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.145 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.145 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.145 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.145 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.145 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.145 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.145 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.145 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.145 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.146 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.146 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.146 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.146 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.146 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.146 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.146 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.146 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.146 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.146 INFO:teuthology.orchestra.run.smithi012.stdout: "orchestrator": { 2024-02-18T11:53:23.146 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "orchestrator", 2024-02-18T11:53:23.146 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.146 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.146 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.146 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.146 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.147 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.147 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.147 INFO:teuthology.orchestra.run.smithi012.stdout: "cephadm", 2024-02-18T11:53:23.147 INFO:teuthology.orchestra.run.smithi012.stdout: "rook", 2024-02-18T11:53:23.147 INFO:teuthology.orchestra.run.smithi012.stdout: "test_orchestrator" 2024-02-18T11:53:23.147 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.147 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Orchestrator backend", 2024-02-18T11:53:23.147 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.147 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.147 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.147 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.147 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.147 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.147 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.147 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "osd_perf_query", 2024-02-18T11:53:23.147 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.148 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.148 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.148 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.148 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.148 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.148 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.148 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.148 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.148 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.148 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.148 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.148 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.148 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.148 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.149 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.149 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.149 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.149 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.149 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.149 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.149 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.149 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.149 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.149 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.149 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.149 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.149 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.149 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.149 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.149 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.150 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.150 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.150 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.150 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.150 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.150 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.150 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.150 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.150 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.150 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.150 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.150 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.150 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.150 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.151 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.151 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.151 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.151 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.151 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.151 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.151 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.151 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.151 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.151 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.151 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.151 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.151 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.151 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.151 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.151 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.152 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.152 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.152 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.152 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.152 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.152 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.152 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.152 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.152 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.152 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.152 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.152 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.152 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.152 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.152 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.153 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "osd_support", 2024-02-18T11:53:23.153 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.153 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.153 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.153 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.153 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.153 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.153 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.153 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.153 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.153 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.153 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.153 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.153 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.153 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.154 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.154 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.154 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.154 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.154 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.154 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.154 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.154 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.154 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.154 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.154 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.154 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.154 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.154 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.155 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.155 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.155 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.155 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.155 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.155 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.155 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.155 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.155 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.155 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.155 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.155 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.155 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.155 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.155 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.155 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.156 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.156 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.156 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.156 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.156 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.156 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.156 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.156 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.156 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.156 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.156 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.156 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.156 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.156 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.156 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.157 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.157 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.157 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.157 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.157 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.157 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.157 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.157 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.157 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.157 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.157 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.157 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.157 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.157 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.157 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.158 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.158 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.158 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pg_autoscaler", 2024-02-18T11:53:23.158 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.158 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.158 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.158 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.158 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.158 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.158 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.158 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.158 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.158 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.158 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.158 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.158 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.159 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.159 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.159 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.159 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.159 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.159 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.159 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.159 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.159 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.159 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.159 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.159 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.159 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.159 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.159 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.159 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.160 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.160 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.160 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.160 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.160 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.160 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.160 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.160 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.160 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.160 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.160 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.160 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.160 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.160 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.160 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.161 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.161 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.161 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.161 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.161 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.161 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.161 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.161 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.161 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.161 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.161 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.164 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.164 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.164 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.164 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.164 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.164 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.164 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.165 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.165 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.165 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.165 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.165 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.165 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.165 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.165 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.165 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.165 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.165 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.165 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-02-18T11:53:23.165 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-02-18T11:53:23.166 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-02-18T11:53:23.166 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.166 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.166 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-02-18T11:53:23.166 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.166 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.166 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.166 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.166 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.166 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.166 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.166 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.166 INFO:teuthology.orchestra.run.smithi012.stdout: "threshold": { 2024-02-18T11:53:23.166 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "threshold", 2024-02-18T11:53:23.166 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-02-18T11:53:23.167 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.167 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.167 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "3.0", 2024-02-18T11:53:23.167 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1.0", 2024-02-18T11:53:23.167 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.167 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.167 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "scaling threshold", 2024-02-18T11:53:23.167 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-02-18T11:53:23.167 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.167 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.167 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.167 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.167 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.167 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.167 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "progress", 2024-02-18T11:53:23.168 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.168 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.168 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.168 INFO:teuthology.orchestra.run.smithi012.stdout: "allow_pg_recovery_event": { 2024-02-18T11:53:23.168 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "allow_pg_recovery_event", 2024-02-18T11:53:23.168 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.168 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.168 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.168 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.168 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.168 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.168 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.168 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "allow the module to show pg recovery progress", 2024-02-18T11:53:23.168 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.168 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.169 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.169 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.169 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": { 2024-02-18T11:53:23.169 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enabled", 2024-02-18T11:53:23.169 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.169 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.169 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.169 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.169 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.169 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.169 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.169 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.169 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.169 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.169 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.170 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.170 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.170 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.170 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.170 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.170 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.170 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.170 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.170 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.170 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.170 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.170 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.170 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.170 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.171 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.171 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.171 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.171 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.171 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.171 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.171 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.171 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.171 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.171 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.171 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.171 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.171 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.171 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.171 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.171 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.172 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.172 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.172 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.172 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.172 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.172 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.172 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.172 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.172 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.172 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.172 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.172 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.172 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.172 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.172 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.173 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.173 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.173 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.173 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.173 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.173 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.173 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.173 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.173 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.173 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.173 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.173 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.173 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.173 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.173 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.174 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.174 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.174 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.174 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.174 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.174 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.174 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.174 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.174 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.174 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.174 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.174 INFO:teuthology.orchestra.run.smithi012.stdout: "max_completed_events": { 2024-02-18T11:53:23.174 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_completed_events", 2024-02-18T11:53:23.174 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.174 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.174 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.175 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "50", 2024-02-18T11:53:23.175 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.175 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.175 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.175 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "number of past completed events to remember", 2024-02-18T11:53:23.175 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.175 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.175 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.175 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.175 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-02-18T11:53:23.175 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-02-18T11:53:23.175 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-02-18T11:53:23.175 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.175 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.175 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-02-18T11:53:23.175 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.176 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.176 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.176 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how long the module is going to sleep", 2024-02-18T11:53:23.176 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.176 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.176 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.176 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.176 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.176 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.176 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.176 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "prometheus", 2024-02-18T11:53:23.176 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.176 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.176 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.176 INFO:teuthology.orchestra.run.smithi012.stdout: "cache": { 2024-02-18T11:53:23.177 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cache", 2024-02-18T11:53:23.177 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.177 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.177 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.177 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.177 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.177 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.177 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.177 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.177 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.177 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.177 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.177 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.177 INFO:teuthology.orchestra.run.smithi012.stdout: "exclude_perf_counters": { 2024-02-18T11:53:23.177 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "exclude_perf_counters", 2024-02-18T11:53:23.177 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.178 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.178 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.178 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.178 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.178 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.178 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.178 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-02-18T11:53:23.178 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-02-18T11:53:23.178 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.178 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.178 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.178 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.178 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.179 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.179 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.179 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.179 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.179 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.179 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.179 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.179 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.179 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.179 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.179 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.179 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.179 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.179 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.179 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.179 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.180 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.180 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.180 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.180 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.180 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.180 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.180 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.180 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.180 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.180 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.180 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.180 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.180 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.180 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.180 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.181 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.181 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.181 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.181 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.181 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.181 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.181 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.181 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.181 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.181 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.181 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.181 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.181 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.181 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.181 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.182 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.182 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.182 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.182 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.182 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.182 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.182 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.182 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.182 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.182 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.182 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.182 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.182 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.182 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.182 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.182 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.183 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.183 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.183 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.183 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.183 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.183 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.183 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_stats_pools": { 2024-02-18T11:53:23.183 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rbd_stats_pools", 2024-02-18T11:53:23.183 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.183 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.183 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.183 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.183 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.183 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.183 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.184 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.184 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.184 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.184 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.184 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.184 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_stats_pools_refresh_interval": { 2024-02-18T11:53:23.184 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-02-18T11:53:23.184 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.184 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.184 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.184 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "300", 2024-02-18T11:53:23.184 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.184 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.184 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.184 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.185 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.185 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.185 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.185 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.185 INFO:teuthology.orchestra.run.smithi012.stdout: "scrape_interval": { 2024-02-18T11:53:23.185 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "scrape_interval", 2024-02-18T11:53:23.185 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-02-18T11:53:23.185 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.185 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.185 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "15.0", 2024-02-18T11:53:23.185 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.185 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.185 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.185 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.185 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.185 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.186 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.186 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.186 INFO:teuthology.orchestra.run.smithi012.stdout: "server_addr": { 2024-02-18T11:53:23.186 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_addr", 2024-02-18T11:53:23.186 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.186 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.186 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.186 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "::", 2024-02-18T11:53:23.186 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.186 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.186 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.186 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-02-18T11:53:23.186 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.186 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.186 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.186 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.187 INFO:teuthology.orchestra.run.smithi012.stdout: "server_port": { 2024-02-18T11:53:23.187 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_port", 2024-02-18T11:53:23.187 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.187 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.187 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.187 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "9283", 2024-02-18T11:53:23.187 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.187 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.187 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.187 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-02-18T11:53:23.187 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.187 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.187 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.187 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.187 INFO:teuthology.orchestra.run.smithi012.stdout: "stale_cache_strategy": { 2024-02-18T11:53:23.188 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "stale_cache_strategy", 2024-02-18T11:53:23.188 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.188 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.188 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.188 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "log", 2024-02-18T11:53:23.188 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.188 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.188 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.188 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.188 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.188 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.188 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.188 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.188 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_behaviour": { 2024-02-18T11:53:23.188 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_behaviour", 2024-02-18T11:53:23.189 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.189 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.189 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.189 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "default", 2024-02-18T11:53:23.189 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.189 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.189 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.189 INFO:teuthology.orchestra.run.smithi012.stdout: "default", 2024-02-18T11:53:23.189 INFO:teuthology.orchestra.run.smithi012.stdout: "error" 2024-02-18T11:53:23.189 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.189 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.189 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.189 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.189 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.189 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.189 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_error_status_code": { 2024-02-18T11:53:23.190 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_error_status_code", 2024-02-18T11:53:23.190 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.190 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.190 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.190 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "500", 2024-02-18T11:53:23.190 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "400", 2024-02-18T11:53:23.190 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "599", 2024-02-18T11:53:23.190 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.190 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.190 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.190 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.190 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.190 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.191 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.191 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.191 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.191 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rbd_support", 2024-02-18T11:53:23.191 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.191 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.191 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.191 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.191 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.191 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.191 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.191 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.191 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.191 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.191 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.191 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.192 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.192 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.192 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.192 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.192 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.192 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.192 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.192 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.192 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.192 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.192 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.192 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.192 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.192 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.192 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.193 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.193 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.193 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.193 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.193 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.193 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.193 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.193 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.193 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.193 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.193 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.193 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.193 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.193 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.193 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.194 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.194 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.194 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.194 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.194 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.194 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.194 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.194 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.194 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.194 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.194 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.194 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.194 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.194 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.194 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.195 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.195 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.195 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.195 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.195 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.195 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.195 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.195 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.195 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.195 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.195 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.195 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.195 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.195 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.195 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.195 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.196 INFO:teuthology.orchestra.run.smithi012.stdout: "max_concurrent_snap_create": { 2024-02-18T11:53:23.196 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_concurrent_snap_create", 2024-02-18T11:53:23.196 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.196 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.196 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.196 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-02-18T11:53:23.196 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.196 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.196 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.196 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.196 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.196 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.196 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.196 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.197 INFO:teuthology.orchestra.run.smithi012.stdout: "mirror_snapshot_schedule": { 2024-02-18T11:53:23.197 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mirror_snapshot_schedule", 2024-02-18T11:53:23.197 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.197 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.197 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.197 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.197 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.197 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.197 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.197 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.197 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.197 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.197 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.197 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.197 INFO:teuthology.orchestra.run.smithi012.stdout: "trash_purge_schedule": { 2024-02-18T11:53:23.197 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "trash_purge_schedule", 2024-02-18T11:53:23.198 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.198 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.198 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.198 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.198 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.198 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.198 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.198 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.198 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.198 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.198 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.198 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.198 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.198 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.198 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.199 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "restful", 2024-02-18T11:53:23.199 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.199 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.199 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.199 INFO:teuthology.orchestra.run.smithi012.stdout: "enable_auth": { 2024-02-18T11:53:23.199 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enable_auth", 2024-02-18T11:53:23.199 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.199 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.199 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.199 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.199 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.199 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.199 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.199 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.199 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.199 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.200 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.200 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.200 INFO:teuthology.orchestra.run.smithi012.stdout: "key_file": { 2024-02-18T11:53:23.200 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "key_file", 2024-02-18T11:53:23.200 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.200 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.200 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.200 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.200 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.200 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.200 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.200 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.200 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.200 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.200 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.201 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.201 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.201 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.201 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.201 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.201 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.201 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.201 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.201 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.201 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.201 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.201 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.201 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.201 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.201 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.201 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.202 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.202 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.202 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.202 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.202 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.202 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.202 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.202 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.202 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.202 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.202 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.202 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.202 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.202 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.202 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.202 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.203 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.203 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.203 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.203 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.203 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.203 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.203 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.203 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.203 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.203 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.204 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.204 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.204 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.204 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.204 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.204 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.204 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.204 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.204 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.204 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.204 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.204 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.204 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.204 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.205 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.205 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.205 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.205 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.205 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.205 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.205 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.205 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.205 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.205 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.205 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.205 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.205 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.205 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.205 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.205 INFO:teuthology.orchestra.run.smithi012.stdout: "server_addr": { 2024-02-18T11:53:23.206 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_addr", 2024-02-18T11:53:23.206 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.206 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.206 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.206 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.206 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.206 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.206 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.206 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.206 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.206 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.206 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.206 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.206 INFO:teuthology.orchestra.run.smithi012.stdout: "server_port": { 2024-02-18T11:53:23.206 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_port", 2024-02-18T11:53:23.207 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.207 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.207 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.207 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.207 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.207 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.207 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.207 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.207 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.207 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.207 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.207 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.207 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.207 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.207 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.207 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rgw", 2024-02-18T11:53:23.208 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.208 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.208 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.208 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.208 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.208 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.208 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.208 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.208 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.208 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.208 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.208 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.208 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.208 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.209 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.209 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.209 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.209 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.209 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.209 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.209 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.209 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.209 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.209 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.209 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.209 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.209 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.209 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.209 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.209 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.210 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.210 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.210 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.210 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.210 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.210 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.210 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.210 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.210 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.210 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.210 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.210 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.210 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.210 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.210 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.211 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.211 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.211 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.211 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.211 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.211 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.211 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.211 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.211 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.211 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.211 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.211 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.211 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.211 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.211 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.211 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.212 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.212 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.212 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.212 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.212 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.212 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.212 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.212 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.212 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.212 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.212 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.212 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.212 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.212 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.212 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.213 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rook", 2024-02-18T11:53:23.213 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.213 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.213 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.213 INFO:teuthology.orchestra.run.smithi012.stdout: "drive_group_interval": { 2024-02-18T11:53:23.213 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "drive_group_interval", 2024-02-18T11:53:23.213 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-02-18T11:53:23.213 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.213 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.213 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "300.0", 2024-02-18T11:53:23.213 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.213 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.213 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.213 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2024-02-18T11:53:23.213 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.214 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.214 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.214 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.214 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.214 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.214 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.214 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.214 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.214 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.214 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.214 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.214 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.214 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.214 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.214 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.214 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.215 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.215 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.215 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.215 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.215 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.215 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.215 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.215 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.215 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.215 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.215 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.215 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.215 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.215 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.215 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.216 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.216 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.216 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.216 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.216 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.216 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.216 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.216 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.216 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.216 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.216 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.216 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.216 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.216 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.217 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.217 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.217 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.217 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.217 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.217 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.217 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.217 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.217 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.217 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.217 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.217 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.217 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.217 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.217 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.218 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.218 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.218 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.218 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.218 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.218 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.218 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.218 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.218 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.218 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.218 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.218 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.218 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.218 INFO:teuthology.orchestra.run.smithi012.stdout: "storage_class": { 2024-02-18T11:53:23.218 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "storage_class", 2024-02-18T11:53:23.219 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.219 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.219 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.219 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "local", 2024-02-18T11:53:23.219 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.219 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.219 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.219 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-02-18T11:53:23.219 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.219 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.219 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.219 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.219 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.219 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.219 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.220 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "selftest", 2024-02-18T11:53:23.220 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.220 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.220 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.220 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.220 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.220 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.220 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.220 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.220 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.220 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.220 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.220 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.220 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.220 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.221 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.221 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.221 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.221 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.221 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.221 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.221 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.221 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.221 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.221 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.221 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.221 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.221 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.221 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.221 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.221 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.222 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.222 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.222 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.222 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.222 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.222 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.222 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.222 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.222 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.222 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.222 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.222 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.222 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.222 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.222 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.223 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.223 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.223 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.223 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.223 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.223 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.223 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.223 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.223 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.223 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.223 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.223 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.223 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.223 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.223 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.223 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.224 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.224 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.224 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.224 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.224 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.224 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.224 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.224 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.224 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.224 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.224 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.224 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.224 INFO:teuthology.orchestra.run.smithi012.stdout: "roption1": { 2024-02-18T11:53:23.224 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "roption1", 2024-02-18T11:53:23.225 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.225 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.225 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.225 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.225 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.225 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.225 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.225 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.225 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.225 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.225 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.225 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.225 INFO:teuthology.orchestra.run.smithi012.stdout: "roption2": { 2024-02-18T11:53:23.225 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "roption2", 2024-02-18T11:53:23.225 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.226 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.226 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.226 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "xyz", 2024-02-18T11:53:23.226 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.226 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.226 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.226 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.226 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.226 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.226 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.226 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.226 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption1": { 2024-02-18T11:53:23.226 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption1", 2024-02-18T11:53:23.226 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.226 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.226 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.227 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.227 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.227 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.227 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.227 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.227 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.227 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.227 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.227 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.227 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption2": { 2024-02-18T11:53:23.227 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption2", 2024-02-18T11:53:23.227 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.227 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.227 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.227 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.228 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.228 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.228 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.228 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.228 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.228 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.228 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.228 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.228 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption3": { 2024-02-18T11:53:23.228 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption3", 2024-02-18T11:53:23.228 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-02-18T11:53:23.228 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.229 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.229 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.229 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.229 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.229 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.229 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.229 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.229 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.229 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.229 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.229 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption4": { 2024-02-18T11:53:23.229 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption4", 2024-02-18T11:53:23.229 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.229 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.230 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.230 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.230 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.230 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.230 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.230 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.230 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.230 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.230 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.230 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.230 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption5": { 2024-02-18T11:53:23.230 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption5", 2024-02-18T11:53:23.230 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.230 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.230 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.230 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.231 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.231 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.231 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.231 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.231 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.231 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.231 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.231 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.231 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption6": { 2024-02-18T11:53:23.231 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption6", 2024-02-18T11:53:23.231 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.231 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.231 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.231 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.231 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.232 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.232 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.232 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.232 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.232 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.232 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.232 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.232 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption7": { 2024-02-18T11:53:23.232 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption7", 2024-02-18T11:53:23.232 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.232 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.232 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.232 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.232 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-02-18T11:53:23.232 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "42", 2024-02-18T11:53:23.233 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.233 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.233 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.233 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.233 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.233 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.233 INFO:teuthology.orchestra.run.smithi012.stdout: "testkey": { 2024-02-18T11:53:23.233 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "testkey", 2024-02-18T11:53:23.233 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.233 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.233 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.233 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.233 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.233 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.233 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.234 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.234 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.234 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.234 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.234 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.234 INFO:teuthology.orchestra.run.smithi012.stdout: "testlkey": { 2024-02-18T11:53:23.234 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "testlkey", 2024-02-18T11:53:23.234 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.234 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.234 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.234 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.234 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.234 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.234 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.234 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.235 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.235 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.235 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.235 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.235 INFO:teuthology.orchestra.run.smithi012.stdout: "testnewline": { 2024-02-18T11:53:23.235 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "testnewline", 2024-02-18T11:53:23.235 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.235 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.235 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.235 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.235 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.235 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.235 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.235 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.235 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.235 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.236 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.236 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.236 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.236 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.236 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.236 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "snap_schedule", 2024-02-18T11:53:23.236 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.236 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.236 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.236 INFO:teuthology.orchestra.run.smithi012.stdout: "allow_m_granularity": { 2024-02-18T11:53:23.236 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "allow_m_granularity", 2024-02-18T11:53:23.236 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.236 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.236 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.237 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.237 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.237 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.237 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.237 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "allow minute scheduled snapshots", 2024-02-18T11:53:23.237 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.237 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.237 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.237 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.237 INFO:teuthology.orchestra.run.smithi012.stdout: "dump_on_update": { 2024-02-18T11:53:23.237 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "dump_on_update", 2024-02-18T11:53:23.237 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.237 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.237 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.237 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.237 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.238 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.238 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.238 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "dump database to debug log on update", 2024-02-18T11:53:23.238 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.238 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.238 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.238 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.238 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.238 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.238 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.238 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.238 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.238 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.238 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.238 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.239 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.239 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.239 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.239 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.239 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.239 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.239 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.239 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.239 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.239 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.239 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.239 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.239 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.239 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.239 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.240 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.240 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.240 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.240 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.240 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.240 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.240 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.240 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.240 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.240 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.240 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.240 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.240 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.240 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.240 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.241 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.241 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.241 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.241 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.241 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.241 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.241 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.241 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.241 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.241 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.241 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.241 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.241 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.241 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.241 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.242 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.242 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.242 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.242 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.242 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.242 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.242 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.242 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.242 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.242 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.242 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.242 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.242 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.242 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.243 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.243 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.243 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.243 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.243 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.243 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.243 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "stats", 2024-02-18T11:53:23.243 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.243 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.243 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.243 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.243 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.243 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.243 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.243 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.243 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.244 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.244 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.244 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.244 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.244 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.244 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.244 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.244 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.244 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.244 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.244 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.244 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.244 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.244 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.244 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.245 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.245 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.245 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.245 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.245 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.245 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.245 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.245 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.245 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.245 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.245 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.245 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.245 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.245 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.245 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.246 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.246 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.246 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.246 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.246 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.246 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.246 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.246 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.246 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.246 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.246 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.246 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.246 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.246 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.246 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.246 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.247 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.247 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.247 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.247 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.247 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.247 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.247 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.247 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.247 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.247 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.247 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.247 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.247 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.247 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.247 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.248 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.248 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.248 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.248 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.248 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.248 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.248 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "status", 2024-02-18T11:53:23.248 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.248 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.248 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.248 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.248 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.248 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.248 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.248 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.249 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.249 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.249 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.249 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.249 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.249 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.249 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.249 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.249 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.249 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.249 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.249 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.249 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.249 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.249 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.249 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.250 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.250 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.250 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.250 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.250 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.250 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.250 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.250 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.250 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.250 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.250 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.250 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.250 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.250 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.250 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.251 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.251 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.251 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.251 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.251 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.251 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.251 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.251 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.251 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.251 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.251 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.251 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.251 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.251 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.251 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.251 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.252 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.252 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.252 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.252 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.252 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.252 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.252 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.252 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.252 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.252 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.252 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.252 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.252 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.252 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.252 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.253 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.253 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.253 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.253 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.253 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.253 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.253 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "telegraf", 2024-02-18T11:53:23.253 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.253 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.253 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.253 INFO:teuthology.orchestra.run.smithi012.stdout: "address": { 2024-02-18T11:53:23.253 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "address", 2024-02-18T11:53:23.253 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.253 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.253 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.253 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-02-18T11:53:23.254 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.254 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.254 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.254 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.254 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.254 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.254 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.254 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.254 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-02-18T11:53:23.254 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-02-18T11:53:23.254 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-02-18T11:53:23.254 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.254 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.254 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "15", 2024-02-18T11:53:23.254 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.254 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.255 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.255 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.255 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.255 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.255 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.255 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.255 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.255 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.255 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.255 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.255 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.255 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.255 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.255 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.256 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.256 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.256 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.256 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.256 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.256 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.256 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.256 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.256 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.256 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.256 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.256 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.256 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.256 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.257 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.257 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.257 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.257 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.257 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.257 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.257 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.257 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.257 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.257 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.257 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.257 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.257 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.257 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.257 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.257 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.258 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.258 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.258 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.258 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.258 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.258 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.258 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.258 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.258 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.258 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.258 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.258 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.258 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.258 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.258 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.259 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.259 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.259 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.259 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.259 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.259 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.259 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.259 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.259 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.259 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.259 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.259 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.259 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.259 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.259 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.260 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.260 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.260 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.260 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.260 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.260 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "telemetry", 2024-02-18T11:53:23.260 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.260 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.260 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.260 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_basic": { 2024-02-18T11:53:23.260 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_basic", 2024-02-18T11:53:23.260 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.260 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.260 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.260 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.261 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.261 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.261 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.261 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share basic cluster information (size, version)", 2024-02-18T11:53:23.261 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.261 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.261 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.261 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.261 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_crash": { 2024-02-18T11:53:23.261 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_crash", 2024-02-18T11:53:23.261 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.261 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.261 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.261 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.261 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.261 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.262 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.262 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-02-18T11:53:23.262 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.262 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.262 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.262 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.262 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_device": { 2024-02-18T11:53:23.262 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_device", 2024-02-18T11:53:23.262 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.262 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.262 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.262 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.262 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.262 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.262 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.263 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-02-18T11:53:23.263 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.263 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.263 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.263 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.263 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_ident": { 2024-02-18T11:53:23.263 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_ident", 2024-02-18T11:53:23.263 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.263 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.263 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.263 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.263 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.263 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.263 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.263 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-02-18T11:53:23.264 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.264 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.264 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.264 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.264 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_perf": { 2024-02-18T11:53:23.264 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_perf", 2024-02-18T11:53:23.264 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.264 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.264 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.264 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.264 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.264 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.264 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.264 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share various performance metrics of a cluster", 2024-02-18T11:53:23.265 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.265 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.265 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.265 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.265 INFO:teuthology.orchestra.run.smithi012.stdout: "contact": { 2024-02-18T11:53:23.265 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "contact", 2024-02-18T11:53:23.265 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.265 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.265 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.265 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.265 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.265 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.265 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.265 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.265 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.266 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.266 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.266 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.266 INFO:teuthology.orchestra.run.smithi012.stdout: "description": { 2024-02-18T11:53:23.266 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "description", 2024-02-18T11:53:23.266 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.266 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.266 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.266 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.266 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.266 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.266 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.266 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.266 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.266 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.266 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.267 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.267 INFO:teuthology.orchestra.run.smithi012.stdout: "device_url": { 2024-02-18T11:53:23.267 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "device_url", 2024-02-18T11:53:23.267 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.267 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.267 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.267 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-02-18T11:53:23.267 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.267 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.267 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.267 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.267 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.267 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.267 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.268 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.268 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": { 2024-02-18T11:53:23.268 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enabled", 2024-02-18T11:53:23.268 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.268 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.268 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.268 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.268 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.268 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.268 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.268 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.268 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.268 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.268 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.269 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.269 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-02-18T11:53:23.269 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-02-18T11:53:23.269 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.269 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.269 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.269 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "24", 2024-02-18T11:53:23.269 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "8", 2024-02-18T11:53:23.269 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.269 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.269 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.269 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.269 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.269 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.269 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.270 INFO:teuthology.orchestra.run.smithi012.stdout: "last_opt_revision": { 2024-02-18T11:53:23.270 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "last_opt_revision", 2024-02-18T11:53:23.270 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.270 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.270 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.270 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "1", 2024-02-18T11:53:23.270 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.270 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.270 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.270 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.270 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.270 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.270 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.270 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.270 INFO:teuthology.orchestra.run.smithi012.stdout: "leaderboard": { 2024-02-18T11:53:23.271 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "leaderboard", 2024-02-18T11:53:23.271 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.271 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.271 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.271 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.271 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.271 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.271 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.271 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.271 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.271 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.271 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.271 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.271 INFO:teuthology.orchestra.run.smithi012.stdout: "leaderboard_description": { 2024-02-18T11:53:23.271 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "leaderboard_description", 2024-02-18T11:53:23.272 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.272 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.272 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.272 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.272 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.272 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.272 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.272 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.272 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.272 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.272 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.272 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.272 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.272 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.273 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.273 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.273 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.273 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.273 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.273 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.273 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.273 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.273 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.273 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.273 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.273 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.273 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.273 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.273 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.274 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.274 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.274 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.274 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.274 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.274 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.274 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.274 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.274 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.274 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.274 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.274 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.274 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.274 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.274 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.274 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.275 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.275 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.275 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.275 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.275 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.275 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.275 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.275 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.275 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.275 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.275 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.275 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.275 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.275 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.275 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.276 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.276 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.276 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.276 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.276 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.276 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.276 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.276 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.276 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.276 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.276 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.276 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.276 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.276 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.277 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.277 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.277 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.277 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.277 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.277 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.277 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.277 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.277 INFO:teuthology.orchestra.run.smithi012.stdout: "organization": { 2024-02-18T11:53:23.277 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "organization", 2024-02-18T11:53:23.277 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.277 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.277 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.277 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.277 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.278 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.278 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.278 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.278 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.278 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.278 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.278 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.278 INFO:teuthology.orchestra.run.smithi012.stdout: "proxy": { 2024-02-18T11:53:23.278 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "proxy", 2024-02-18T11:53:23.278 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.278 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.278 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.278 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.278 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.278 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.278 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.279 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.279 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.279 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.279 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.279 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.279 INFO:teuthology.orchestra.run.smithi012.stdout: "url": { 2024-02-18T11:53:23.279 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "url", 2024-02-18T11:53:23.279 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.279 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.279 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.279 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-02-18T11:53:23.279 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.279 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.279 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.279 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.280 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.280 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.280 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.280 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.280 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.280 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.280 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.280 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "test_orchestrator", 2024-02-18T11:53:23.280 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.280 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.280 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.280 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.280 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.280 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.281 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.281 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.281 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.281 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.281 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.281 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.281 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.281 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.281 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.281 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.281 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.281 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.281 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.281 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.281 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.282 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.282 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.282 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.282 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.282 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.282 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.282 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.282 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.282 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.282 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.282 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.282 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.282 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.282 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.282 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.283 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.283 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.283 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.283 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.283 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.283 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.283 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.283 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.283 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.283 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.283 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.283 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.283 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.283 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.283 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.284 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.284 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.284 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.284 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.284 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.284 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.284 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.284 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.284 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.284 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.284 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.284 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.284 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.284 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.284 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.285 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.285 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.285 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.285 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.285 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.285 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.285 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.285 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.285 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.285 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.285 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "volumes", 2024-02-18T11:53:23.285 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.285 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.285 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.285 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.285 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.286 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.286 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.286 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.286 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.286 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.286 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.286 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.286 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.286 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.286 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.286 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.286 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.286 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.286 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.286 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.287 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.287 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.287 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.287 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.287 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.287 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.287 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.287 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.287 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.287 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.287 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.287 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.287 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.287 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.287 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.288 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.288 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.288 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.288 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.288 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.288 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.288 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.288 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.288 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.288 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.288 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.288 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.288 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.288 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.288 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.289 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.289 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.289 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.289 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.289 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.289 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.289 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.289 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.289 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.289 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.289 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.289 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.289 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.289 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.289 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.290 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.290 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.290 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.290 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.290 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.290 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.290 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.290 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.290 INFO:teuthology.orchestra.run.smithi012.stdout: "max_concurrent_clones": { 2024-02-18T11:53:23.290 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_concurrent_clones", 2024-02-18T11:53:23.290 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.290 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.290 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.290 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "4", 2024-02-18T11:53:23.290 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.291 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.291 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.291 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Number of asynchronous cloner threads", 2024-02-18T11:53:23.291 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.291 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.291 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.291 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.291 INFO:teuthology.orchestra.run.smithi012.stdout: "periodic_async_work": { 2024-02-18T11:53:23.291 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "periodic_async_work", 2024-02-18T11:53:23.291 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.291 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.291 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.291 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.291 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.291 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.292 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.292 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Periodically check for async work", 2024-02-18T11:53:23.292 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.292 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.292 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.292 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.292 INFO:teuthology.orchestra.run.smithi012.stdout: "snapshot_clone_delay": { 2024-02-18T11:53:23.292 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "snapshot_clone_delay", 2024-02-18T11:53:23.292 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.292 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.292 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.292 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-02-18T11:53:23.292 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.292 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.292 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.293 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-02-18T11:53:23.293 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.293 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.293 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.293 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.293 INFO:teuthology.orchestra.run.smithi012.stdout: "snapshot_clone_no_wait": { 2024-02-18T11:53:23.293 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "snapshot_clone_no_wait", 2024-02-18T11:53:23.293 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.293 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.293 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.293 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.293 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.293 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.293 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.294 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-02-18T11:53:23.294 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.294 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.294 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.294 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.294 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.294 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.294 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.294 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix", 2024-02-18T11:53:23.294 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.294 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.294 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.294 INFO:teuthology.orchestra.run.smithi012.stdout: "discovery_interval": { 2024-02-18T11:53:23.294 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "discovery_interval", 2024-02-18T11:53:23.294 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-02-18T11:53:23.294 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.295 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.295 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "100", 2024-02-18T11:53:23.295 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.295 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.295 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.295 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.295 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.295 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.295 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.295 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.295 INFO:teuthology.orchestra.run.smithi012.stdout: "identifier": { 2024-02-18T11:53:23.295 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "identifier", 2024-02-18T11:53:23.295 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.295 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.295 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.296 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.296 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.296 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.296 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.296 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.296 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.296 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.296 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.296 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.296 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-02-18T11:53:23.296 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-02-18T11:53:23.296 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-02-18T11:53:23.296 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.296 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.296 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-02-18T11:53:23.297 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.297 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.297 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.297 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.297 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.297 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.297 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.297 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.297 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.297 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.297 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.297 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.297 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.297 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.297 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.298 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.298 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.298 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.298 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.298 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.298 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.298 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.298 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.298 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.298 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.298 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.298 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.298 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.298 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.298 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.299 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.299 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.299 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.299 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.299 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.299 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.299 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.299 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.299 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.299 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.299 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.299 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.299 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.299 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.299 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.300 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.300 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.300 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.300 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.300 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.300 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.300 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.300 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.300 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.300 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.300 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.300 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.300 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.300 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.300 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.301 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.301 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.301 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.301 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.301 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.301 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.301 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.301 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.301 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.301 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.301 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.301 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.301 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.301 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.301 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.301 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.302 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.302 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.302 INFO:teuthology.orchestra.run.smithi012.stdout: "zabbix_host": { 2024-02-18T11:53:23.302 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix_host", 2024-02-18T11:53:23.302 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.302 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.302 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.302 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.302 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.302 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.302 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.302 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.302 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.302 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.302 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.302 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.303 INFO:teuthology.orchestra.run.smithi012.stdout: "zabbix_port": { 2024-02-18T11:53:23.303 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix_port", 2024-02-18T11:53:23.303 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.303 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.303 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.303 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10051", 2024-02-18T11:53:23.303 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.303 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.303 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.303 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.303 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.303 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.303 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.303 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.303 INFO:teuthology.orchestra.run.smithi012.stdout: "zabbix_sender": { 2024-02-18T11:53:23.304 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix_sender", 2024-02-18T11:53:23.304 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.304 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.304 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.304 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-02-18T11:53:23.304 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.304 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.304 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.304 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.304 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.304 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.304 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.304 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.304 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.304 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.305 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-02-18T11:53:23.305 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.305 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.305 INFO:teuthology.orchestra.run.smithi012.stdout: "gid": 24112, 2024-02-18T11:53:23.305 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "y", 2024-02-18T11:53:23.305 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr_features": 4540701547738038271, 2024-02-18T11:53:23.305 INFO:teuthology.orchestra.run.smithi012.stdout: "available_modules": [ 2024-02-18T11:53:23.305 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.305 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "alerts", 2024-02-18T11:53:23.305 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.305 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.305 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.305 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-02-18T11:53:23.305 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-02-18T11:53:23.305 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-02-18T11:53:23.305 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.306 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.306 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-02-18T11:53:23.306 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.306 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.306 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.306 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How frequently to reexamine health status", 2024-02-18T11:53:23.306 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.306 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.306 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.306 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.306 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.306 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.306 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.307 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.307 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.307 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.307 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.307 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.307 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.307 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.307 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.307 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.307 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.307 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.307 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.307 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.307 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.307 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.307 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.308 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.308 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.308 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.308 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.308 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.308 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.308 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.308 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.308 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.308 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.308 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.308 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.308 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.308 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.308 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.309 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.309 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.309 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.309 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.309 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.309 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.309 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.309 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.309 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.309 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.309 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.309 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.309 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.309 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.309 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.310 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.310 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.310 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.310 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.310 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.310 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.310 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.310 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.310 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.310 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.310 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.310 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.310 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.310 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.310 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.311 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.311 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.311 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.311 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.311 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.311 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.311 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_destination": { 2024-02-18T11:53:23.311 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_destination", 2024-02-18T11:53:23.311 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.311 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.311 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.311 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.311 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.311 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.311 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.312 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Email address to send alerts to", 2024-02-18T11:53:23.312 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.312 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.312 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.312 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.312 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_from_name": { 2024-02-18T11:53:23.312 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_from_name", 2024-02-18T11:53:23.312 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.312 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.312 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.312 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "Ceph", 2024-02-18T11:53:23.312 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.312 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.312 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.312 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Email From: name", 2024-02-18T11:53:23.313 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.313 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.313 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.313 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.313 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_host": { 2024-02-18T11:53:23.313 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_host", 2024-02-18T11:53:23.313 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.313 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.313 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.313 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.313 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.313 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.313 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.313 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SMTP server", 2024-02-18T11:53:23.313 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.314 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.314 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.314 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.314 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_password": { 2024-02-18T11:53:23.314 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_password", 2024-02-18T11:53:23.314 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.314 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.314 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.314 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.314 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.314 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.314 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.314 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Password to authenticate with", 2024-02-18T11:53:23.314 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.314 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.314 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.315 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.315 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_port": { 2024-02-18T11:53:23.315 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_port", 2024-02-18T11:53:23.315 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.315 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.315 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.315 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "465", 2024-02-18T11:53:23.315 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.315 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.315 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.315 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SMTP port", 2024-02-18T11:53:23.315 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.315 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.315 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.315 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.315 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_sender": { 2024-02-18T11:53:23.316 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_sender", 2024-02-18T11:53:23.316 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.316 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.316 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.316 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.316 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.316 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.316 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.316 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SMTP envelope sender", 2024-02-18T11:53:23.316 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.316 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.316 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.316 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.316 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_ssl": { 2024-02-18T11:53:23.316 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_ssl", 2024-02-18T11:53:23.317 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.317 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.317 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.317 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.317 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.317 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.317 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.317 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use SSL to connect to SMTP server", 2024-02-18T11:53:23.317 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.317 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.317 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.317 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.317 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_user": { 2024-02-18T11:53:23.317 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_user", 2024-02-18T11:53:23.317 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.317 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.318 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.318 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.318 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.318 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.318 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.318 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "User to authenticate as", 2024-02-18T11:53:23.318 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.318 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.318 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.318 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.318 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.318 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.318 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.319 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "balancer", 2024-02-18T11:53:23.319 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.319 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.319 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.319 INFO:teuthology.orchestra.run.smithi012.stdout: "active": { 2024-02-18T11:53:23.319 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "active", 2024-02-18T11:53:23.319 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.319 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.319 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.319 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.319 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.319 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.319 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.319 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "automatically balance PGs across cluster", 2024-02-18T11:53:23.319 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.319 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.320 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.320 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.320 INFO:teuthology.orchestra.run.smithi012.stdout: "begin_time": { 2024-02-18T11:53:23.320 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "begin_time", 2024-02-18T11:53:23.320 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.320 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.320 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.320 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0000", 2024-02-18T11:53:23.320 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.320 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.320 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.320 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "beginning time of day to automatically balance", 2024-02-18T11:53:23.320 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-02-18T11:53:23.320 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.321 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.321 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.321 INFO:teuthology.orchestra.run.smithi012.stdout: "begin_weekday": { 2024-02-18T11:53:23.321 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "begin_weekday", 2024-02-18T11:53:23.321 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-02-18T11:53:23.321 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.321 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.321 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-02-18T11:53:23.321 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "0", 2024-02-18T11:53:23.321 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "6", 2024-02-18T11:53:23.321 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.321 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-02-18T11:53:23.321 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-02-18T11:53:23.321 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.321 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.321 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.322 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_compat_max_iterations": { 2024-02-18T11:53:23.322 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crush_compat_max_iterations", 2024-02-18T11:53:23.322 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-02-18T11:53:23.322 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.322 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.322 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "25", 2024-02-18T11:53:23.322 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-02-18T11:53:23.322 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "250", 2024-02-18T11:53:23.322 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.322 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-02-18T11:53:23.322 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.322 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.322 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.322 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.322 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_compat_metrics": { 2024-02-18T11:53:23.323 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crush_compat_metrics", 2024-02-18T11:53:23.323 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.323 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.323 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.323 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "pgs,objects,bytes", 2024-02-18T11:53:23.323 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.323 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.323 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.323 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-02-18T11:53:23.323 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-02-18T11:53:23.323 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.323 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.323 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.323 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_compat_step": { 2024-02-18T11:53:23.323 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crush_compat_step", 2024-02-18T11:53:23.324 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-02-18T11:53:23.324 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.324 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.324 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0.5", 2024-02-18T11:53:23.324 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "0.001", 2024-02-18T11:53:23.324 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "0.999", 2024-02-18T11:53:23.324 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.324 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "aggressiveness of optimization", 2024-02-18T11:53:23.324 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-02-18T11:53:23.324 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.324 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.324 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.324 INFO:teuthology.orchestra.run.smithi012.stdout: "end_time": { 2024-02-18T11:53:23.324 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "end_time", 2024-02-18T11:53:23.325 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.325 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.325 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.325 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "2359", 2024-02-18T11:53:23.325 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.325 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.325 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.325 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "ending time of day to automatically balance", 2024-02-18T11:53:23.325 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-02-18T11:53:23.325 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.325 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.325 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.325 INFO:teuthology.orchestra.run.smithi012.stdout: "end_weekday": { 2024-02-18T11:53:23.325 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "end_weekday", 2024-02-18T11:53:23.325 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-02-18T11:53:23.326 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.326 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.326 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-02-18T11:53:23.326 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "0", 2024-02-18T11:53:23.326 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "6", 2024-02-18T11:53:23.326 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.326 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-02-18T11:53:23.326 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-02-18T11:53:23.326 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.326 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.326 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.326 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.326 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.326 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.326 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.327 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.327 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.327 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.327 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.327 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.327 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.327 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.327 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.327 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.327 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.327 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.327 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.327 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.327 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.327 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.327 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.328 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.328 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.328 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.328 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.328 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.328 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.328 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.328 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.328 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.328 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.328 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.328 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.328 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.328 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.328 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.329 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.329 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.329 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.329 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.329 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.329 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.329 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.329 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.329 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.329 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.329 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.329 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.329 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.329 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.329 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.330 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.330 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.330 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.330 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.330 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.330 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.330 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.330 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.330 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.330 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.330 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.330 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.330 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.330 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.331 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.331 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.331 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.331 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.331 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.331 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.331 INFO:teuthology.orchestra.run.smithi012.stdout: "min_score": { 2024-02-18T11:53:23.331 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "min_score", 2024-02-18T11:53:23.331 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-02-18T11:53:23.331 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.331 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.331 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-02-18T11:53:23.331 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.331 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.331 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.331 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-02-18T11:53:23.332 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.332 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.332 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.332 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.332 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": { 2024-02-18T11:53:23.332 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mode", 2024-02-18T11:53:23.332 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.332 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.332 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.332 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "upmap", 2024-02-18T11:53:23.332 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.332 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.332 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.332 INFO:teuthology.orchestra.run.smithi012.stdout: "crush-compat", 2024-02-18T11:53:23.333 INFO:teuthology.orchestra.run.smithi012.stdout: "none", 2024-02-18T11:53:23.333 INFO:teuthology.orchestra.run.smithi012.stdout: "read", 2024-02-18T11:53:23.333 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap", 2024-02-18T11:53:23.333 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap-read" 2024-02-18T11:53:23.333 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.333 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Balancer mode", 2024-02-18T11:53:23.333 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.333 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.333 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.333 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.333 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_ids": { 2024-02-18T11:53:23.333 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pool_ids", 2024-02-18T11:53:23.333 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.333 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.333 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.333 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.334 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.334 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.334 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.334 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-02-18T11:53:23.334 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.334 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.334 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.334 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.334 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-02-18T11:53:23.334 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-02-18T11:53:23.334 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-02-18T11:53:23.334 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.334 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.334 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-02-18T11:53:23.334 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.335 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.335 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.335 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-02-18T11:53:23.335 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.335 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.335 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.335 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.335 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap_max_deviation": { 2024-02-18T11:53:23.335 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "upmap_max_deviation", 2024-02-18T11:53:23.335 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.335 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.335 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.335 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-02-18T11:53:23.335 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-02-18T11:53:23.335 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.336 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.336 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "deviation below which no optimization is attempted", 2024-02-18T11:53:23.336 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-02-18T11:53:23.336 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.336 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.336 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.336 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap_max_optimizations": { 2024-02-18T11:53:23.336 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "upmap_max_optimizations", 2024-02-18T11:53:23.336 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-02-18T11:53:23.336 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.336 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.336 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-02-18T11:53:23.336 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.336 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.336 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.337 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-02-18T11:53:23.337 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.337 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.337 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.337 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.337 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.337 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.337 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.337 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cephadm", 2024-02-18T11:53:23.337 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.337 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.337 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.337 INFO:teuthology.orchestra.run.smithi012.stdout: "agent_down_multiplier": { 2024-02-18T11:53:23.337 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "agent_down_multiplier", 2024-02-18T11:53:23.337 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-02-18T11:53:23.338 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.338 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.338 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "3.0", 2024-02-18T11:53:23.338 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.338 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.338 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.338 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-02-18T11:53:23.338 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.338 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.338 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.338 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.338 INFO:teuthology.orchestra.run.smithi012.stdout: "agent_refresh_rate": { 2024-02-18T11:53:23.338 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "agent_refresh_rate", 2024-02-18T11:53:23.338 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-02-18T11:53:23.338 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.339 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.339 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "20", 2024-02-18T11:53:23.339 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.339 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.339 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.339 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-02-18T11:53:23.339 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.339 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.339 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.339 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.339 INFO:teuthology.orchestra.run.smithi012.stdout: "agent_starting_port": { 2024-02-18T11:53:23.339 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "agent_starting_port", 2024-02-18T11:53:23.339 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.339 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.339 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.340 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "4721", 2024-02-18T11:53:23.340 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.340 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.340 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.340 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-02-18T11:53:23.340 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.340 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.340 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.340 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.340 INFO:teuthology.orchestra.run.smithi012.stdout: "allow_ptrace": { 2024-02-18T11:53:23.340 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "allow_ptrace", 2024-02-18T11:53:23.340 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.340 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.340 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.340 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.341 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.341 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.341 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.341 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-02-18T11:53:23.341 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-02-18T11:53:23.341 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.341 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.341 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.341 INFO:teuthology.orchestra.run.smithi012.stdout: "autotune_interval": { 2024-02-18T11:53:23.341 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "autotune_interval", 2024-02-18T11:53:23.341 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-02-18T11:53:23.341 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.341 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.341 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-02-18T11:53:23.342 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.342 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.342 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.342 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to autotune daemon memory", 2024-02-18T11:53:23.342 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.342 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.342 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.342 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.342 INFO:teuthology.orchestra.run.smithi012.stdout: "autotune_memory_target_ratio": { 2024-02-18T11:53:23.342 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "autotune_memory_target_ratio", 2024-02-18T11:53:23.342 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-02-18T11:53:23.342 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.342 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.342 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0.7", 2024-02-18T11:53:23.342 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.343 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.343 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.343 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-02-18T11:53:23.343 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.343 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.343 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.343 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.343 INFO:teuthology.orchestra.run.smithi012.stdout: "cephadm_log_destination": { 2024-02-18T11:53:23.343 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cephadm_log_destination", 2024-02-18T11:53:23.343 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.343 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.343 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.343 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.343 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.343 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.344 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.344 INFO:teuthology.orchestra.run.smithi012.stdout: "file", 2024-02-18T11:53:23.344 INFO:teuthology.orchestra.run.smithi012.stdout: "file,syslog", 2024-02-18T11:53:23.344 INFO:teuthology.orchestra.run.smithi012.stdout: "syslog" 2024-02-18T11:53:23.344 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.344 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-02-18T11:53:23.344 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.344 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.344 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.344 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.344 INFO:teuthology.orchestra.run.smithi012.stdout: "cgroups_split": { 2024-02-18T11:53:23.344 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cgroups_split", 2024-02-18T11:53:23.344 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.344 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.345 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.345 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.345 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.345 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.345 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.345 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-02-18T11:53:23.345 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.345 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.345 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.345 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.345 INFO:teuthology.orchestra.run.smithi012.stdout: "config_checks_enabled": { 2024-02-18T11:53:23.345 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "config_checks_enabled", 2024-02-18T11:53:23.345 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.345 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.345 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.346 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.346 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.346 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.346 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.346 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-02-18T11:53:23.346 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.346 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.346 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.346 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.346 INFO:teuthology.orchestra.run.smithi012.stdout: "config_dashboard": { 2024-02-18T11:53:23.346 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "config_dashboard", 2024-02-18T11:53:23.346 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.346 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.346 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.346 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.347 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.347 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.347 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.347 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-02-18T11:53:23.347 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.347 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.347 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.347 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.347 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_alertmanager": { 2024-02-18T11:53:23.347 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_alertmanager", 2024-02-18T11:53:23.347 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.347 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.347 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.347 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-02-18T11:53:23.347 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.348 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.348 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.348 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-02-18T11:53:23.348 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.348 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.348 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.348 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.348 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_base": { 2024-02-18T11:53:23.348 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_base", 2024-02-18T11:53:23.348 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.348 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.348 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.348 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/ceph", 2024-02-18T11:53:23.348 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.348 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.349 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.349 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Container image name, without the tag", 2024-02-18T11:53:23.349 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.349 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.349 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.349 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.349 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_elasticsearch": { 2024-02-18T11:53:23.349 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_elasticsearch", 2024-02-18T11:53:23.349 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.349 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.349 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.349 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-02-18T11:53:23.349 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.349 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.349 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.350 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "elasticsearch container image", 2024-02-18T11:53:23.350 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.350 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.350 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.350 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.350 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_grafana": { 2024-02-18T11:53:23.350 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_grafana", 2024-02-18T11:53:23.350 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.350 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.350 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.350 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-02-18T11:53:23.350 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.350 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.350 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.350 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-02-18T11:53:23.350 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.351 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.351 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.351 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.351 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_haproxy": { 2024-02-18T11:53:23.351 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_haproxy", 2024-02-18T11:53:23.351 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.351 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.351 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.351 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-02-18T11:53:23.351 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.351 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.351 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.351 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "HAproxy container image", 2024-02-18T11:53:23.351 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.351 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.352 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.352 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.352 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_jaeger_agent": { 2024-02-18T11:53:23.352 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_jaeger_agent", 2024-02-18T11:53:23.352 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.352 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.352 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.352 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-02-18T11:53:23.352 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.352 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.352 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.352 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Jaeger agent container image", 2024-02-18T11:53:23.352 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.352 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.352 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.353 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.353 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_jaeger_collector": { 2024-02-18T11:53:23.353 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_jaeger_collector", 2024-02-18T11:53:23.353 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.353 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.353 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.353 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-02-18T11:53:23.353 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.353 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.353 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.353 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Jaeger collector container image", 2024-02-18T11:53:23.353 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.353 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.353 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.354 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.354 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_jaeger_query": { 2024-02-18T11:53:23.354 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_jaeger_query", 2024-02-18T11:53:23.354 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.354 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.354 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.354 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-02-18T11:53:23.354 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.354 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.354 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.354 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Jaeger query container image", 2024-02-18T11:53:23.354 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.354 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.354 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.354 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.354 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_keepalived": { 2024-02-18T11:53:23.355 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_keepalived", 2024-02-18T11:53:23.355 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.355 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.355 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.355 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-02-18T11:53:23.355 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.355 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.355 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.355 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Keepalived container image", 2024-02-18T11:53:23.355 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.355 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.355 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.355 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.355 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_loki": { 2024-02-18T11:53:23.355 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_loki", 2024-02-18T11:53:23.356 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.356 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.356 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.356 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-02-18T11:53:23.356 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.356 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.356 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.356 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Loki container image", 2024-02-18T11:53:23.356 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.356 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.356 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.356 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.356 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_node_exporter": { 2024-02-18T11:53:23.356 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_node_exporter", 2024-02-18T11:53:23.357 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.357 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.357 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.357 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-02-18T11:53:23.357 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.357 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.357 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.357 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-02-18T11:53:23.357 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.357 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.357 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.357 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.357 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_nvmeof": { 2024-02-18T11:53:23.357 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_nvmeof", 2024-02-18T11:53:23.357 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.358 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.358 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.358 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2024-02-18T11:53:23.358 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.358 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.358 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.358 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Nvme-of container image", 2024-02-18T11:53:23.358 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.358 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.358 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.358 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.358 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_prometheus": { 2024-02-18T11:53:23.358 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_prometheus", 2024-02-18T11:53:23.358 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.358 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.359 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.359 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-02-18T11:53:23.359 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.359 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.359 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.359 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-02-18T11:53:23.359 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.359 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.359 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.359 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.359 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_promtail": { 2024-02-18T11:53:23.359 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_promtail", 2024-02-18T11:53:23.359 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.359 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.359 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.360 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-02-18T11:53:23.360 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.360 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.360 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.360 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Promtail container image", 2024-02-18T11:53:23.360 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.360 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.360 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.360 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.360 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_snmp_gateway": { 2024-02-18T11:53:23.360 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_snmp_gateway", 2024-02-18T11:53:23.360 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.360 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.360 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.360 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-02-18T11:53:23.361 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.361 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.361 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.361 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SNMP Gateway container image", 2024-02-18T11:53:23.361 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.361 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.361 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.361 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.361 INFO:teuthology.orchestra.run.smithi012.stdout: "container_init": { 2024-02-18T11:53:23.361 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_init", 2024-02-18T11:53:23.361 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.361 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.361 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.361 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.361 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.362 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.362 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.362 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Run podman/docker with `--init`", 2024-02-18T11:53:23.362 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.362 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.362 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.362 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.362 INFO:teuthology.orchestra.run.smithi012.stdout: "daemon_cache_timeout": { 2024-02-18T11:53:23.362 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "daemon_cache_timeout", 2024-02-18T11:53:23.362 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-02-18T11:53:23.362 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.362 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.362 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-02-18T11:53:23.362 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.362 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.362 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.363 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-02-18T11:53:23.363 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.363 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.363 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.363 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.363 INFO:teuthology.orchestra.run.smithi012.stdout: "default_cephadm_command_timeout": { 2024-02-18T11:53:23.363 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "default_cephadm_command_timeout", 2024-02-18T11:53:23.363 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.363 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.363 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.363 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "900", 2024-02-18T11:53:23.363 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.363 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.363 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.363 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-02-18T11:53:23.363 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.364 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.364 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.364 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.364 INFO:teuthology.orchestra.run.smithi012.stdout: "default_registry": { 2024-02-18T11:53:23.364 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "default_registry", 2024-02-18T11:53:23.364 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.364 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.364 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.364 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io", 2024-02-18T11:53:23.364 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.364 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.364 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.364 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-02-18T11:53:23.365 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.365 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.368 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.368 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.368 INFO:teuthology.orchestra.run.smithi012.stdout: "device_cache_timeout": { 2024-02-18T11:53:23.368 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "device_cache_timeout", 2024-02-18T11:53:23.368 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-02-18T11:53:23.368 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.368 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.368 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "1800", 2024-02-18T11:53:23.368 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.368 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.368 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.368 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "seconds to cache device inventory", 2024-02-18T11:53:23.369 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.369 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.369 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.369 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.369 INFO:teuthology.orchestra.run.smithi012.stdout: "device_enhanced_scan": { 2024-02-18T11:53:23.369 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "device_enhanced_scan", 2024-02-18T11:53:23.369 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.369 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.369 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.369 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.369 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.369 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.369 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.369 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use libstoragemgmt during device scans", 2024-02-18T11:53:23.369 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.370 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.370 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.370 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.370 INFO:teuthology.orchestra.run.smithi012.stdout: "facts_cache_timeout": { 2024-02-18T11:53:23.370 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "facts_cache_timeout", 2024-02-18T11:53:23.370 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-02-18T11:53:23.370 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.370 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.370 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-02-18T11:53:23.370 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.370 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.370 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.370 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "seconds to cache host facts data", 2024-02-18T11:53:23.370 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.371 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.371 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.371 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.371 INFO:teuthology.orchestra.run.smithi012.stdout: "host_check_interval": { 2024-02-18T11:53:23.371 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "host_check_interval", 2024-02-18T11:53:23.371 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-02-18T11:53:23.371 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.371 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.371 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-02-18T11:53:23.371 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.371 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.371 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.371 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to perform a host check", 2024-02-18T11:53:23.371 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.371 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.372 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.372 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.372 INFO:teuthology.orchestra.run.smithi012.stdout: "hw_monitoring": { 2024-02-18T11:53:23.372 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "hw_monitoring", 2024-02-18T11:53:23.372 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.372 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.372 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.372 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.372 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.372 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.372 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.372 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-02-18T11:53:23.372 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.372 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.372 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.373 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.373 INFO:teuthology.orchestra.run.smithi012.stdout: "inventory_list_all": { 2024-02-18T11:53:23.373 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "inventory_list_all", 2024-02-18T11:53:23.373 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.373 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.373 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.373 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.373 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.373 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.373 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.373 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-02-18T11:53:23.373 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.373 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.373 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.373 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.374 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.374 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.374 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.374 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.374 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.374 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.374 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.374 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.374 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.374 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.374 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.374 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.374 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.374 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.374 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.374 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.375 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.375 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.375 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.375 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.375 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.375 INFO:teuthology.orchestra.run.smithi012.stdout: "log_refresh_metadata": { 2024-02-18T11:53:23.375 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_refresh_metadata", 2024-02-18T11:53:23.375 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.375 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.375 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.375 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.375 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.375 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.375 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.375 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-02-18T11:53:23.376 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.376 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.376 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.376 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.376 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.376 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.376 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.376 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.376 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.376 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.376 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.376 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.376 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.376 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-02-18T11:53:23.376 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.377 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.377 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.377 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.377 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.377 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.377 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.377 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.377 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.377 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.377 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.377 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.377 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.377 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.377 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.377 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.378 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.378 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.378 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.378 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.378 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.378 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.378 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.378 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.378 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.378 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.378 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.378 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.378 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.378 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.378 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.378 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.379 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.379 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.379 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.379 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.379 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.379 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.379 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.379 INFO:teuthology.orchestra.run.smithi012.stdout: "manage_etc_ceph_ceph_conf": { 2024-02-18T11:53:23.379 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-02-18T11:53:23.379 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.379 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.379 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.379 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.379 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.380 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.380 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.380 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-02-18T11:53:23.380 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.380 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.380 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.380 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.380 INFO:teuthology.orchestra.run.smithi012.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-02-18T11:53:23.380 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-02-18T11:53:23.380 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.380 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.380 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.380 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "*", 2024-02-18T11:53:23.380 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.381 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.381 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.381 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-02-18T11:53:23.381 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.381 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.381 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.381 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.381 INFO:teuthology.orchestra.run.smithi012.stdout: "max_count_per_host": { 2024-02-18T11:53:23.381 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_count_per_host", 2024-02-18T11:53:23.381 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.381 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.381 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.381 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-02-18T11:53:23.381 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.381 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.381 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.382 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "max number of daemons per service per host", 2024-02-18T11:53:23.382 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.382 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.382 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.382 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.382 INFO:teuthology.orchestra.run.smithi012.stdout: "max_osd_draining_count": { 2024-02-18T11:53:23.382 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_osd_draining_count", 2024-02-18T11:53:23.382 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.382 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.382 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.382 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-02-18T11:53:23.382 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.382 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.382 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.382 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-02-18T11:53:23.383 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.383 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.383 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.383 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.383 INFO:teuthology.orchestra.run.smithi012.stdout: "migration_current": { 2024-02-18T11:53:23.383 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "migration_current", 2024-02-18T11:53:23.383 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.383 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.383 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.383 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.383 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.383 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.383 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.383 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "internal - do not modify", 2024-02-18T11:53:23.383 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.384 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.384 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.384 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.384 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": { 2024-02-18T11:53:23.384 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mode", 2024-02-18T11:53:23.384 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.384 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.384 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.384 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "root", 2024-02-18T11:53:23.384 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.384 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.384 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.384 INFO:teuthology.orchestra.run.smithi012.stdout: "cephadm-package", 2024-02-18T11:53:23.384 INFO:teuthology.orchestra.run.smithi012.stdout: "root" 2024-02-18T11:53:23.384 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.385 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "mode for remote execution of cephadm", 2024-02-18T11:53:23.385 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.385 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.385 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.385 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.385 INFO:teuthology.orchestra.run.smithi012.stdout: "oob_default_addr": { 2024-02-18T11:53:23.385 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "oob_default_addr", 2024-02-18T11:53:23.385 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.385 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.385 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.385 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "169.254.1.1", 2024-02-18T11:53:23.385 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.385 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.385 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.385 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Default address for RedFish API (oob management).", 2024-02-18T11:53:23.386 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.386 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.386 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.386 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.386 INFO:teuthology.orchestra.run.smithi012.stdout: "prometheus_alerts_path": { 2024-02-18T11:53:23.386 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "prometheus_alerts_path", 2024-02-18T11:53:23.386 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.386 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.386 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.386 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-02-18T11:53:23.386 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.386 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.386 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.386 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-02-18T11:53:23.386 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.387 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.387 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.387 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.387 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_insecure": { 2024-02-18T11:53:23.387 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_insecure", 2024-02-18T11:53:23.387 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.387 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.387 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.387 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.387 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.387 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.387 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.387 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-02-18T11:53:23.387 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.387 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.388 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.388 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.388 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_password": { 2024-02-18T11:53:23.388 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_password", 2024-02-18T11:53:23.388 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.388 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.388 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.388 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.388 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.388 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.388 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.388 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-02-18T11:53:23.388 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.388 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.389 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.389 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.389 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_url": { 2024-02-18T11:53:23.389 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_url", 2024-02-18T11:53:23.389 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.389 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.389 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.389 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.389 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.389 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.389 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.389 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-02-18T11:53:23.389 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.389 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.389 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.390 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.390 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_username": { 2024-02-18T11:53:23.390 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_username", 2024-02-18T11:53:23.390 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.390 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.390 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.390 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.390 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.390 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.390 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.390 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-02-18T11:53:23.390 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.390 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.390 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.390 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.391 INFO:teuthology.orchestra.run.smithi012.stdout: "secure_monitoring_stack": { 2024-02-18T11:53:23.391 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "secure_monitoring_stack", 2024-02-18T11:53:23.391 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.391 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.391 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.391 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.391 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.391 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.391 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.391 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-02-18T11:53:23.391 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.391 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.391 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.391 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.391 INFO:teuthology.orchestra.run.smithi012.stdout: "service_discovery_port": { 2024-02-18T11:53:23.392 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "service_discovery_port", 2024-02-18T11:53:23.392 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.392 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.392 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.392 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8765", 2024-02-18T11:53:23.392 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.392 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.392 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.392 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "cephadm service discovery port", 2024-02-18T11:53:23.392 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.392 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.392 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.392 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.392 INFO:teuthology.orchestra.run.smithi012.stdout: "ssh_config_file": { 2024-02-18T11:53:23.393 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssh_config_file", 2024-02-18T11:53:23.393 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.393 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.393 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.393 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.393 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.393 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.393 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.393 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-02-18T11:53:23.393 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.393 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.393 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.393 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.393 INFO:teuthology.orchestra.run.smithi012.stdout: "use_agent": { 2024-02-18T11:53:23.393 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "use_agent", 2024-02-18T11:53:23.393 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.394 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.394 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.394 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.394 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.394 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.394 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.394 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-02-18T11:53:23.394 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.394 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.394 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.394 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.394 INFO:teuthology.orchestra.run.smithi012.stdout: "use_repo_digest": { 2024-02-18T11:53:23.394 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "use_repo_digest", 2024-02-18T11:53:23.394 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.394 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.395 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.395 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.395 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.395 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.395 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.395 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-02-18T11:53:23.395 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.395 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.395 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.395 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.395 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_on_failed_host_check": { 2024-02-18T11:53:23.395 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_on_failed_host_check", 2024-02-18T11:53:23.395 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.395 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.395 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.396 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.396 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.396 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.396 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.396 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise a health warning if the host check fails", 2024-02-18T11:53:23.396 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.396 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.396 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.396 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.396 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_on_stray_daemons": { 2024-02-18T11:53:23.396 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_on_stray_daemons", 2024-02-18T11:53:23.396 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.396 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.396 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.396 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.397 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.397 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.397 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.397 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-02-18T11:53:23.397 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.397 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.397 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.397 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.397 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_on_stray_hosts": { 2024-02-18T11:53:23.397 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_on_stray_hosts", 2024-02-18T11:53:23.397 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.397 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.397 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.397 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.397 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.397 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.398 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.398 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-02-18T11:53:23.398 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.398 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.398 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.398 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.398 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.398 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.398 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.398 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crash", 2024-02-18T11:53:23.398 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.398 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.398 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.398 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.398 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.399 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.399 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.399 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.399 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.399 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.399 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.399 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.399 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.399 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.399 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.399 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.399 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.399 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.399 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.399 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.399 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.400 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.400 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.400 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.400 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.400 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.400 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.400 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.400 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.400 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.400 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.400 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.400 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.400 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.400 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.400 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.401 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.401 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.401 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.401 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.401 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.401 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.401 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.401 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.401 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.401 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.401 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.401 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.401 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.401 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.401 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.401 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.402 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.402 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.402 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.402 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.402 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.402 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.402 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.402 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.402 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.402 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.402 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.402 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.402 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.402 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.403 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.403 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.403 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.403 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.403 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.403 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.403 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.403 INFO:teuthology.orchestra.run.smithi012.stdout: "retain_interval": { 2024-02-18T11:53:23.403 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "retain_interval", 2024-02-18T11:53:23.403 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-02-18T11:53:23.403 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.403 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.403 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "31536000", 2024-02-18T11:53:23.403 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.403 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.404 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.404 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how long to retain crashes before pruning them", 2024-02-18T11:53:23.404 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.404 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.404 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.404 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.404 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_recent_interval": { 2024-02-18T11:53:23.404 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_recent_interval", 2024-02-18T11:53:23.404 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-02-18T11:53:23.404 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.404 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.404 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "1209600", 2024-02-18T11:53:23.404 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.404 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.404 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.405 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "time interval in which to warn about recent crashes", 2024-02-18T11:53:23.405 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.405 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.405 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.405 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.405 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.405 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.405 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.405 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "dashboard", 2024-02-18T11:53:23.405 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.405 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.405 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.405 INFO:teuthology.orchestra.run.smithi012.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-02-18T11:53:23.405 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-02-18T11:53:23.405 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.405 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.406 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.406 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-02-18T11:53:23.406 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.406 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.406 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.406 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.406 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.406 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.406 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.406 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.406 INFO:teuthology.orchestra.run.smithi012.stdout: "ALERTMANAGER_API_HOST": { 2024-02-18T11:53:23.406 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ALERTMANAGER_API_HOST", 2024-02-18T11:53:23.406 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.406 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.406 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.407 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.407 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.407 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.407 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.407 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.407 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.407 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.407 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.407 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.407 INFO:teuthology.orchestra.run.smithi012.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-02-18T11:53:23.407 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-02-18T11:53:23.407 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.407 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.407 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.407 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.408 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.408 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.408 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.408 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.408 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.408 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.408 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.408 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.408 INFO:teuthology.orchestra.run.smithi012.stdout: "AUDIT_API_ENABLED": { 2024-02-18T11:53:23.408 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "AUDIT_API_ENABLED", 2024-02-18T11:53:23.408 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.408 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.408 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.408 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.409 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.409 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.409 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.409 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.409 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.409 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.409 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.409 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.409 INFO:teuthology.orchestra.run.smithi012.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-02-18T11:53:23.409 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-02-18T11:53:23.409 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.409 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.409 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.409 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.409 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.409 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.410 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.410 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.410 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.410 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.410 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.410 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.410 INFO:teuthology.orchestra.run.smithi012.stdout: "ENABLE_BROWSABLE_API": { 2024-02-18T11:53:23.410 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ENABLE_BROWSABLE_API", 2024-02-18T11:53:23.410 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.410 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.410 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.410 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.410 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.410 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.410 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.411 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.411 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.411 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.411 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.411 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.411 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-02-18T11:53:23.411 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-02-18T11:53:23.411 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.411 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.411 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.411 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.411 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.411 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.411 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.412 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.412 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.412 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.412 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.412 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.412 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-02-18T11:53:23.412 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-02-18T11:53:23.412 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.412 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.412 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.412 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.412 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.412 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.412 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.412 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.413 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.413 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.413 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.413 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.413 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-02-18T11:53:23.413 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-02-18T11:53:23.413 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.413 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.413 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.413 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.413 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.413 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.413 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.413 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.413 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.413 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.414 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.414 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.414 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-02-18T11:53:23.414 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-02-18T11:53:23.414 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.414 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.414 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.414 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.414 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.414 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.414 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.414 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.414 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.414 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.414 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.414 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.415 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_NFS": { 2024-02-18T11:53:23.415 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-02-18T11:53:23.415 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.415 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.415 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.415 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.415 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.415 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.415 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.415 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.415 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.415 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.415 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.415 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.416 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_RBD": { 2024-02-18T11:53:23.416 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-02-18T11:53:23.416 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.416 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.416 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.416 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.416 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.416 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.416 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.416 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.416 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.416 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.416 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.416 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.417 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_RGW": { 2024-02-18T11:53:23.417 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-02-18T11:53:23.417 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.417 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.417 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.417 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.417 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.417 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.417 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.417 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.417 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.417 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.417 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.417 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.417 INFO:teuthology.orchestra.run.smithi012.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-02-18T11:53:23.417 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-02-18T11:53:23.418 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.418 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.418 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.418 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.418 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.418 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.418 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.418 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.418 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.418 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.418 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.418 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.418 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_PASSWORD": { 2024-02-18T11:53:23.418 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_PASSWORD", 2024-02-18T11:53:23.419 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.419 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.419 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.419 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "admin", 2024-02-18T11:53:23.419 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.419 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.419 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.419 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.419 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.419 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.419 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.419 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.419 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-02-18T11:53:23.419 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-02-18T11:53:23.419 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.420 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.420 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.420 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.420 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.420 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.420 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.420 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.420 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.420 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.420 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.420 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.420 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_URL": { 2024-02-18T11:53:23.420 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_URL", 2024-02-18T11:53:23.420 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.420 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.421 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.421 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.421 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.421 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.421 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.421 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.421 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.421 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.421 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.421 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.421 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_USERNAME": { 2024-02-18T11:53:23.421 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_USERNAME", 2024-02-18T11:53:23.421 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.421 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.421 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.422 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "admin", 2024-02-18T11:53:23.422 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.422 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.422 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.422 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.422 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.422 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.422 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.422 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.422 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-02-18T11:53:23.422 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-02-18T11:53:23.422 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.422 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.422 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.422 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.422 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.423 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.423 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.423 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.423 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.423 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.423 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.423 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.423 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-02-18T11:53:23.423 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-02-18T11:53:23.423 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.423 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.423 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.423 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.423 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.423 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.424 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.424 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.424 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.424 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.424 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.424 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.424 INFO:teuthology.orchestra.run.smithi012.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-02-18T11:53:23.424 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-02-18T11:53:23.424 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.424 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.424 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.424 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.424 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.424 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.424 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.424 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.425 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.425 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.425 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.425 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.425 INFO:teuthology.orchestra.run.smithi012.stdout: "ISSUE_TRACKER_API_KEY": { 2024-02-18T11:53:23.425 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-02-18T11:53:23.425 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.425 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.425 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.425 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.425 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.425 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.425 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.425 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.425 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.426 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.426 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.426 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.426 INFO:teuthology.orchestra.run.smithi012.stdout: "PROMETHEUS_API_HOST": { 2024-02-18T11:53:23.426 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PROMETHEUS_API_HOST", 2024-02-18T11:53:23.426 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.426 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.426 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.426 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.426 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.426 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.426 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.426 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.426 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.426 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.426 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.427 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.427 INFO:teuthology.orchestra.run.smithi012.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-02-18T11:53:23.427 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-02-18T11:53:23.427 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.427 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.427 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.427 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.427 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.427 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.427 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.427 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.427 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.427 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.427 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.428 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.428 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-02-18T11:53:23.428 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-02-18T11:53:23.428 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.428 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.428 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.428 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.428 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.428 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.428 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.428 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.428 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.428 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.428 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.428 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.429 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-02-18T11:53:23.429 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-02-18T11:53:23.429 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.429 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.429 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.429 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.429 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.429 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.429 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.429 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.429 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.429 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.429 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.429 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.429 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-02-18T11:53:23.430 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-02-18T11:53:23.430 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.430 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.430 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.430 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.430 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.430 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.430 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.430 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.430 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.430 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.430 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.430 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.430 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-02-18T11:53:23.430 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-02-18T11:53:23.430 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.431 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.431 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.431 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.431 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.431 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.431 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.431 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.431 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.431 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.431 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.431 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.431 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-02-18T11:53:23.431 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-02-18T11:53:23.431 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.431 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.432 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.432 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.432 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.432 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.432 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.432 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.432 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.432 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.432 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.432 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.432 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-02-18T11:53:23.432 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-02-18T11:53:23.432 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.432 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.433 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.433 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.433 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.433 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.433 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.433 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.433 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.433 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.433 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.433 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.433 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-02-18T11:53:23.433 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-02-18T11:53:23.433 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.433 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.433 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.433 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.434 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.434 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.434 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.434 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.434 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.434 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.434 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.434 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.434 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_ENABLED": { 2024-02-18T11:53:23.434 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_ENABLED", 2024-02-18T11:53:23.434 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.434 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.434 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.434 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.434 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.435 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.435 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.435 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.435 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.435 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.435 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.435 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.435 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-02-18T11:53:23.435 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-02-18T11:53:23.435 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.435 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.435 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.435 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-02-18T11:53:23.435 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.435 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.435 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.436 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.436 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.436 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.436 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.436 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.436 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-02-18T11:53:23.436 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-02-18T11:53:23.436 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.436 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.436 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.436 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-02-18T11:53:23.436 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.436 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.436 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.436 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.437 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.437 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.437 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.437 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.437 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-02-18T11:53:23.437 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-02-18T11:53:23.437 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.437 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.437 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.437 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8", 2024-02-18T11:53:23.437 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.437 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.437 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.437 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.437 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.438 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.438 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.438 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.438 INFO:teuthology.orchestra.run.smithi012.stdout: "REST_REQUESTS_TIMEOUT": { 2024-02-18T11:53:23.438 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-02-18T11:53:23.438 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.438 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.438 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.438 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "45", 2024-02-18T11:53:23.438 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.438 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.438 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.438 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.438 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.438 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.438 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.439 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.439 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_ACCESS_KEY": { 2024-02-18T11:53:23.439 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_ACCESS_KEY", 2024-02-18T11:53:23.439 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.439 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.439 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.439 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.439 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.439 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.439 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.439 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.439 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.439 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.439 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.440 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.440 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-02-18T11:53:23.440 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-02-18T11:53:23.440 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.440 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.440 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.440 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "admin", 2024-02-18T11:53:23.440 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.440 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.440 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.440 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.440 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.440 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.440 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.441 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.441 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_SECRET_KEY": { 2024-02-18T11:53:23.441 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_SECRET_KEY", 2024-02-18T11:53:23.441 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.441 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.441 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.441 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.441 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.441 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.441 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.441 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.441 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.441 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.441 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.441 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.441 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_SSL_VERIFY": { 2024-02-18T11:53:23.442 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_SSL_VERIFY", 2024-02-18T11:53:23.442 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.442 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.442 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.442 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.442 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.442 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.442 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.442 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.442 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.442 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.442 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.442 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.442 INFO:teuthology.orchestra.run.smithi012.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-02-18T11:53:23.443 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-02-18T11:53:23.443 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.443 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.443 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.443 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-02-18T11:53:23.443 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.443 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.443 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.443 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.443 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.443 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.443 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.443 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.443 INFO:teuthology.orchestra.run.smithi012.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-02-18T11:53:23.443 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-02-18T11:53:23.444 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.444 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.444 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.444 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-02-18T11:53:23.444 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.444 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.444 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.444 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.444 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.444 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.444 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.444 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.444 INFO:teuthology.orchestra.run.smithi012.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-02-18T11:53:23.444 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-02-18T11:53:23.444 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.445 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.445 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.445 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-02-18T11:53:23.445 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.445 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.445 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.445 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.445 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.445 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.445 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.445 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.445 INFO:teuthology.orchestra.run.smithi012.stdout: "cross_origin_url": { 2024-02-18T11:53:23.445 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cross_origin_url", 2024-02-18T11:53:23.445 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.445 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.446 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.446 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.446 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.446 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.446 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.446 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.446 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.446 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.446 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.446 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.446 INFO:teuthology.orchestra.run.smithi012.stdout: "crt_file": { 2024-02-18T11:53:23.446 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crt_file", 2024-02-18T11:53:23.446 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.446 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.446 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.446 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.447 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.447 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.447 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.447 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.447 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.447 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.447 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.447 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.447 INFO:teuthology.orchestra.run.smithi012.stdout: "debug": { 2024-02-18T11:53:23.447 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "debug", 2024-02-18T11:53:23.447 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.447 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.447 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.447 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.447 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.448 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.448 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.448 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Enable/disable debug options", 2024-02-18T11:53:23.448 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.448 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.448 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.448 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.448 INFO:teuthology.orchestra.run.smithi012.stdout: "jwt_token_ttl": { 2024-02-18T11:53:23.448 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "jwt_token_ttl", 2024-02-18T11:53:23.448 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.448 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.448 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.448 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "28800", 2024-02-18T11:53:23.448 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.448 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.448 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.449 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.449 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.449 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.449 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.449 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.449 INFO:teuthology.orchestra.run.smithi012.stdout: "key_file": { 2024-02-18T11:53:23.449 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "key_file", 2024-02-18T11:53:23.449 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.449 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.449 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.449 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.449 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.449 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.449 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.449 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.450 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.450 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.450 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.450 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.450 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.450 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.450 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.450 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.450 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.450 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.450 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.450 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.450 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.450 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.450 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.450 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.451 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.451 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.451 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.451 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.451 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.451 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.451 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.451 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.451 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.451 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.451 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.451 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.451 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.451 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.452 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.452 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.452 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.452 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.452 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.452 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.452 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.452 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.452 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.452 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.452 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.452 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.452 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.452 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.452 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.453 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.453 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.453 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.453 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.453 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.453 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.453 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.453 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.453 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.453 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.453 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.453 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.453 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.453 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.453 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.454 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.454 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.454 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.454 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.454 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.454 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.454 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.454 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.454 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.454 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.454 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.454 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.454 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.454 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.455 INFO:teuthology.orchestra.run.smithi012.stdout: "motd": { 2024-02-18T11:53:23.455 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "motd", 2024-02-18T11:53:23.455 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.455 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.455 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.455 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.455 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.455 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.455 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.455 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "The message of the day", 2024-02-18T11:53:23.455 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.455 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.455 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.455 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.455 INFO:teuthology.orchestra.run.smithi012.stdout: "redirect_resolve_ip_addr": { 2024-02-18T11:53:23.455 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "redirect_resolve_ip_addr", 2024-02-18T11:53:23.456 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.456 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.456 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.456 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.456 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.456 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.456 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.456 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.456 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.456 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.456 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.456 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.456 INFO:teuthology.orchestra.run.smithi012.stdout: "server_addr": { 2024-02-18T11:53:23.456 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_addr", 2024-02-18T11:53:23.457 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.457 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.457 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.457 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "::", 2024-02-18T11:53:23.457 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.457 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.457 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.457 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.457 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.457 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.457 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.457 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.457 INFO:teuthology.orchestra.run.smithi012.stdout: "server_port": { 2024-02-18T11:53:23.457 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_port", 2024-02-18T11:53:23.457 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.457 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.458 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.458 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8080", 2024-02-18T11:53:23.458 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.458 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.458 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.458 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.458 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.458 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.458 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.458 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.458 INFO:teuthology.orchestra.run.smithi012.stdout: "ssl": { 2024-02-18T11:53:23.458 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssl", 2024-02-18T11:53:23.458 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.458 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.458 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.458 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.459 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.459 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.459 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.459 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.459 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.459 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.459 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.459 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.459 INFO:teuthology.orchestra.run.smithi012.stdout: "ssl_server_port": { 2024-02-18T11:53:23.459 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssl_server_port", 2024-02-18T11:53:23.459 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.459 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.459 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.459 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8443", 2024-02-18T11:53:23.459 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.460 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.460 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.460 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.460 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.460 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.460 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.460 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.460 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_behaviour": { 2024-02-18T11:53:23.460 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_behaviour", 2024-02-18T11:53:23.460 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.460 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.460 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.460 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "redirect", 2024-02-18T11:53:23.460 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.460 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.461 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.461 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.461 INFO:teuthology.orchestra.run.smithi012.stdout: "redirect" 2024-02-18T11:53:23.461 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.461 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.461 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.461 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.461 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.461 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.461 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_error_status_code": { 2024-02-18T11:53:23.461 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_error_status_code", 2024-02-18T11:53:23.461 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.461 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.461 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.461 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "500", 2024-02-18T11:53:23.461 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "400", 2024-02-18T11:53:23.462 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "599", 2024-02-18T11:53:23.462 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.462 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.462 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.462 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.462 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.462 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.462 INFO:teuthology.orchestra.run.smithi012.stdout: "url_prefix": { 2024-02-18T11:53:23.462 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "url_prefix", 2024-02-18T11:53:23.462 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.462 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.462 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.462 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.462 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.462 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.463 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.463 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.463 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.463 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.463 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.463 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.463 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.463 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.463 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.463 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "devicehealth", 2024-02-18T11:53:23.463 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.463 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.463 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.463 INFO:teuthology.orchestra.run.smithi012.stdout: "enable_monitoring": { 2024-02-18T11:53:23.463 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enable_monitoring", 2024-02-18T11:53:23.463 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.464 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.464 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.464 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.464 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.464 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.464 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.464 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "monitor device health metrics", 2024-02-18T11:53:23.464 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.464 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.464 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.464 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.464 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.464 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.465 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.465 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.465 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.465 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.465 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.465 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.465 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.465 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.465 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.465 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.465 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.465 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.465 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.465 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.465 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.465 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.466 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.466 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.466 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.466 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.466 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.466 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.466 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.466 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.466 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.466 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.466 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.466 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.466 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.466 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.466 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.467 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.467 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.467 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.467 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.467 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.467 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.467 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.467 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.467 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.467 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.467 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.467 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.467 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.467 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.467 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.468 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.468 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.468 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.468 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.468 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.468 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.468 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.468 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.468 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.468 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.468 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.468 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.468 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.468 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.468 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.469 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.469 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.469 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.469 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.469 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.469 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.469 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.469 INFO:teuthology.orchestra.run.smithi012.stdout: "mark_out_threshold": { 2024-02-18T11:53:23.469 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mark_out_threshold", 2024-02-18T11:53:23.469 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-02-18T11:53:23.469 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.469 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.469 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "2419200", 2024-02-18T11:53:23.469 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.469 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.470 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.470 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-02-18T11:53:23.470 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.470 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.470 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.470 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.470 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_name": { 2024-02-18T11:53:23.470 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pool_name", 2024-02-18T11:53:23.470 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.470 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.470 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.470 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "device_health_metrics", 2024-02-18T11:53:23.470 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.470 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.470 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.471 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "name of pool in which to store device health metrics", 2024-02-18T11:53:23.471 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.471 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.471 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.471 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.471 INFO:teuthology.orchestra.run.smithi012.stdout: "retention_period": { 2024-02-18T11:53:23.471 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "retention_period", 2024-02-18T11:53:23.471 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-02-18T11:53:23.471 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.471 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.471 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "15552000", 2024-02-18T11:53:23.471 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.471 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.471 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.471 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how long to retain device health metrics", 2024-02-18T11:53:23.471 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.472 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.472 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.472 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.472 INFO:teuthology.orchestra.run.smithi012.stdout: "scrape_frequency": { 2024-02-18T11:53:23.472 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "scrape_frequency", 2024-02-18T11:53:23.472 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-02-18T11:53:23.472 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.472 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.472 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "86400", 2024-02-18T11:53:23.472 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.472 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.472 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.472 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to scrape device health metrics", 2024-02-18T11:53:23.472 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.472 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.473 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.473 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.473 INFO:teuthology.orchestra.run.smithi012.stdout: "self_heal": { 2024-02-18T11:53:23.473 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "self_heal", 2024-02-18T11:53:23.473 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.473 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.473 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.473 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.473 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.473 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.473 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.473 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-02-18T11:53:23.473 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.473 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.473 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.473 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.474 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-02-18T11:53:23.474 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-02-18T11:53:23.474 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-02-18T11:53:23.474 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.474 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.474 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-02-18T11:53:23.474 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.474 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.474 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.474 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to wake up and check device health", 2024-02-18T11:53:23.474 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.474 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.474 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.474 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.474 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_threshold": { 2024-02-18T11:53:23.475 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_threshold", 2024-02-18T11:53:23.475 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-02-18T11:53:23.475 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.475 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.475 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "7257600", 2024-02-18T11:53:23.475 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.475 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.475 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.475 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-02-18T11:53:23.475 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.475 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.475 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.475 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.475 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.475 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.475 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.476 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "diskprediction_local", 2024-02-18T11:53:23.476 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.476 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.476 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.476 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.476 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.476 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.476 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.476 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.476 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.476 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.476 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.476 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.477 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.477 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.477 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.477 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.477 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.477 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.477 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.477 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.477 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.477 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.477 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.477 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.477 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.477 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.477 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.478 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.478 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.478 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.478 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.478 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.478 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.478 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.478 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.478 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.478 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.478 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.478 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.478 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.478 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.478 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.478 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.479 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.479 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.479 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.479 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.479 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.479 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.479 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.479 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.479 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.479 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.479 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.479 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.479 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.479 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.479 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.480 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.480 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.480 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.480 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.480 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.480 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.480 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.480 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.480 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.480 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.480 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.480 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.480 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.480 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.481 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.481 INFO:teuthology.orchestra.run.smithi012.stdout: "predict_interval": { 2024-02-18T11:53:23.481 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "predict_interval", 2024-02-18T11:53:23.481 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.481 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.481 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.481 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "86400", 2024-02-18T11:53:23.481 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.481 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.481 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.481 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.481 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.481 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.481 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.481 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.482 INFO:teuthology.orchestra.run.smithi012.stdout: "predictor_model": { 2024-02-18T11:53:23.482 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "predictor_model", 2024-02-18T11:53:23.482 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.482 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.482 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.482 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "prophetstor", 2024-02-18T11:53:23.482 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.482 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.482 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.482 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.482 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.482 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.482 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.482 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.482 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-02-18T11:53:23.483 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-02-18T11:53:23.483 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.483 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.483 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.483 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-02-18T11:53:23.483 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.483 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.483 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.483 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.483 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.483 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.483 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.483 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.483 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.483 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.483 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.484 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "influx", 2024-02-18T11:53:23.484 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": false, 2024-02-18T11:53:23.484 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "influxdb python module not found", 2024-02-18T11:53:23.484 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.484 INFO:teuthology.orchestra.run.smithi012.stdout: "batch_size": { 2024-02-18T11:53:23.484 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "batch_size", 2024-02-18T11:53:23.484 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.484 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.484 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.484 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5000", 2024-02-18T11:53:23.484 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.484 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.484 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.485 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-02-18T11:53:23.485 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.485 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.485 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.485 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.485 INFO:teuthology.orchestra.run.smithi012.stdout: "database": { 2024-02-18T11:53:23.485 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "database", 2024-02-18T11:53:23.485 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.485 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.485 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.485 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "ceph", 2024-02-18T11:53:23.485 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.485 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.485 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.485 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-02-18T11:53:23.486 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.486 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.486 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.486 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.486 INFO:teuthology.orchestra.run.smithi012.stdout: "hostname": { 2024-02-18T11:53:23.486 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "hostname", 2024-02-18T11:53:23.486 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.486 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.486 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.486 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.486 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.486 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.486 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.486 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "InfluxDB server hostname", 2024-02-18T11:53:23.486 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.487 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.487 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.487 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.487 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-02-18T11:53:23.487 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-02-18T11:53:23.487 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-02-18T11:53:23.487 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.487 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.487 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "30", 2024-02-18T11:53:23.487 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "5", 2024-02-18T11:53:23.487 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.487 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.487 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-02-18T11:53:23.487 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.487 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.487 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.488 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.488 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.488 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.488 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.488 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.488 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.488 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.488 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.488 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.488 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.488 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.488 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.488 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.488 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.488 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.489 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.489 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.489 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.489 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.489 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.489 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.489 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.489 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.489 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.489 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.489 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.489 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.489 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.490 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.490 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.490 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.490 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.490 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.490 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.490 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.490 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.490 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.490 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.490 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.490 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.490 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.490 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.490 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.491 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.491 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.491 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.491 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.491 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.491 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.491 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.491 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.491 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.491 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.491 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.491 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.491 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.491 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.491 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.491 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.492 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.492 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.492 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.492 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.492 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.492 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.492 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.492 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.492 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.492 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.492 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.492 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.492 INFO:teuthology.orchestra.run.smithi012.stdout: "password": { 2024-02-18T11:53:23.492 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "password", 2024-02-18T11:53:23.493 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.493 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.493 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.493 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.493 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.493 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.493 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.493 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "password of InfluxDB server user", 2024-02-18T11:53:23.493 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.493 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.493 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.493 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.493 INFO:teuthology.orchestra.run.smithi012.stdout: "port": { 2024-02-18T11:53:23.493 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "port", 2024-02-18T11:53:23.493 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.494 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.494 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.494 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8086", 2024-02-18T11:53:23.494 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.494 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.494 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.494 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "InfluxDB server port", 2024-02-18T11:53:23.494 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.494 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.494 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.494 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.494 INFO:teuthology.orchestra.run.smithi012.stdout: "ssl": { 2024-02-18T11:53:23.494 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssl", 2024-02-18T11:53:23.494 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.494 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.495 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.495 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "false", 2024-02-18T11:53:23.495 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.495 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.495 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.495 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-02-18T11:53:23.495 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.495 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.495 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.495 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.495 INFO:teuthology.orchestra.run.smithi012.stdout: "threads": { 2024-02-18T11:53:23.495 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "threads", 2024-02-18T11:53:23.495 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.495 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.495 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.496 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-02-18T11:53:23.496 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-02-18T11:53:23.496 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "32", 2024-02-18T11:53:23.496 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.496 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-02-18T11:53:23.496 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.496 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.496 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.496 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.496 INFO:teuthology.orchestra.run.smithi012.stdout: "username": { 2024-02-18T11:53:23.496 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "username", 2024-02-18T11:53:23.496 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.496 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.496 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.496 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.497 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.497 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.497 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.497 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "username of InfluxDB server user", 2024-02-18T11:53:23.497 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.497 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.497 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.497 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.497 INFO:teuthology.orchestra.run.smithi012.stdout: "verify_ssl": { 2024-02-18T11:53:23.497 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "verify_ssl", 2024-02-18T11:53:23.497 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.497 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.497 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.497 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "true", 2024-02-18T11:53:23.497 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.498 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.498 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.498 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-02-18T11:53:23.498 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.498 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.498 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.498 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.498 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.498 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.498 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.498 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "insights", 2024-02-18T11:53:23.498 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.498 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.498 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.498 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.498 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.499 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.499 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.499 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.499 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.499 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.499 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.499 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.499 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.499 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.499 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.499 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.499 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.499 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.499 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.499 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.500 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.500 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.500 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.500 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.500 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.500 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.500 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.500 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.500 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.500 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.500 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.500 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.500 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.500 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.500 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.501 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.501 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.501 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.501 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.501 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.501 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.501 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.501 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.501 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.501 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.501 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.501 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.501 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.501 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.502 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.502 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.502 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.502 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.502 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.502 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.502 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.502 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.502 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.502 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.502 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.502 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.502 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.502 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.502 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.502 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.503 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.503 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.503 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.503 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.503 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.503 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.503 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.503 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.503 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.503 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.503 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.503 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "iostat", 2024-02-18T11:53:23.503 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.503 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.503 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.503 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.504 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.504 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.504 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.504 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.504 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.504 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.504 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.504 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.504 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.504 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.504 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.504 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.504 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.504 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.504 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.505 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.505 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.505 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.505 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.505 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.505 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.505 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.505 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.505 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.505 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.505 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.505 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.505 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.505 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.505 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.506 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.506 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.506 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.506 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.506 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.506 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.506 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.506 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.506 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.506 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.506 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.506 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.506 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.506 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.506 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.507 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.507 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.507 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.507 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.507 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.507 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.507 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.507 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.507 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.507 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.507 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.507 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.507 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.507 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.507 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.508 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.508 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.508 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.508 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.508 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.508 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.508 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.508 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.508 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.508 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.508 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.508 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.508 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "k8sevents", 2024-02-18T11:53:23.508 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.508 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.509 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.509 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_event_retention_days": { 2024-02-18T11:53:23.509 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ceph_event_retention_days", 2024-02-18T11:53:23.509 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.509 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.509 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.509 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "7", 2024-02-18T11:53:23.509 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.509 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.509 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.509 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Days to hold ceph event information within local cache", 2024-02-18T11:53:23.509 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.509 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.509 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.509 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.509 INFO:teuthology.orchestra.run.smithi012.stdout: "config_check_secs": { 2024-02-18T11:53:23.510 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "config_check_secs", 2024-02-18T11:53:23.510 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.510 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.510 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.510 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-02-18T11:53:23.510 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "10", 2024-02-18T11:53:23.510 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.510 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.510 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-02-18T11:53:23.510 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.510 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.510 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.510 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.510 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.510 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.511 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.511 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.511 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.511 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.511 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.511 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.511 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.511 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.511 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.511 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.511 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.511 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.511 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.511 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.511 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.511 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.512 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.512 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.512 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.512 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.512 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.512 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.512 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.512 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.512 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.512 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.512 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.512 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.512 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.512 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.512 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.513 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.513 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.513 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.513 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.513 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.513 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.513 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.513 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.513 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.513 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.513 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.513 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.513 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.513 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.513 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.513 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.514 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.514 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.514 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.514 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.514 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.514 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.514 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.514 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.514 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.514 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.514 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.514 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.514 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.514 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.514 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.515 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.515 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.515 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.515 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.515 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.515 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.515 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.515 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.515 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.515 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "localpool", 2024-02-18T11:53:23.515 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.515 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.515 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.515 INFO:teuthology.orchestra.run.smithi012.stdout: "failure_domain": { 2024-02-18T11:53:23.515 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "failure_domain", 2024-02-18T11:53:23.516 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.516 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.516 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.516 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "host", 2024-02-18T11:53:23.516 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.516 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.516 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.516 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "failure domain for any created local pool", 2024-02-18T11:53:23.516 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-02-18T11:53:23.516 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.516 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.516 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.516 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.516 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.516 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.517 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.517 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.517 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.517 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.517 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.517 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.517 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.517 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.517 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.517 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.517 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.517 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.517 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.517 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.517 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.517 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.518 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.518 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.518 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.518 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.518 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.518 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.518 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.518 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.518 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.518 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.518 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.518 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.518 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.518 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.518 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.519 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.519 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.519 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.519 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.519 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.519 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.519 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.519 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.519 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.519 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.519 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.519 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.519 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.519 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.519 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.520 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.520 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.520 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.520 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.520 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.520 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.520 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.520 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.520 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.520 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.520 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.520 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.520 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.520 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.520 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.521 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.521 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.521 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.521 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.521 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.521 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.521 INFO:teuthology.orchestra.run.smithi012.stdout: "min_size": { 2024-02-18T11:53:23.521 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "min_size", 2024-02-18T11:53:23.521 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.521 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.521 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.521 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.521 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.521 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.521 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.521 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "default min_size for any created local pool", 2024-02-18T11:53:23.522 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-02-18T11:53:23.522 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.522 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.522 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.522 INFO:teuthology.orchestra.run.smithi012.stdout: "num_rep": { 2024-02-18T11:53:23.522 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "num_rep", 2024-02-18T11:53:23.522 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.522 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.522 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.522 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "3", 2024-02-18T11:53:23.522 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.522 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.522 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.522 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "default replica count for any created local pool", 2024-02-18T11:53:23.522 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.523 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.523 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.523 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.523 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num": { 2024-02-18T11:53:23.523 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pg_num", 2024-02-18T11:53:23.523 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.523 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.523 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.523 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "128", 2024-02-18T11:53:23.523 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.523 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.523 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.523 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "default pg_num for any created local pool", 2024-02-18T11:53:23.523 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.523 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.524 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.524 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.524 INFO:teuthology.orchestra.run.smithi012.stdout: "prefix": { 2024-02-18T11:53:23.524 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "prefix", 2024-02-18T11:53:23.524 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.524 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.524 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.524 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.524 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.524 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.524 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.524 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "name prefix for any created local pool", 2024-02-18T11:53:23.524 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.524 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.524 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.525 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.525 INFO:teuthology.orchestra.run.smithi012.stdout: "subtree": { 2024-02-18T11:53:23.525 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "subtree", 2024-02-18T11:53:23.525 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.525 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.525 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.525 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "rack", 2024-02-18T11:53:23.525 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.525 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.525 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.525 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "CRUSH level for which to create a local pool", 2024-02-18T11:53:23.525 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-02-18T11:53:23.525 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.525 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.525 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.526 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.526 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.526 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.526 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mds_autoscaler", 2024-02-18T11:53:23.526 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.526 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.526 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.526 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.526 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.526 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.526 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.527 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.527 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.527 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.527 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.527 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.527 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.527 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.527 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.527 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.527 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.527 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.527 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.527 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.527 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.527 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.528 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.528 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.528 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.528 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.528 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.528 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.528 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.528 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.528 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.528 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.528 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.528 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.528 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.528 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.528 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.529 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.529 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.529 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.529 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.529 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.529 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.529 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.529 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.529 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.529 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.529 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.529 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.529 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.529 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.529 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.529 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.530 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.530 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.530 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.530 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.530 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.530 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.530 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.530 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.530 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.530 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.530 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.530 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.530 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.530 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.530 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.531 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.531 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.531 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.531 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.531 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.531 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.531 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.531 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.531 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mirroring", 2024-02-18T11:53:23.531 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.531 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.531 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.531 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.531 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.531 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.532 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.532 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.532 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.532 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.532 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.532 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.532 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.532 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.532 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.532 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.532 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.532 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.532 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.532 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.532 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.533 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.533 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.533 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.533 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.533 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.533 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.533 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.533 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.533 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.533 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.533 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.533 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.533 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.533 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.533 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.533 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.534 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.534 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.534 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.534 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.534 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.534 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.534 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.534 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.534 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.534 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.534 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.534 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.534 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.534 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.534 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.534 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.535 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.535 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.535 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.535 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.535 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.535 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.535 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.535 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.535 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.535 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.535 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.535 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.535 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.535 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.535 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.536 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.536 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.536 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.536 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.536 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.536 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.536 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.536 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.536 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "nfs", 2024-02-18T11:53:23.536 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.536 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.536 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.536 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.536 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.536 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.536 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.537 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.537 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.537 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.537 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.537 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.537 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.537 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.537 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.537 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.537 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.537 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.537 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.537 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.537 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.537 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.538 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.538 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.538 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.538 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.538 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.538 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.538 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.538 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.538 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.538 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.538 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.538 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.538 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.538 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.538 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.538 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.539 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.539 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.539 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.539 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.539 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.539 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.539 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.539 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.539 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.539 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.539 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.539 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.539 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.539 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.540 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.540 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.540 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.540 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.540 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.540 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.540 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.540 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.540 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.540 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.540 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.540 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.540 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.540 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.540 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.541 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.541 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.541 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.541 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.541 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.541 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.541 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.541 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.541 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.541 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "orchestrator", 2024-02-18T11:53:23.541 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.541 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.541 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.541 INFO:teuthology.orchestra.run.smithi012.stdout: "fail_fs": { 2024-02-18T11:53:23.541 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "fail_fs", 2024-02-18T11:53:23.541 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.542 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.542 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.542 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.542 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.542 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.542 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.542 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-02-18T11:53:23.542 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.542 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.542 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.542 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.542 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.542 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.542 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.542 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.543 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.543 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.543 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.543 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.543 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.543 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.543 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.543 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.543 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.543 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.543 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.543 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.543 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.543 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.543 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.543 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.544 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.544 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.544 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.544 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.544 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.544 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.544 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.544 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.544 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.544 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.544 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.544 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.544 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.544 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.544 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.545 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.545 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.545 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.545 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.545 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.545 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.545 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.545 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.545 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.545 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.545 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.545 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.545 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.545 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.545 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.546 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.546 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.546 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.546 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.546 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.546 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.546 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.546 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.546 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.546 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.546 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.546 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.546 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.546 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.546 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.547 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.547 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.547 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.547 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.547 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.547 INFO:teuthology.orchestra.run.smithi012.stdout: "orchestrator": { 2024-02-18T11:53:23.547 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "orchestrator", 2024-02-18T11:53:23.547 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.547 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.547 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.547 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.547 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.547 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.547 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.547 INFO:teuthology.orchestra.run.smithi012.stdout: "cephadm", 2024-02-18T11:53:23.547 INFO:teuthology.orchestra.run.smithi012.stdout: "rook", 2024-02-18T11:53:23.548 INFO:teuthology.orchestra.run.smithi012.stdout: "test_orchestrator" 2024-02-18T11:53:23.548 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.548 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Orchestrator backend", 2024-02-18T11:53:23.548 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.548 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.548 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.548 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.548 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.548 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.548 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.548 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "osd_perf_query", 2024-02-18T11:53:23.548 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.548 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.548 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.548 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.549 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.549 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.549 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.549 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.549 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.549 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.549 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.549 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.549 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.549 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.549 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.549 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.549 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.549 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.549 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.550 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.550 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.550 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.550 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.550 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.550 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.550 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.550 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.550 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.550 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.550 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.550 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.550 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.550 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.550 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.550 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.551 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.551 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.551 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.551 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.551 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.551 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.551 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.551 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.551 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.551 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.551 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.551 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.551 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.551 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.551 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.552 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.552 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.552 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.552 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.552 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.552 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.552 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.552 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.552 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.552 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.552 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.552 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.552 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.552 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.553 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.553 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.553 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.553 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.553 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.553 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.553 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.553 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.553 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.553 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.553 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.553 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.553 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "osd_support", 2024-02-18T11:53:23.553 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.553 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.553 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.554 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.554 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.554 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.554 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.554 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.554 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.554 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.554 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.554 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.554 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.554 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.554 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.554 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.554 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.554 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.555 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.555 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.555 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.555 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.555 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.555 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.555 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.555 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.555 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.555 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.555 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.555 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.555 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.555 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.555 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.556 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.556 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.556 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.556 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.556 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.556 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.556 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.556 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.556 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.556 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.556 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.556 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.556 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.556 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.556 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.557 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.557 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.557 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.557 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.557 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.557 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.557 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.557 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.557 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.557 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.557 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.557 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.557 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.557 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.557 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.558 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.558 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.558 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.558 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.558 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.558 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.558 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.558 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.558 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.558 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.558 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.558 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.558 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.558 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pg_autoscaler", 2024-02-18T11:53:23.558 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.559 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.559 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.559 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.559 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.559 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.559 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.559 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.559 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.559 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.559 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.559 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.559 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.559 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.559 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.559 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.559 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.560 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.560 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.560 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.560 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.560 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.560 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.560 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.560 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.560 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.560 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.560 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.560 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.560 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.560 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.560 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.561 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.561 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.561 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.561 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.561 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.561 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.561 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.561 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.561 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.561 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.561 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.561 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.561 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.561 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.561 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.562 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.562 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.562 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.562 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.562 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.562 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.562 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.562 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.562 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.562 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.562 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.562 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.562 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.562 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.562 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.562 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.563 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.563 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.563 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.563 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.563 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.563 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.563 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.563 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.563 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.563 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.563 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-02-18T11:53:23.563 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-02-18T11:53:23.563 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-02-18T11:53:23.563 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.563 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.564 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-02-18T11:53:23.564 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.564 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.564 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.564 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.564 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.564 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.564 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.564 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.564 INFO:teuthology.orchestra.run.smithi012.stdout: "threshold": { 2024-02-18T11:53:23.564 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "threshold", 2024-02-18T11:53:23.564 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-02-18T11:53:23.564 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.564 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.564 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "3.0", 2024-02-18T11:53:23.564 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1.0", 2024-02-18T11:53:23.565 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.565 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.565 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "scaling threshold", 2024-02-18T11:53:23.565 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-02-18T11:53:23.565 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.565 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.565 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.565 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.565 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.565 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.565 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "progress", 2024-02-18T11:53:23.565 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.565 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.565 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.566 INFO:teuthology.orchestra.run.smithi012.stdout: "allow_pg_recovery_event": { 2024-02-18T11:53:23.566 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "allow_pg_recovery_event", 2024-02-18T11:53:23.566 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.566 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.566 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.566 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.566 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.566 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.566 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.566 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "allow the module to show pg recovery progress", 2024-02-18T11:53:23.566 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.566 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.566 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.566 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.566 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": { 2024-02-18T11:53:23.567 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enabled", 2024-02-18T11:53:23.567 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.567 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.567 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.567 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.567 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.567 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.567 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.567 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.567 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.567 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.567 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.567 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.567 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.567 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.567 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.568 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.568 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.568 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.568 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.568 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.568 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.568 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.568 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.568 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.568 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.568 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.568 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.568 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.568 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.568 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.569 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.569 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.569 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.569 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.569 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.569 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.569 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.569 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.569 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.569 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.569 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.569 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.569 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.569 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.569 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.570 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.570 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.570 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.570 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.570 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.570 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.570 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.570 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.570 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.570 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.570 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.570 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.570 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.570 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.570 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.571 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.571 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.571 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.571 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.571 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.571 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.571 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.571 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.571 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.571 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.571 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.571 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.571 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.571 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.571 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.571 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.572 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.572 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.572 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.572 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.572 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.572 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.572 INFO:teuthology.orchestra.run.smithi012.stdout: "max_completed_events": { 2024-02-18T11:53:23.572 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_completed_events", 2024-02-18T11:53:23.572 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.572 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.572 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.572 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "50", 2024-02-18T11:53:23.572 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.572 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.572 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.573 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "number of past completed events to remember", 2024-02-18T11:53:23.573 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.573 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.573 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.573 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.573 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-02-18T11:53:23.573 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-02-18T11:53:23.573 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-02-18T11:53:23.573 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.573 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.573 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-02-18T11:53:23.573 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.573 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.573 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.573 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how long the module is going to sleep", 2024-02-18T11:53:23.573 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.574 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.574 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.574 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.574 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.574 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.574 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.574 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "prometheus", 2024-02-18T11:53:23.574 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.574 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.574 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.574 INFO:teuthology.orchestra.run.smithi012.stdout: "cache": { 2024-02-18T11:53:23.574 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cache", 2024-02-18T11:53:23.574 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.574 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.574 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.575 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.575 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.575 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.575 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.575 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.575 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.575 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.575 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.575 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.575 INFO:teuthology.orchestra.run.smithi012.stdout: "exclude_perf_counters": { 2024-02-18T11:53:23.575 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "exclude_perf_counters", 2024-02-18T11:53:23.575 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.575 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.575 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.575 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.576 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.576 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.576 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.576 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-02-18T11:53:23.576 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-02-18T11:53:23.576 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.576 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.576 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.576 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.576 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.576 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.576 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.576 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.576 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.577 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.577 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.577 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.577 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.577 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.577 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.577 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.577 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.577 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.577 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.577 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.577 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.577 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.577 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.578 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.578 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.578 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.578 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.578 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.578 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.578 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.578 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.578 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.578 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.578 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.578 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.578 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.578 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.578 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.579 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.579 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.579 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.579 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.579 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.579 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.579 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.579 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.579 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.579 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.579 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.579 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.579 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.579 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.579 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.580 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.580 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.580 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.580 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.580 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.580 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.580 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.580 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.580 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.580 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.580 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.580 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.580 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.580 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.580 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.581 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.581 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.581 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.581 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.581 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.581 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_stats_pools": { 2024-02-18T11:53:23.581 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rbd_stats_pools", 2024-02-18T11:53:23.581 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.581 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.581 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.581 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.581 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.581 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.581 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.581 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.581 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.582 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.582 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.582 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.582 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_stats_pools_refresh_interval": { 2024-02-18T11:53:23.582 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-02-18T11:53:23.582 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.582 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.582 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.582 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "300", 2024-02-18T11:53:23.582 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.582 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.582 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.582 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.582 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.583 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.583 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.583 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.583 INFO:teuthology.orchestra.run.smithi012.stdout: "scrape_interval": { 2024-02-18T11:53:23.583 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "scrape_interval", 2024-02-18T11:53:23.583 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-02-18T11:53:23.583 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.583 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.583 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "15.0", 2024-02-18T11:53:23.583 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.583 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.583 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.583 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.583 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.583 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.583 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.584 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.584 INFO:teuthology.orchestra.run.smithi012.stdout: "server_addr": { 2024-02-18T11:53:23.584 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_addr", 2024-02-18T11:53:23.584 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.584 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.584 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.584 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "::", 2024-02-18T11:53:23.584 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.584 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.584 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.584 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-02-18T11:53:23.584 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.584 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.584 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.584 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.585 INFO:teuthology.orchestra.run.smithi012.stdout: "server_port": { 2024-02-18T11:53:23.585 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_port", 2024-02-18T11:53:23.585 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.585 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.585 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.585 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "9283", 2024-02-18T11:53:23.585 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.585 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.585 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.585 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-02-18T11:53:23.585 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.585 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.585 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.585 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.585 INFO:teuthology.orchestra.run.smithi012.stdout: "stale_cache_strategy": { 2024-02-18T11:53:23.586 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "stale_cache_strategy", 2024-02-18T11:53:23.586 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.586 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.586 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.586 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "log", 2024-02-18T11:53:23.586 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.586 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.586 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.586 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.586 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.586 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.586 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.586 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.586 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_behaviour": { 2024-02-18T11:53:23.586 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_behaviour", 2024-02-18T11:53:23.587 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.587 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.587 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.587 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "default", 2024-02-18T11:53:23.587 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.587 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.587 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.587 INFO:teuthology.orchestra.run.smithi012.stdout: "default", 2024-02-18T11:53:23.587 INFO:teuthology.orchestra.run.smithi012.stdout: "error" 2024-02-18T11:53:23.587 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.587 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.587 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.587 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.587 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.587 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.587 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_error_status_code": { 2024-02-18T11:53:23.588 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_error_status_code", 2024-02-18T11:53:23.588 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.588 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.588 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.588 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "500", 2024-02-18T11:53:23.588 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "400", 2024-02-18T11:53:23.588 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "599", 2024-02-18T11:53:23.588 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.588 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.588 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.588 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.588 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.588 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.588 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.588 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.589 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.589 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rbd_support", 2024-02-18T11:53:23.589 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.589 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.589 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.589 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.589 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.589 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.589 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.589 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.589 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.589 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.589 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.589 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.589 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.590 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.590 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.590 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.590 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.590 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.590 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.590 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.590 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.590 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.590 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.590 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.590 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.590 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.590 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.591 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.591 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.591 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.591 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.591 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.591 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.591 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.591 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.591 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.591 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.591 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.591 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.591 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.591 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.591 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.592 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.592 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.592 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.592 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.592 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.592 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.592 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.592 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.592 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.592 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.592 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.592 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.592 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.592 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.592 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.593 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.593 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.593 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.593 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.593 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.593 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.593 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.593 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.593 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.593 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.593 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.593 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.593 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.593 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.593 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.594 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.594 INFO:teuthology.orchestra.run.smithi012.stdout: "max_concurrent_snap_create": { 2024-02-18T11:53:23.594 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_concurrent_snap_create", 2024-02-18T11:53:23.594 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.594 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.594 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.594 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-02-18T11:53:23.594 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.594 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.594 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.594 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.594 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.594 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.594 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.594 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.595 INFO:teuthology.orchestra.run.smithi012.stdout: "mirror_snapshot_schedule": { 2024-02-18T11:53:23.595 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mirror_snapshot_schedule", 2024-02-18T11:53:23.595 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.595 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.595 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.595 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.595 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.595 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.595 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.595 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.595 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.595 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.595 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.595 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.595 INFO:teuthology.orchestra.run.smithi012.stdout: "trash_purge_schedule": { 2024-02-18T11:53:23.596 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "trash_purge_schedule", 2024-02-18T11:53:23.596 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.596 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.596 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.596 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.596 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.596 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.596 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.596 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.596 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.596 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.596 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.596 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.596 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.597 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.597 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.597 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "restful", 2024-02-18T11:53:23.597 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.597 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.597 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.597 INFO:teuthology.orchestra.run.smithi012.stdout: "enable_auth": { 2024-02-18T11:53:23.597 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enable_auth", 2024-02-18T11:53:23.597 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.597 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.597 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.597 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.597 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.597 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.597 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.597 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.598 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.598 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.598 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.598 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.598 INFO:teuthology.orchestra.run.smithi012.stdout: "key_file": { 2024-02-18T11:53:23.598 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "key_file", 2024-02-18T11:53:23.598 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.598 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.598 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.598 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.598 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.598 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.598 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.598 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.598 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.598 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.599 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.599 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.599 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.599 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.599 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.599 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.599 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.599 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.599 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.599 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.599 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.599 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.599 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.599 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.599 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.600 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.600 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.600 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.600 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.600 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.600 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.600 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.600 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.600 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.600 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.600 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.600 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.600 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.600 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.600 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.601 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.601 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.601 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.601 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.601 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.601 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.601 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.601 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.601 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.601 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.601 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.601 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.601 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.601 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.601 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.601 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.602 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.602 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.602 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.602 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.602 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.602 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.602 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.602 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.602 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.602 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.602 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.602 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.602 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.602 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.602 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.603 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.603 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.603 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.603 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.603 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.603 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.603 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.603 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.603 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.603 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.603 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.603 INFO:teuthology.orchestra.run.smithi012.stdout: "server_addr": { 2024-02-18T11:53:23.603 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_addr", 2024-02-18T11:53:23.603 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.604 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.604 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.604 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.604 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.604 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.604 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.604 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.604 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.604 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.604 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.604 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.604 INFO:teuthology.orchestra.run.smithi012.stdout: "server_port": { 2024-02-18T11:53:23.604 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_port", 2024-02-18T11:53:23.604 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.604 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.605 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.605 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.605 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.605 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.605 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.605 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.605 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.605 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.605 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.605 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.605 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.605 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.605 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.605 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rgw", 2024-02-18T11:53:23.605 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.605 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.606 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.606 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.606 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.606 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.606 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.606 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.606 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.606 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.606 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.606 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.606 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.606 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.606 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.606 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.606 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.607 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.607 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.607 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.607 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.607 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.607 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.607 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.607 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.607 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.607 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.607 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.607 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.607 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.607 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.607 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.608 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.608 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.608 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.608 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.608 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.608 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.608 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.608 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.608 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.608 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.608 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.608 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.608 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.608 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.608 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.609 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.609 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.609 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.609 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.609 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.609 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.609 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.609 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.609 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.609 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.609 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.609 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.609 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.609 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.609 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.610 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.610 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.610 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.610 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.610 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.610 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.610 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.610 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.610 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.610 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.610 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.610 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.610 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.610 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.610 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rook", 2024-02-18T11:53:23.610 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.611 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.611 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.611 INFO:teuthology.orchestra.run.smithi012.stdout: "drive_group_interval": { 2024-02-18T11:53:23.611 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "drive_group_interval", 2024-02-18T11:53:23.611 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-02-18T11:53:23.611 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.611 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.611 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "300.0", 2024-02-18T11:53:23.611 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.611 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.611 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.611 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2024-02-18T11:53:23.611 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.611 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.611 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.612 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.612 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.612 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.612 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.612 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.612 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.612 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.612 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.612 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.612 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.612 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.612 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.612 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.612 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.612 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.613 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.613 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.613 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.613 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.613 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.613 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.613 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.613 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.613 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.613 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.613 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.613 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.613 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.613 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.613 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.614 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.614 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.614 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.614 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.614 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.614 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.614 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.614 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.614 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.614 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.614 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.614 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.614 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.614 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.614 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.614 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.615 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.615 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.615 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.615 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.615 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.615 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.615 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.615 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.615 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.615 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.615 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.615 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.615 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.615 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.616 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.616 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.616 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.616 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.616 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.616 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.616 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.616 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.616 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.616 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.616 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.616 INFO:teuthology.orchestra.run.smithi012.stdout: "storage_class": { 2024-02-18T11:53:23.616 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "storage_class", 2024-02-18T11:53:23.616 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.616 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.617 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.617 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "local", 2024-02-18T11:53:23.617 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.617 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.617 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.617 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-02-18T11:53:23.617 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.617 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.617 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.617 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.617 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.617 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.617 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.617 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "selftest", 2024-02-18T11:53:23.617 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.618 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.618 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.618 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.618 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.618 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.618 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.618 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.618 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.618 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.618 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.618 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.618 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.618 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.618 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.618 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.619 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.619 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.619 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.619 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.619 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.619 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.619 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.619 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.619 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.619 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.619 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.619 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.619 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.619 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.619 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.619 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.620 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.620 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.620 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.620 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.620 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.620 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.620 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.620 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.620 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.620 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.620 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.620 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.620 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.620 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.621 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.621 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.621 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.621 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.621 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.621 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.621 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.621 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.621 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.621 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.621 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.621 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.621 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.621 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.621 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.622 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.622 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.622 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.622 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.622 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.622 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.622 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.622 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.622 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.622 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.622 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.622 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.622 INFO:teuthology.orchestra.run.smithi012.stdout: "roption1": { 2024-02-18T11:53:23.622 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "roption1", 2024-02-18T11:53:23.622 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.622 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.623 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.623 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.623 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.623 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.623 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.623 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.623 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.623 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.623 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.623 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.623 INFO:teuthology.orchestra.run.smithi012.stdout: "roption2": { 2024-02-18T11:53:23.623 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "roption2", 2024-02-18T11:53:23.623 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.623 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.623 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.624 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "xyz", 2024-02-18T11:53:23.624 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.624 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.624 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.624 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.624 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.624 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.624 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.624 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.624 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption1": { 2024-02-18T11:53:23.624 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption1", 2024-02-18T11:53:23.624 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.624 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.624 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.624 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.625 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.625 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.625 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.625 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.625 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.625 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.625 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.625 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.625 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption2": { 2024-02-18T11:53:23.625 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption2", 2024-02-18T11:53:23.625 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.625 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.625 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.625 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.625 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.626 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.626 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.626 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.626 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.626 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.626 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.626 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.626 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption3": { 2024-02-18T11:53:23.626 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption3", 2024-02-18T11:53:23.626 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-02-18T11:53:23.626 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.626 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.626 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.626 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.626 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.626 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.627 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.627 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.627 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.627 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.627 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.627 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption4": { 2024-02-18T11:53:23.627 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption4", 2024-02-18T11:53:23.627 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.627 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.627 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.627 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.627 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.627 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.627 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.627 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.628 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.628 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.628 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.628 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.628 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption5": { 2024-02-18T11:53:23.628 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption5", 2024-02-18T11:53:23.628 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.628 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.628 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.628 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.628 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.628 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.628 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.628 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.629 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.629 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.629 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.629 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.629 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption6": { 2024-02-18T11:53:23.629 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption6", 2024-02-18T11:53:23.629 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.629 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.629 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.629 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.629 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.629 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.629 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.629 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.629 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.629 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.630 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.630 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.630 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption7": { 2024-02-18T11:53:23.630 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption7", 2024-02-18T11:53:23.630 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.630 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.630 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.630 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.630 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-02-18T11:53:23.630 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "42", 2024-02-18T11:53:23.630 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.630 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.630 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.630 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.630 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.631 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.631 INFO:teuthology.orchestra.run.smithi012.stdout: "testkey": { 2024-02-18T11:53:23.631 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "testkey", 2024-02-18T11:53:23.631 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.631 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.631 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.631 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.631 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.631 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.631 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.631 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.631 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.631 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.631 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.631 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.631 INFO:teuthology.orchestra.run.smithi012.stdout: "testlkey": { 2024-02-18T11:53:23.632 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "testlkey", 2024-02-18T11:53:23.632 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.632 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.632 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.632 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.632 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.632 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.632 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.632 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.632 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.632 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.632 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.632 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.632 INFO:teuthology.orchestra.run.smithi012.stdout: "testnewline": { 2024-02-18T11:53:23.632 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "testnewline", 2024-02-18T11:53:23.633 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.633 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.633 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.633 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.633 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.633 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.633 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.633 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.633 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.633 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.633 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.633 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.633 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.633 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.633 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.634 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "snap_schedule", 2024-02-18T11:53:23.634 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.634 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.634 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.634 INFO:teuthology.orchestra.run.smithi012.stdout: "allow_m_granularity": { 2024-02-18T11:53:23.634 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "allow_m_granularity", 2024-02-18T11:53:23.634 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.634 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.634 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.634 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.634 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.634 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.634 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.634 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "allow minute scheduled snapshots", 2024-02-18T11:53:23.634 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.634 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.635 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.635 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.635 INFO:teuthology.orchestra.run.smithi012.stdout: "dump_on_update": { 2024-02-18T11:53:23.635 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "dump_on_update", 2024-02-18T11:53:23.635 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.635 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.635 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.635 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.635 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.635 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.635 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.635 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "dump database to debug log on update", 2024-02-18T11:53:23.635 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.635 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.635 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.636 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.636 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.636 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.636 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.636 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.636 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.636 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.636 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.636 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.636 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.636 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.636 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.636 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.636 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.636 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.636 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.637 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.637 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.637 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.637 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.637 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.637 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.637 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.637 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.637 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.637 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.637 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.637 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.637 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.637 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.637 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.638 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.638 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.638 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.638 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.638 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.638 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.638 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.638 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.638 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.638 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.638 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.638 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.638 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.638 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.638 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.639 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.639 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.639 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.639 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.639 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.639 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.639 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.639 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.639 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.639 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.639 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.639 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.639 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.639 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.639 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.639 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.640 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.640 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.640 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.640 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.640 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.640 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.640 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.640 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.640 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.640 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.640 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.640 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.640 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "stats", 2024-02-18T11:53:23.640 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.640 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.641 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.641 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.641 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.641 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.641 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.641 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.641 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.641 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.641 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.641 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.641 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.641 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.641 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.641 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.642 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.642 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.642 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.642 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.642 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.642 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.642 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.642 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.642 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.642 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.642 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.642 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.642 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.642 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.642 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.643 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.643 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.643 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.643 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.643 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.643 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.643 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.643 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.643 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.643 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.643 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.643 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.643 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.643 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.643 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.644 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.644 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.644 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.644 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.644 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.644 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.644 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.644 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.644 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.644 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.644 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.644 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.644 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.644 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.644 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.645 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.645 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.645 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.645 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.645 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.645 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.645 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.645 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.645 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.645 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.645 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.645 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.645 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.645 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.645 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.645 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "status", 2024-02-18T11:53:23.646 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.646 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.646 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.646 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.646 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.646 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.646 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.646 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.646 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.646 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.646 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.646 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.646 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.646 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.646 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.647 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.647 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.647 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.647 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.647 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.647 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.647 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.647 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.647 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.647 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.647 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.647 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.647 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.647 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.647 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.648 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.648 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.648 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.648 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.648 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.648 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.648 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.648 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.648 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.648 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.648 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.648 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.648 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.648 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.649 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.649 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.649 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.649 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.649 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.649 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.649 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.649 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.649 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.649 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.649 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.649 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.649 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.649 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.649 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.650 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.650 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.650 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.650 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.650 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.650 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.650 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.650 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.650 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.650 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.650 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.650 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.650 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.650 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.650 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.650 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.651 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.651 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "telegraf", 2024-02-18T11:53:23.651 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.651 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.651 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.651 INFO:teuthology.orchestra.run.smithi012.stdout: "address": { 2024-02-18T11:53:23.651 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "address", 2024-02-18T11:53:23.651 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.651 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.651 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.651 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-02-18T11:53:23.651 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.651 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.651 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.651 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.652 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.652 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.652 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.652 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.652 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-02-18T11:53:23.652 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-02-18T11:53:23.652 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-02-18T11:53:23.652 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.652 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.652 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "15", 2024-02-18T11:53:23.652 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.652 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.652 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.652 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.652 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.653 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.653 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.653 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.653 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.653 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.653 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.653 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.653 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.653 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.653 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.653 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.653 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.653 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.653 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.653 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.653 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.654 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.654 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.654 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.654 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.654 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.654 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.654 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.654 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.654 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.654 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.654 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.654 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.654 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.654 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.654 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.655 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.655 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.655 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.655 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.655 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.655 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.655 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.655 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.655 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.655 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.655 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.655 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.655 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.655 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.655 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.656 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.656 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.656 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.656 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.656 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.656 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.656 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.656 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.656 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.656 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.656 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.656 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.656 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.656 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.657 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.657 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.657 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.657 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.657 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.657 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.657 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.657 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.657 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.657 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.657 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.657 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.657 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.657 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.657 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.658 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.658 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "telemetry", 2024-02-18T11:53:23.658 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.658 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.658 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.658 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_basic": { 2024-02-18T11:53:23.658 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_basic", 2024-02-18T11:53:23.658 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.658 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.658 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.658 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.658 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.658 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.658 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.658 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share basic cluster information (size, version)", 2024-02-18T11:53:23.659 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.659 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.659 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.659 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.659 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_crash": { 2024-02-18T11:53:23.659 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_crash", 2024-02-18T11:53:23.659 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.659 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.659 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.659 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.659 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.659 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.659 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.659 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-02-18T11:53:23.659 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.660 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.660 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.660 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.660 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_device": { 2024-02-18T11:53:23.660 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_device", 2024-02-18T11:53:23.660 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.660 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.660 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.660 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.660 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.660 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.660 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.660 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-02-18T11:53:23.660 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.660 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.661 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.661 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.661 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_ident": { 2024-02-18T11:53:23.661 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_ident", 2024-02-18T11:53:23.661 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.661 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.661 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.661 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.661 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.661 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.661 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.661 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-02-18T11:53:23.661 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.661 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.661 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.662 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.662 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_perf": { 2024-02-18T11:53:23.662 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_perf", 2024-02-18T11:53:23.662 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.662 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.662 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.662 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.662 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.662 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.662 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.662 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share various performance metrics of a cluster", 2024-02-18T11:53:23.662 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.662 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.662 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.662 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.663 INFO:teuthology.orchestra.run.smithi012.stdout: "contact": { 2024-02-18T11:53:23.663 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "contact", 2024-02-18T11:53:23.663 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.663 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.663 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.663 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.663 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.663 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.663 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.663 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.663 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.663 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.663 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.663 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.663 INFO:teuthology.orchestra.run.smithi012.stdout: "description": { 2024-02-18T11:53:23.663 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "description", 2024-02-18T11:53:23.664 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.664 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.664 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.664 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.664 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.664 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.664 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.664 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.664 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.664 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.664 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.664 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.664 INFO:teuthology.orchestra.run.smithi012.stdout: "device_url": { 2024-02-18T11:53:23.664 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "device_url", 2024-02-18T11:53:23.664 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.665 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.665 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.665 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-02-18T11:53:23.665 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.665 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.665 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.665 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.665 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.665 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.665 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.665 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.665 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": { 2024-02-18T11:53:23.665 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enabled", 2024-02-18T11:53:23.665 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.665 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.665 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.666 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.666 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.666 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.666 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.666 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.666 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.666 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.666 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.666 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.666 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-02-18T11:53:23.666 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-02-18T11:53:23.666 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.666 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.666 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.666 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "24", 2024-02-18T11:53:23.667 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "8", 2024-02-18T11:53:23.667 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.667 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.667 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.667 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.667 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.667 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.667 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.667 INFO:teuthology.orchestra.run.smithi012.stdout: "last_opt_revision": { 2024-02-18T11:53:23.667 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "last_opt_revision", 2024-02-18T11:53:23.667 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.667 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.667 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.667 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "1", 2024-02-18T11:53:23.668 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.668 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.668 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.668 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.668 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.668 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.668 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.668 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.668 INFO:teuthology.orchestra.run.smithi012.stdout: "leaderboard": { 2024-02-18T11:53:23.668 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "leaderboard", 2024-02-18T11:53:23.668 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.668 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.668 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.668 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.668 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.669 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.669 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.669 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.669 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.669 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.669 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.669 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.669 INFO:teuthology.orchestra.run.smithi012.stdout: "leaderboard_description": { 2024-02-18T11:53:23.669 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "leaderboard_description", 2024-02-18T11:53:23.669 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.669 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.669 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.669 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.669 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.669 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.670 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.670 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.670 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.670 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.670 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.670 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.670 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.670 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.670 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.670 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.670 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.670 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.670 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.670 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.670 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.671 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.671 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.671 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.671 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.671 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.671 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.671 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.671 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.671 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.671 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.671 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.671 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.671 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.671 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.671 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.672 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.672 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.672 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.672 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.672 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.672 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.672 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.672 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.672 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.672 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.672 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.672 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.672 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.672 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.672 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.673 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.673 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.673 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.673 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.673 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.673 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.673 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.673 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.673 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.673 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.673 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.673 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.673 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.673 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.673 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.673 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.674 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.674 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.674 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.674 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.674 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.674 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.674 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.674 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.674 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.674 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.674 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.674 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.674 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.674 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.674 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.675 INFO:teuthology.orchestra.run.smithi012.stdout: "organization": { 2024-02-18T11:53:23.675 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "organization", 2024-02-18T11:53:23.675 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.675 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.675 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.675 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.675 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.675 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.675 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.675 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.675 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.675 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.675 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.675 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.675 INFO:teuthology.orchestra.run.smithi012.stdout: "proxy": { 2024-02-18T11:53:23.676 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "proxy", 2024-02-18T11:53:23.676 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.676 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.676 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.676 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.676 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.676 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.676 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.676 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.676 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.676 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.676 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.676 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.676 INFO:teuthology.orchestra.run.smithi012.stdout: "url": { 2024-02-18T11:53:23.676 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "url", 2024-02-18T11:53:23.677 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.677 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.677 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.677 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-02-18T11:53:23.677 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.677 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.677 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.677 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.677 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.677 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.677 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.677 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.677 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.677 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.677 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.678 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "test_orchestrator", 2024-02-18T11:53:23.678 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.678 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.678 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.678 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.678 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.678 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.678 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.678 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.678 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.678 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.678 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.678 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.678 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.678 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.679 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.679 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.679 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.679 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.679 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.679 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.679 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.679 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.679 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.679 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.679 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.679 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.679 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.679 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.679 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.680 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.680 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.680 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.680 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.680 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.680 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.680 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.680 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.680 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.680 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.680 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.680 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.680 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.680 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.681 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.681 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.681 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.681 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.681 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.681 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.681 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.681 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.681 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.681 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.681 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.681 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.681 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.681 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.681 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.681 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.682 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.682 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.682 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.682 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.682 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.682 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.682 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.682 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.682 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.682 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.682 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.682 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.682 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.682 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.682 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.683 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.683 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.683 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "volumes", 2024-02-18T11:53:23.683 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.683 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.683 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.683 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.683 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.683 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.683 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.683 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.683 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.683 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.683 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.683 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.684 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.684 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.684 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.684 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.684 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.684 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.684 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.684 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.684 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.684 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.684 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.684 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.684 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.684 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.685 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.685 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.685 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.685 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.685 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.685 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.685 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.685 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.685 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.685 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.685 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.685 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.685 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.685 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.685 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.686 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.686 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.686 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.686 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.686 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.686 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.686 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.686 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.686 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.686 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.686 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.686 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.686 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.686 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.686 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.687 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.687 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.687 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.687 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.687 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.687 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.687 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.687 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.687 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.687 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.687 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.687 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.687 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.687 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.687 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.687 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.688 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.688 INFO:teuthology.orchestra.run.smithi012.stdout: "max_concurrent_clones": { 2024-02-18T11:53:23.688 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_concurrent_clones", 2024-02-18T11:53:23.688 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.688 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.688 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.688 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "4", 2024-02-18T11:53:23.688 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.688 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.688 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.688 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Number of asynchronous cloner threads", 2024-02-18T11:53:23.688 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.688 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.688 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.689 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.689 INFO:teuthology.orchestra.run.smithi012.stdout: "periodic_async_work": { 2024-02-18T11:53:23.689 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "periodic_async_work", 2024-02-18T11:53:23.689 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.689 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.689 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.689 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.689 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.689 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.689 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.689 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Periodically check for async work", 2024-02-18T11:53:23.689 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.689 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.689 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.689 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.690 INFO:teuthology.orchestra.run.smithi012.stdout: "snapshot_clone_delay": { 2024-02-18T11:53:23.690 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "snapshot_clone_delay", 2024-02-18T11:53:23.690 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.690 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.690 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.690 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-02-18T11:53:23.690 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.690 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.690 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.690 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-02-18T11:53:23.690 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.690 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.690 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.690 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.690 INFO:teuthology.orchestra.run.smithi012.stdout: "snapshot_clone_no_wait": { 2024-02-18T11:53:23.690 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "snapshot_clone_no_wait", 2024-02-18T11:53:23.691 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.691 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.691 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.691 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.691 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.691 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.691 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.691 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-02-18T11:53:23.691 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.691 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.691 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.691 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.691 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.691 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.691 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.692 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix", 2024-02-18T11:53:23.692 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.692 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.692 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.692 INFO:teuthology.orchestra.run.smithi012.stdout: "discovery_interval": { 2024-02-18T11:53:23.692 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "discovery_interval", 2024-02-18T11:53:23.692 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-02-18T11:53:23.692 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.692 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.692 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "100", 2024-02-18T11:53:23.692 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.692 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.692 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.693 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.693 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.693 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.693 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.693 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.693 INFO:teuthology.orchestra.run.smithi012.stdout: "identifier": { 2024-02-18T11:53:23.693 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "identifier", 2024-02-18T11:53:23.693 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.693 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.693 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.693 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.693 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.693 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.693 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.693 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.694 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.694 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.694 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.694 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.694 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-02-18T11:53:23.694 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-02-18T11:53:23.694 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-02-18T11:53:23.694 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.694 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.694 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-02-18T11:53:23.694 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.694 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.694 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.694 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.694 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.695 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.695 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.695 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.695 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.695 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.695 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.695 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.695 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.695 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.695 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.695 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.695 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.695 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.695 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.695 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.695 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.696 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.696 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.696 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.696 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.696 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.696 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.696 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.696 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.696 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.696 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.696 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.696 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.696 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.696 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.697 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.697 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.697 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.697 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.697 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.697 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.697 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.697 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.697 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.697 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.697 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.697 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.697 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.697 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.697 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.697 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.698 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.698 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.698 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.698 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.698 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.698 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.698 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.698 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.698 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.698 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.698 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.698 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.698 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.698 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.698 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.699 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.699 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.699 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.699 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.699 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.699 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.699 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.699 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.699 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.699 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.699 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.699 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.699 INFO:teuthology.orchestra.run.smithi012.stdout: "zabbix_host": { 2024-02-18T11:53:23.699 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix_host", 2024-02-18T11:53:23.699 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.699 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.700 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.700 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.700 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.700 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.700 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.700 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.700 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.700 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.700 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.700 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.700 INFO:teuthology.orchestra.run.smithi012.stdout: "zabbix_port": { 2024-02-18T11:53:23.700 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix_port", 2024-02-18T11:53:23.700 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.700 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.700 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.701 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10051", 2024-02-18T11:53:23.701 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.701 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.701 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.701 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.701 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.701 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.701 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.701 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.701 INFO:teuthology.orchestra.run.smithi012.stdout: "zabbix_sender": { 2024-02-18T11:53:23.701 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix_sender", 2024-02-18T11:53:23.701 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.701 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.701 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.701 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-02-18T11:53:23.702 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.702 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.702 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.702 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.702 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.702 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.702 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.702 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.702 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.702 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.702 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-02-18T11:53:23.702 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.702 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.702 INFO:teuthology.orchestra.run.smithi012.stdout: "modules": [ 2024-02-18T11:53:23.702 INFO:teuthology.orchestra.run.smithi012.stdout: "cephadm", 2024-02-18T11:53:23.702 INFO:teuthology.orchestra.run.smithi012.stdout: "dashboard", 2024-02-18T11:53:23.703 INFO:teuthology.orchestra.run.smithi012.stdout: "iostat", 2024-02-18T11:53:23.703 INFO:teuthology.orchestra.run.smithi012.stdout: "nfs", 2024-02-18T11:53:23.703 INFO:teuthology.orchestra.run.smithi012.stdout: "restful" 2024-02-18T11:53:23.703 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.703 INFO:teuthology.orchestra.run.smithi012.stdout: "available_modules": [ 2024-02-18T11:53:23.703 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.703 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "alerts", 2024-02-18T11:53:23.703 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.703 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.703 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.703 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-02-18T11:53:23.703 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-02-18T11:53:23.703 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-02-18T11:53:23.703 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.703 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.703 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-02-18T11:53:23.704 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.704 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.704 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.704 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How frequently to reexamine health status", 2024-02-18T11:53:23.704 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.704 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.704 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.704 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.704 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.704 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.704 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.704 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.704 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.704 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.704 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.705 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.705 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.705 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.705 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.705 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.705 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.705 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.705 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.705 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.705 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.705 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.705 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.705 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.705 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.705 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.705 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.706 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.706 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.706 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.706 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.706 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.706 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.706 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.706 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.706 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.706 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.706 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.706 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.706 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.706 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.707 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.707 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.707 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.707 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.707 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.707 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.707 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.707 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.707 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.707 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.707 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.707 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.707 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.707 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.707 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.707 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.708 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.708 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.708 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.708 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.708 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.708 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.708 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.708 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.708 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.708 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.708 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.708 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.708 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.708 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.709 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.709 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.709 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.709 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_destination": { 2024-02-18T11:53:23.709 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_destination", 2024-02-18T11:53:23.709 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.709 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.709 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.709 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.709 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.709 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.709 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.709 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Email address to send alerts to", 2024-02-18T11:53:23.709 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.709 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.709 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.710 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.710 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_from_name": { 2024-02-18T11:53:23.710 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_from_name", 2024-02-18T11:53:23.710 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.710 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.710 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.710 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "Ceph", 2024-02-18T11:53:23.710 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.710 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.710 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.710 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Email From: name", 2024-02-18T11:53:23.710 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.710 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.710 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.710 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.711 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_host": { 2024-02-18T11:53:23.711 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_host", 2024-02-18T11:53:23.711 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.711 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.711 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.711 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.711 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.711 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.711 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.711 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SMTP server", 2024-02-18T11:53:23.711 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.711 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.711 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.711 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.711 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_password": { 2024-02-18T11:53:23.712 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_password", 2024-02-18T11:53:23.712 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.712 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.712 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.712 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.712 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.712 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.712 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.712 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Password to authenticate with", 2024-02-18T11:53:23.712 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.712 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.712 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.712 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.712 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_port": { 2024-02-18T11:53:23.712 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_port", 2024-02-18T11:53:23.713 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.713 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.713 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.713 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "465", 2024-02-18T11:53:23.713 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.713 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.713 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.713 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SMTP port", 2024-02-18T11:53:23.713 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.713 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.713 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.713 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.713 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_sender": { 2024-02-18T11:53:23.713 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_sender", 2024-02-18T11:53:23.713 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.714 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.714 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.714 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.714 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.714 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.714 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.714 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SMTP envelope sender", 2024-02-18T11:53:23.714 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.714 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.714 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.714 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.714 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_ssl": { 2024-02-18T11:53:23.714 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_ssl", 2024-02-18T11:53:23.714 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.714 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.714 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.715 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.715 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.715 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.715 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.715 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use SSL to connect to SMTP server", 2024-02-18T11:53:23.715 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.715 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.715 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.715 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.715 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_user": { 2024-02-18T11:53:23.715 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_user", 2024-02-18T11:53:23.715 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.715 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.715 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.715 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.715 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.716 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.716 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.716 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "User to authenticate as", 2024-02-18T11:53:23.716 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.716 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.716 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.716 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.716 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.716 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.716 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.716 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "balancer", 2024-02-18T11:53:23.716 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.716 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.716 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.717 INFO:teuthology.orchestra.run.smithi012.stdout: "active": { 2024-02-18T11:53:23.717 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "active", 2024-02-18T11:53:23.717 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.717 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.717 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.717 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.717 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.717 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.717 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.717 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "automatically balance PGs across cluster", 2024-02-18T11:53:23.717 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.717 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.717 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.717 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.717 INFO:teuthology.orchestra.run.smithi012.stdout: "begin_time": { 2024-02-18T11:53:23.717 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "begin_time", 2024-02-18T11:53:23.718 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.718 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.718 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.718 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0000", 2024-02-18T11:53:23.718 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.718 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.718 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.718 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "beginning time of day to automatically balance", 2024-02-18T11:53:23.718 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-02-18T11:53:23.718 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.718 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.718 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.718 INFO:teuthology.orchestra.run.smithi012.stdout: "begin_weekday": { 2024-02-18T11:53:23.718 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "begin_weekday", 2024-02-18T11:53:23.718 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-02-18T11:53:23.718 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.719 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.719 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-02-18T11:53:23.719 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "0", 2024-02-18T11:53:23.719 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "6", 2024-02-18T11:53:23.719 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.719 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-02-18T11:53:23.719 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-02-18T11:53:23.719 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.719 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.719 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.719 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_compat_max_iterations": { 2024-02-18T11:53:23.719 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crush_compat_max_iterations", 2024-02-18T11:53:23.719 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-02-18T11:53:23.719 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.719 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.719 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "25", 2024-02-18T11:53:23.720 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-02-18T11:53:23.720 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "250", 2024-02-18T11:53:23.720 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.720 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-02-18T11:53:23.720 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.720 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.720 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.720 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.720 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_compat_metrics": { 2024-02-18T11:53:23.720 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crush_compat_metrics", 2024-02-18T11:53:23.720 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.720 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.720 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.720 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "pgs,objects,bytes", 2024-02-18T11:53:23.720 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.721 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.721 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.721 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-02-18T11:53:23.721 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-02-18T11:53:23.721 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.721 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.721 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.721 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_compat_step": { 2024-02-18T11:53:23.721 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crush_compat_step", 2024-02-18T11:53:23.721 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-02-18T11:53:23.721 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.721 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.721 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0.5", 2024-02-18T11:53:23.721 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "0.001", 2024-02-18T11:53:23.722 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "0.999", 2024-02-18T11:53:23.722 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.722 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "aggressiveness of optimization", 2024-02-18T11:53:23.722 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-02-18T11:53:23.722 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.722 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.722 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.722 INFO:teuthology.orchestra.run.smithi012.stdout: "end_time": { 2024-02-18T11:53:23.722 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "end_time", 2024-02-18T11:53:23.722 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.722 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.722 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.722 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "2359", 2024-02-18T11:53:23.722 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.722 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.723 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.723 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "ending time of day to automatically balance", 2024-02-18T11:53:23.723 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-02-18T11:53:23.723 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.723 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.723 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.723 INFO:teuthology.orchestra.run.smithi012.stdout: "end_weekday": { 2024-02-18T11:53:23.723 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "end_weekday", 2024-02-18T11:53:23.723 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-02-18T11:53:23.723 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.723 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.723 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-02-18T11:53:23.723 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "0", 2024-02-18T11:53:23.723 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "6", 2024-02-18T11:53:23.723 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.724 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-02-18T11:53:23.724 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-02-18T11:53:23.724 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.724 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.724 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.724 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.724 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.724 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.724 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.724 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.724 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.724 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.724 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.724 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.725 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.725 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.725 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.725 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.725 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.725 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.725 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.725 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.725 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.725 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.725 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.725 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.725 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.725 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.725 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.726 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.726 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.726 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.726 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.726 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.726 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.726 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.726 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.726 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.726 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.726 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.726 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.726 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.726 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.726 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.726 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.727 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.727 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.727 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.727 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.727 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.727 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.727 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.727 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.727 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.727 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.727 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.727 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.727 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.727 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.727 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.728 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.728 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.728 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.728 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.728 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.728 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.728 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.728 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.728 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.728 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.728 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.728 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.728 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.728 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.728 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.729 INFO:teuthology.orchestra.run.smithi012.stdout: "min_score": { 2024-02-18T11:53:23.729 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "min_score", 2024-02-18T11:53:23.729 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-02-18T11:53:23.729 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.729 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.729 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-02-18T11:53:23.729 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.729 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.729 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.729 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-02-18T11:53:23.729 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.729 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.729 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.729 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.730 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": { 2024-02-18T11:53:23.730 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mode", 2024-02-18T11:53:23.730 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.730 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.730 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.730 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "upmap", 2024-02-18T11:53:23.730 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.730 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.730 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.730 INFO:teuthology.orchestra.run.smithi012.stdout: "crush-compat", 2024-02-18T11:53:23.730 INFO:teuthology.orchestra.run.smithi012.stdout: "none", 2024-02-18T11:53:23.730 INFO:teuthology.orchestra.run.smithi012.stdout: "read", 2024-02-18T11:53:23.730 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap", 2024-02-18T11:53:23.730 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap-read" 2024-02-18T11:53:23.730 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.730 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Balancer mode", 2024-02-18T11:53:23.731 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.731 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.731 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.731 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.731 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_ids": { 2024-02-18T11:53:23.731 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pool_ids", 2024-02-18T11:53:23.731 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.731 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.731 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.731 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.731 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.731 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.731 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.731 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-02-18T11:53:23.731 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.732 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.732 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.732 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.732 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-02-18T11:53:23.732 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-02-18T11:53:23.732 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-02-18T11:53:23.732 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.732 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.732 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-02-18T11:53:23.732 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.732 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.732 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.732 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-02-18T11:53:23.732 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.732 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.733 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.733 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.733 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap_max_deviation": { 2024-02-18T11:53:23.733 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "upmap_max_deviation", 2024-02-18T11:53:23.733 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.733 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.733 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.733 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-02-18T11:53:23.733 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-02-18T11:53:23.733 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.733 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.733 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "deviation below which no optimization is attempted", 2024-02-18T11:53:23.733 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-02-18T11:53:23.733 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.733 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.733 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.734 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap_max_optimizations": { 2024-02-18T11:53:23.734 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "upmap_max_optimizations", 2024-02-18T11:53:23.734 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-02-18T11:53:23.734 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.734 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.734 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-02-18T11:53:23.734 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.734 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.734 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.734 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-02-18T11:53:23.734 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.734 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.734 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.734 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.734 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.735 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.735 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.735 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cephadm", 2024-02-18T11:53:23.735 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.735 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.735 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.735 INFO:teuthology.orchestra.run.smithi012.stdout: "agent_down_multiplier": { 2024-02-18T11:53:23.735 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "agent_down_multiplier", 2024-02-18T11:53:23.735 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-02-18T11:53:23.735 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.735 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.735 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "3.0", 2024-02-18T11:53:23.735 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.735 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.735 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.736 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-02-18T11:53:23.736 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.736 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.736 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.736 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.736 INFO:teuthology.orchestra.run.smithi012.stdout: "agent_refresh_rate": { 2024-02-18T11:53:23.736 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "agent_refresh_rate", 2024-02-18T11:53:23.736 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-02-18T11:53:23.736 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.736 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.736 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "20", 2024-02-18T11:53:23.736 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.736 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.736 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.736 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-02-18T11:53:23.737 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.737 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.737 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.737 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.737 INFO:teuthology.orchestra.run.smithi012.stdout: "agent_starting_port": { 2024-02-18T11:53:23.737 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "agent_starting_port", 2024-02-18T11:53:23.737 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.737 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.737 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.737 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "4721", 2024-02-18T11:53:23.737 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.737 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.737 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.737 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-02-18T11:53:23.737 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.738 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.738 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.738 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.738 INFO:teuthology.orchestra.run.smithi012.stdout: "allow_ptrace": { 2024-02-18T11:53:23.738 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "allow_ptrace", 2024-02-18T11:53:23.738 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.738 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.738 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.738 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.738 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.738 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.738 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.738 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-02-18T11:53:23.738 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-02-18T11:53:23.738 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.738 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.739 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.739 INFO:teuthology.orchestra.run.smithi012.stdout: "autotune_interval": { 2024-02-18T11:53:23.739 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "autotune_interval", 2024-02-18T11:53:23.739 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-02-18T11:53:23.739 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.739 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.739 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-02-18T11:53:23.739 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.739 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.739 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.739 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to autotune daemon memory", 2024-02-18T11:53:23.739 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.739 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.739 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.740 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.740 INFO:teuthology.orchestra.run.smithi012.stdout: "autotune_memory_target_ratio": { 2024-02-18T11:53:23.740 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "autotune_memory_target_ratio", 2024-02-18T11:53:23.740 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-02-18T11:53:23.740 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.740 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.740 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0.7", 2024-02-18T11:53:23.740 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.740 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.740 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.740 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-02-18T11:53:23.740 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.740 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.740 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.740 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.741 INFO:teuthology.orchestra.run.smithi012.stdout: "cephadm_log_destination": { 2024-02-18T11:53:23.741 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cephadm_log_destination", 2024-02-18T11:53:23.741 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.741 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.741 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.741 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.741 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.741 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.741 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.741 INFO:teuthology.orchestra.run.smithi012.stdout: "file", 2024-02-18T11:53:23.741 INFO:teuthology.orchestra.run.smithi012.stdout: "file,syslog", 2024-02-18T11:53:23.741 INFO:teuthology.orchestra.run.smithi012.stdout: "syslog" 2024-02-18T11:53:23.741 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.741 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-02-18T11:53:23.741 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.742 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.742 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.742 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.742 INFO:teuthology.orchestra.run.smithi012.stdout: "cgroups_split": { 2024-02-18T11:53:23.742 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cgroups_split", 2024-02-18T11:53:23.742 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.742 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.742 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.742 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.742 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.742 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.742 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.742 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-02-18T11:53:23.742 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.742 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.742 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.743 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.743 INFO:teuthology.orchestra.run.smithi012.stdout: "config_checks_enabled": { 2024-02-18T11:53:23.743 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "config_checks_enabled", 2024-02-18T11:53:23.743 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.743 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.743 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.743 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.743 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.743 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.743 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.743 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-02-18T11:53:23.743 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.743 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.743 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.743 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.744 INFO:teuthology.orchestra.run.smithi012.stdout: "config_dashboard": { 2024-02-18T11:53:23.744 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "config_dashboard", 2024-02-18T11:53:23.744 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.744 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.744 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.744 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.744 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.744 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.744 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.744 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-02-18T11:53:23.744 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.744 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.744 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.744 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.745 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_alertmanager": { 2024-02-18T11:53:23.745 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_alertmanager", 2024-02-18T11:53:23.745 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.745 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.745 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.745 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-02-18T11:53:23.745 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.745 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.745 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.745 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-02-18T11:53:23.745 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.745 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.745 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.745 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.745 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_base": { 2024-02-18T11:53:23.745 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_base", 2024-02-18T11:53:23.746 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.746 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.746 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.746 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/ceph", 2024-02-18T11:53:23.746 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.746 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.746 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.746 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Container image name, without the tag", 2024-02-18T11:53:23.746 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.746 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.746 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.746 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.746 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_elasticsearch": { 2024-02-18T11:53:23.746 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_elasticsearch", 2024-02-18T11:53:23.746 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.747 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.747 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.747 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-02-18T11:53:23.747 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.747 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.747 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.747 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "elasticsearch container image", 2024-02-18T11:53:23.747 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.747 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.747 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.747 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.747 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_grafana": { 2024-02-18T11:53:23.747 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_grafana", 2024-02-18T11:53:23.747 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.747 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.747 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.748 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-02-18T11:53:23.748 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.748 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.748 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.748 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-02-18T11:53:23.748 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.748 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.748 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.748 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.748 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_haproxy": { 2024-02-18T11:53:23.748 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_haproxy", 2024-02-18T11:53:23.748 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.748 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.748 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.749 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-02-18T11:53:23.749 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.749 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.749 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.749 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "HAproxy container image", 2024-02-18T11:53:23.749 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.749 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.749 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.749 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.749 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_jaeger_agent": { 2024-02-18T11:53:23.749 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_jaeger_agent", 2024-02-18T11:53:23.749 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.749 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.749 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.749 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-02-18T11:53:23.749 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.750 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.750 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.750 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Jaeger agent container image", 2024-02-18T11:53:23.750 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.750 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.750 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.750 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.750 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_jaeger_collector": { 2024-02-18T11:53:23.750 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_jaeger_collector", 2024-02-18T11:53:23.750 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.750 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.750 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.750 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-02-18T11:53:23.750 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.750 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.751 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.751 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Jaeger collector container image", 2024-02-18T11:53:23.751 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.751 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.751 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.751 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.751 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_jaeger_query": { 2024-02-18T11:53:23.751 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_jaeger_query", 2024-02-18T11:53:23.751 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.751 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.751 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.751 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-02-18T11:53:23.751 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.751 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.751 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.751 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Jaeger query container image", 2024-02-18T11:53:23.752 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.752 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.752 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.752 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.752 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_keepalived": { 2024-02-18T11:53:23.752 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_keepalived", 2024-02-18T11:53:23.752 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.752 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.752 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.752 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-02-18T11:53:23.752 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.752 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.752 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.752 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Keepalived container image", 2024-02-18T11:53:23.752 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.753 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.753 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.753 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.753 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_loki": { 2024-02-18T11:53:23.753 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_loki", 2024-02-18T11:53:23.753 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.753 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.753 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.753 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-02-18T11:53:23.753 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.753 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.753 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.753 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Loki container image", 2024-02-18T11:53:23.753 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.753 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.754 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.754 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.754 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_node_exporter": { 2024-02-18T11:53:23.754 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_node_exporter", 2024-02-18T11:53:23.754 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.754 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.754 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.754 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-02-18T11:53:23.754 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.754 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.754 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.754 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-02-18T11:53:23.754 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.754 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.754 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.755 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.755 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_nvmeof": { 2024-02-18T11:53:23.755 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_nvmeof", 2024-02-18T11:53:23.755 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.755 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.755 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.755 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2024-02-18T11:53:23.755 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.755 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.755 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.755 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Nvme-of container image", 2024-02-18T11:53:23.755 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.755 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.755 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.755 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.755 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_prometheus": { 2024-02-18T11:53:23.756 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_prometheus", 2024-02-18T11:53:23.756 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.756 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.756 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.756 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-02-18T11:53:23.756 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.756 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.756 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.756 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-02-18T11:53:23.756 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.756 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.756 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.756 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.756 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_promtail": { 2024-02-18T11:53:23.756 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_promtail", 2024-02-18T11:53:23.757 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.757 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.757 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.757 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-02-18T11:53:23.757 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.757 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.757 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.757 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Promtail container image", 2024-02-18T11:53:23.757 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.757 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.757 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.757 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.757 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_snmp_gateway": { 2024-02-18T11:53:23.757 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_snmp_gateway", 2024-02-18T11:53:23.757 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.758 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.758 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.758 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-02-18T11:53:23.758 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.758 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.758 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.758 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SNMP Gateway container image", 2024-02-18T11:53:23.758 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.758 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.758 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.758 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.758 INFO:teuthology.orchestra.run.smithi012.stdout: "container_init": { 2024-02-18T11:53:23.758 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_init", 2024-02-18T11:53:23.758 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.758 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.758 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.759 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.759 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.759 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.759 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.759 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Run podman/docker with `--init`", 2024-02-18T11:53:23.759 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.759 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.759 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.759 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.759 INFO:teuthology.orchestra.run.smithi012.stdout: "daemon_cache_timeout": { 2024-02-18T11:53:23.759 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "daemon_cache_timeout", 2024-02-18T11:53:23.759 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-02-18T11:53:23.759 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.759 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.759 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-02-18T11:53:23.760 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.760 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.760 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.760 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-02-18T11:53:23.760 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.760 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.760 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.760 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.760 INFO:teuthology.orchestra.run.smithi012.stdout: "default_cephadm_command_timeout": { 2024-02-18T11:53:23.760 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "default_cephadm_command_timeout", 2024-02-18T11:53:23.760 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.760 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.760 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.760 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "900", 2024-02-18T11:53:23.760 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.760 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.761 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.761 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-02-18T11:53:23.761 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.761 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.761 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.761 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.761 INFO:teuthology.orchestra.run.smithi012.stdout: "default_registry": { 2024-02-18T11:53:23.761 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "default_registry", 2024-02-18T11:53:23.761 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.761 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.761 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.761 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io", 2024-02-18T11:53:23.761 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.761 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.762 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.762 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-02-18T11:53:23.762 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.762 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.762 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.762 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.762 INFO:teuthology.orchestra.run.smithi012.stdout: "device_cache_timeout": { 2024-02-18T11:53:23.762 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "device_cache_timeout", 2024-02-18T11:53:23.762 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-02-18T11:53:23.762 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.762 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.762 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "1800", 2024-02-18T11:53:23.762 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.762 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.762 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.763 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "seconds to cache device inventory", 2024-02-18T11:53:23.763 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.763 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.763 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.763 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.763 INFO:teuthology.orchestra.run.smithi012.stdout: "device_enhanced_scan": { 2024-02-18T11:53:23.763 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "device_enhanced_scan", 2024-02-18T11:53:23.763 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.763 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.763 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.763 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.763 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.763 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.763 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.763 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use libstoragemgmt during device scans", 2024-02-18T11:53:23.764 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.764 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.764 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.764 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.764 INFO:teuthology.orchestra.run.smithi012.stdout: "facts_cache_timeout": { 2024-02-18T11:53:23.764 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "facts_cache_timeout", 2024-02-18T11:53:23.764 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-02-18T11:53:23.764 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.764 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.764 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-02-18T11:53:23.764 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.764 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.764 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.764 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "seconds to cache host facts data", 2024-02-18T11:53:23.764 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.765 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.765 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.765 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.765 INFO:teuthology.orchestra.run.smithi012.stdout: "host_check_interval": { 2024-02-18T11:53:23.765 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "host_check_interval", 2024-02-18T11:53:23.765 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-02-18T11:53:23.765 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.765 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.765 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-02-18T11:53:23.765 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.765 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.765 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.765 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to perform a host check", 2024-02-18T11:53:23.765 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.765 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.765 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.766 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.766 INFO:teuthology.orchestra.run.smithi012.stdout: "hw_monitoring": { 2024-02-18T11:53:23.766 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "hw_monitoring", 2024-02-18T11:53:23.766 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.766 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.766 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.766 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.766 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.766 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.766 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.766 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-02-18T11:53:23.766 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.766 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.766 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.766 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.767 INFO:teuthology.orchestra.run.smithi012.stdout: "inventory_list_all": { 2024-02-18T11:53:23.767 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "inventory_list_all", 2024-02-18T11:53:23.767 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.767 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.767 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.767 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.767 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.767 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.767 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.767 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-02-18T11:53:23.767 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.767 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.767 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.767 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.767 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.768 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.768 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.768 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.768 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.768 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.768 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.768 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.768 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.768 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.768 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.768 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.768 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.768 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.768 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.768 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.769 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.769 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.769 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.769 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.769 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.769 INFO:teuthology.orchestra.run.smithi012.stdout: "log_refresh_metadata": { 2024-02-18T11:53:23.769 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_refresh_metadata", 2024-02-18T11:53:23.769 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.769 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.769 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.769 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.769 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.769 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.769 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.769 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-02-18T11:53:23.770 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.770 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.770 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.770 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.770 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.770 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.770 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.770 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.770 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.770 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.770 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.770 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.770 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.770 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-02-18T11:53:23.770 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.770 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.771 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.771 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.771 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.771 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.771 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.771 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.771 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.771 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.771 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.771 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.771 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.771 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.771 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.771 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.771 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.772 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.772 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.772 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.772 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.772 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.772 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.772 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.772 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.772 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.772 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.772 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.772 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.772 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.772 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.772 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.773 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.773 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.773 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.773 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.773 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.773 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.773 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.773 INFO:teuthology.orchestra.run.smithi012.stdout: "manage_etc_ceph_ceph_conf": { 2024-02-18T11:53:23.773 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-02-18T11:53:23.773 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.773 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.773 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.773 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.773 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.773 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.774 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.774 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-02-18T11:53:23.774 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.774 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.774 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.774 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.774 INFO:teuthology.orchestra.run.smithi012.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-02-18T11:53:23.774 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-02-18T11:53:23.774 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.774 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.774 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.774 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "*", 2024-02-18T11:53:23.774 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.774 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.774 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.774 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-02-18T11:53:23.775 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.775 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.775 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.775 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.775 INFO:teuthology.orchestra.run.smithi012.stdout: "max_count_per_host": { 2024-02-18T11:53:23.775 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_count_per_host", 2024-02-18T11:53:23.775 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.775 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.775 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.775 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-02-18T11:53:23.775 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.775 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.775 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.775 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "max number of daemons per service per host", 2024-02-18T11:53:23.775 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.776 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.776 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.776 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.776 INFO:teuthology.orchestra.run.smithi012.stdout: "max_osd_draining_count": { 2024-02-18T11:53:23.776 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_osd_draining_count", 2024-02-18T11:53:23.776 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.776 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.776 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.776 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-02-18T11:53:23.779 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.780 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.780 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.780 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-02-18T11:53:23.780 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.780 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.780 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.780 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.781 INFO:teuthology.orchestra.run.smithi012.stdout: "migration_current": { 2024-02-18T11:53:23.781 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "migration_current", 2024-02-18T11:53:23.781 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.781 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.781 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.781 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.781 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.781 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.781 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.781 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "internal - do not modify", 2024-02-18T11:53:23.781 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.781 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.781 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.781 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.781 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": { 2024-02-18T11:53:23.782 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mode", 2024-02-18T11:53:23.782 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.782 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.782 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.782 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "root", 2024-02-18T11:53:23.782 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.782 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.782 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.782 INFO:teuthology.orchestra.run.smithi012.stdout: "cephadm-package", 2024-02-18T11:53:23.782 INFO:teuthology.orchestra.run.smithi012.stdout: "root" 2024-02-18T11:53:23.782 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.782 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "mode for remote execution of cephadm", 2024-02-18T11:53:23.782 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.782 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.782 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.783 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.783 INFO:teuthology.orchestra.run.smithi012.stdout: "oob_default_addr": { 2024-02-18T11:53:23.783 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "oob_default_addr", 2024-02-18T11:53:23.783 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.783 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.783 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.783 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "169.254.1.1", 2024-02-18T11:53:23.783 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.783 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.783 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.783 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Default address for RedFish API (oob management).", 2024-02-18T11:53:23.783 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.783 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.783 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.783 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.783 INFO:teuthology.orchestra.run.smithi012.stdout: "prometheus_alerts_path": { 2024-02-18T11:53:23.784 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "prometheus_alerts_path", 2024-02-18T11:53:23.784 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.784 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.784 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.784 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-02-18T11:53:23.784 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.784 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.784 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.784 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-02-18T11:53:23.784 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.784 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.784 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.784 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.784 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_insecure": { 2024-02-18T11:53:23.785 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_insecure", 2024-02-18T11:53:23.785 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.785 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.785 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.785 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.785 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.785 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.785 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.785 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-02-18T11:53:23.785 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.785 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.785 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.785 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.785 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_password": { 2024-02-18T11:53:23.785 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_password", 2024-02-18T11:53:23.785 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.786 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.786 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.786 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.786 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.786 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.786 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.786 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-02-18T11:53:23.786 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.786 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.786 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.786 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.786 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_url": { 2024-02-18T11:53:23.786 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_url", 2024-02-18T11:53:23.786 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.786 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.787 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.787 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.787 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.787 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.787 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.787 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-02-18T11:53:23.787 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.787 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.787 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.787 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.787 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_username": { 2024-02-18T11:53:23.787 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_username", 2024-02-18T11:53:23.787 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.787 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.787 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.788 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.788 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.788 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.788 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.788 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-02-18T11:53:23.788 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.788 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.788 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.788 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.788 INFO:teuthology.orchestra.run.smithi012.stdout: "secure_monitoring_stack": { 2024-02-18T11:53:23.788 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "secure_monitoring_stack", 2024-02-18T11:53:23.788 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.788 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.788 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.788 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.789 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.789 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.789 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.789 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-02-18T11:53:23.789 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.789 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.789 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.789 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.789 INFO:teuthology.orchestra.run.smithi012.stdout: "service_discovery_port": { 2024-02-18T11:53:23.789 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "service_discovery_port", 2024-02-18T11:53:23.789 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.789 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.789 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.789 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8765", 2024-02-18T11:53:23.789 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.789 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.790 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.790 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "cephadm service discovery port", 2024-02-18T11:53:23.790 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.790 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.790 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.790 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.790 INFO:teuthology.orchestra.run.smithi012.stdout: "ssh_config_file": { 2024-02-18T11:53:23.790 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssh_config_file", 2024-02-18T11:53:23.790 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.790 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.790 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.790 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.790 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.790 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.790 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.791 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-02-18T11:53:23.791 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.791 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.791 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.791 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.791 INFO:teuthology.orchestra.run.smithi012.stdout: "use_agent": { 2024-02-18T11:53:23.791 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "use_agent", 2024-02-18T11:53:23.791 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.791 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.791 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.791 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.791 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.791 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.791 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.791 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-02-18T11:53:23.791 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.792 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.792 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.792 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.792 INFO:teuthology.orchestra.run.smithi012.stdout: "use_repo_digest": { 2024-02-18T11:53:23.792 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "use_repo_digest", 2024-02-18T11:53:23.792 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.792 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.792 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.792 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.792 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.792 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.792 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.792 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-02-18T11:53:23.792 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.792 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.793 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.793 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.793 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_on_failed_host_check": { 2024-02-18T11:53:23.793 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_on_failed_host_check", 2024-02-18T11:53:23.793 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.793 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.793 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.793 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.793 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.793 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.793 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.793 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise a health warning if the host check fails", 2024-02-18T11:53:23.793 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.793 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.794 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.794 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.794 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_on_stray_daemons": { 2024-02-18T11:53:23.794 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_on_stray_daemons", 2024-02-18T11:53:23.794 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.794 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.794 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.794 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.794 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.794 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.794 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.794 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-02-18T11:53:23.794 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.794 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.794 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.795 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.795 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_on_stray_hosts": { 2024-02-18T11:53:23.795 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_on_stray_hosts", 2024-02-18T11:53:23.795 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.795 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.795 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.795 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.795 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.795 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.795 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.795 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-02-18T11:53:23.795 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.795 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.795 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.795 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.795 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.796 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.796 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.796 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crash", 2024-02-18T11:53:23.796 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.796 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.796 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.796 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.796 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.796 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.796 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.796 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.796 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.796 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.796 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.796 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.797 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.797 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.797 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.797 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.797 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.797 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.797 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.797 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.797 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.797 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.797 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.797 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.797 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.797 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.797 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.798 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.798 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.798 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.798 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.798 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.798 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.798 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.798 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.798 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.798 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.798 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.798 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.798 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.798 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.798 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.799 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.799 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.799 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.799 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.799 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.799 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.799 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.799 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.799 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.799 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.799 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.799 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.799 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.799 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.799 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.800 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.800 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.800 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.800 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.800 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.800 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.800 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.800 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.800 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.800 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.800 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.800 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.800 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.800 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.800 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.801 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.801 INFO:teuthology.orchestra.run.smithi012.stdout: "retain_interval": { 2024-02-18T11:53:23.801 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "retain_interval", 2024-02-18T11:53:23.801 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-02-18T11:53:23.801 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.801 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.801 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "31536000", 2024-02-18T11:53:23.801 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.801 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.801 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.801 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how long to retain crashes before pruning them", 2024-02-18T11:53:23.801 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.801 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.801 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.801 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.801 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_recent_interval": { 2024-02-18T11:53:23.802 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_recent_interval", 2024-02-18T11:53:23.802 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-02-18T11:53:23.802 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.802 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.802 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "1209600", 2024-02-18T11:53:23.802 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.802 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.802 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.802 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "time interval in which to warn about recent crashes", 2024-02-18T11:53:23.802 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.802 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.802 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.802 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.802 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.802 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.802 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.803 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "dashboard", 2024-02-18T11:53:23.803 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.803 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.803 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.803 INFO:teuthology.orchestra.run.smithi012.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-02-18T11:53:23.803 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-02-18T11:53:23.803 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.803 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.803 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.803 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-02-18T11:53:23.803 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.803 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.803 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.803 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.803 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.804 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.804 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.804 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.804 INFO:teuthology.orchestra.run.smithi012.stdout: "ALERTMANAGER_API_HOST": { 2024-02-18T11:53:23.804 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ALERTMANAGER_API_HOST", 2024-02-18T11:53:23.804 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.804 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.804 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.804 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.804 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.804 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.804 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.804 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.804 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.804 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.805 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.805 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.805 INFO:teuthology.orchestra.run.smithi012.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-02-18T11:53:23.805 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-02-18T11:53:23.805 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.805 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.805 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.805 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.805 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.805 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.805 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.805 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.805 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.805 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.805 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.805 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.806 INFO:teuthology.orchestra.run.smithi012.stdout: "AUDIT_API_ENABLED": { 2024-02-18T11:53:23.806 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "AUDIT_API_ENABLED", 2024-02-18T11:53:23.806 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.806 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.806 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.806 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.806 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.806 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.806 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.806 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.806 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.806 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.806 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.806 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.806 INFO:teuthology.orchestra.run.smithi012.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-02-18T11:53:23.807 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-02-18T11:53:23.807 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.807 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.807 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.807 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.807 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.807 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.807 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.807 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.807 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.807 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.807 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.807 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.807 INFO:teuthology.orchestra.run.smithi012.stdout: "ENABLE_BROWSABLE_API": { 2024-02-18T11:53:23.807 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ENABLE_BROWSABLE_API", 2024-02-18T11:53:23.807 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.808 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.808 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.808 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.808 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.808 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.808 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.808 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.808 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.808 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.808 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.808 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.808 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-02-18T11:53:23.808 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-02-18T11:53:23.808 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.809 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.809 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.809 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.809 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.809 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.809 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.809 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.809 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.809 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.809 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.809 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.809 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-02-18T11:53:23.809 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-02-18T11:53:23.809 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.809 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.810 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.810 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.810 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.810 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.810 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.810 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.810 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.810 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.810 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.810 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.810 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-02-18T11:53:23.810 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-02-18T11:53:23.810 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.810 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.810 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.810 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.811 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.811 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.811 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.811 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.811 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.811 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.811 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.811 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.811 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-02-18T11:53:23.811 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-02-18T11:53:23.811 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.811 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.811 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.811 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.811 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.812 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.812 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.812 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.812 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.812 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.812 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.812 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.812 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_NFS": { 2024-02-18T11:53:23.812 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-02-18T11:53:23.812 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.812 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.812 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.812 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.812 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.812 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.813 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.813 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.813 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.813 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.813 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.813 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.813 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_RBD": { 2024-02-18T11:53:23.813 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-02-18T11:53:23.813 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.813 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.813 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.813 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.813 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.813 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.813 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.814 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.814 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.814 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.814 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.814 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.814 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_RGW": { 2024-02-18T11:53:23.814 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-02-18T11:53:23.814 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.814 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.814 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.814 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.814 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.814 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.814 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.814 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.815 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.815 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.815 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.815 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.815 INFO:teuthology.orchestra.run.smithi012.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-02-18T11:53:23.815 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-02-18T11:53:23.815 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.815 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.815 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.815 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.815 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.815 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.815 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.815 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.815 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.816 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.816 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.816 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.816 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_PASSWORD": { 2024-02-18T11:53:23.816 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_PASSWORD", 2024-02-18T11:53:23.816 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.816 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.816 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.816 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "admin", 2024-02-18T11:53:23.816 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.816 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.816 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.816 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.816 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.816 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.817 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.817 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.817 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-02-18T11:53:23.817 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-02-18T11:53:23.817 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.817 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.817 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.817 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.817 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.817 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.817 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.817 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.817 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.817 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.817 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.817 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.818 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_URL": { 2024-02-18T11:53:23.818 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_URL", 2024-02-18T11:53:23.818 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.818 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.818 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.818 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.818 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.818 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.818 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.818 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.818 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.818 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.818 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.818 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.818 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_USERNAME": { 2024-02-18T11:53:23.819 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_USERNAME", 2024-02-18T11:53:23.819 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.819 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.819 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.819 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "admin", 2024-02-18T11:53:23.819 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.819 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.819 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.819 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.819 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.819 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.819 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.819 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.819 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-02-18T11:53:23.819 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-02-18T11:53:23.819 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.820 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.820 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.820 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.820 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.820 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.820 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.820 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.820 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.820 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.820 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.820 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.820 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-02-18T11:53:23.820 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-02-18T11:53:23.820 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.820 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.821 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.821 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.821 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.821 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.821 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.821 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.821 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.821 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.821 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.821 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.821 INFO:teuthology.orchestra.run.smithi012.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-02-18T11:53:23.821 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-02-18T11:53:23.821 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.821 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.821 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.821 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.822 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.822 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.822 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.822 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.822 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.822 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.822 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.822 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.822 INFO:teuthology.orchestra.run.smithi012.stdout: "ISSUE_TRACKER_API_KEY": { 2024-02-18T11:53:23.822 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-02-18T11:53:23.822 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.822 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.822 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.822 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.822 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.823 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.823 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.823 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.823 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.823 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.823 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.823 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.823 INFO:teuthology.orchestra.run.smithi012.stdout: "PROMETHEUS_API_HOST": { 2024-02-18T11:53:23.823 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PROMETHEUS_API_HOST", 2024-02-18T11:53:23.823 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.823 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.823 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.823 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.823 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.824 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.824 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.824 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.824 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.824 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.824 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.824 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.824 INFO:teuthology.orchestra.run.smithi012.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-02-18T11:53:23.824 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-02-18T11:53:23.824 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.824 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.824 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.824 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.824 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.824 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.825 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.825 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.825 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.825 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.825 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.825 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.825 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-02-18T11:53:23.825 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-02-18T11:53:23.825 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.825 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.825 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.825 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.825 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.825 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.825 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.826 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.826 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.826 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.826 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.826 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.826 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-02-18T11:53:23.826 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-02-18T11:53:23.826 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.826 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.826 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.826 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.826 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.826 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.826 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.826 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.827 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.827 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.827 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.827 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.827 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-02-18T11:53:23.827 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-02-18T11:53:23.827 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.827 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.827 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.827 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.827 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.827 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.827 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.827 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.827 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.828 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.828 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.828 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.828 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-02-18T11:53:23.828 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-02-18T11:53:23.828 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.828 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.828 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.828 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.828 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.828 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.828 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.828 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.828 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.828 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.829 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.829 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.829 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-02-18T11:53:23.829 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-02-18T11:53:23.829 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.829 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.829 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.829 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.829 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.829 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.829 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.829 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.829 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.829 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.829 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.829 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.830 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-02-18T11:53:23.830 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-02-18T11:53:23.830 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.830 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.830 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.830 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.830 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.830 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.830 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.830 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.830 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.830 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.830 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.830 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.830 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-02-18T11:53:23.830 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-02-18T11:53:23.831 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.831 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.831 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.831 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.831 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.831 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.831 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.831 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.831 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.831 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.831 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.831 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.831 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_ENABLED": { 2024-02-18T11:53:23.831 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_ENABLED", 2024-02-18T11:53:23.831 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.832 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.832 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.832 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.832 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.832 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.832 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.832 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.832 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.832 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.832 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.832 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.832 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-02-18T11:53:23.832 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-02-18T11:53:23.832 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.832 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.833 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.833 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-02-18T11:53:23.833 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.833 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.833 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.833 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.833 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.833 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.833 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.833 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.833 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-02-18T11:53:23.833 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-02-18T11:53:23.833 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.833 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.833 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.833 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-02-18T11:53:23.834 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.834 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.834 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.834 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.834 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.834 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.834 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.834 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.834 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-02-18T11:53:23.834 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-02-18T11:53:23.834 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.834 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.834 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.834 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8", 2024-02-18T11:53:23.835 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.835 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.835 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.835 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.835 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.835 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.835 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.835 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.835 INFO:teuthology.orchestra.run.smithi012.stdout: "REST_REQUESTS_TIMEOUT": { 2024-02-18T11:53:23.835 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-02-18T11:53:23.836 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.836 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.836 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.836 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "45", 2024-02-18T11:53:23.836 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.836 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.836 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.836 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.836 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.837 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.837 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.837 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.837 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_ACCESS_KEY": { 2024-02-18T11:53:23.837 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_ACCESS_KEY", 2024-02-18T11:53:23.837 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.837 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.837 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.837 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.837 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.837 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.837 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.837 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.837 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.837 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.837 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.838 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.838 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-02-18T11:53:23.838 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-02-18T11:53:23.838 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.838 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.838 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.838 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "admin", 2024-02-18T11:53:23.838 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.838 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.838 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.838 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.838 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.838 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.838 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.839 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.839 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_SECRET_KEY": { 2024-02-18T11:53:23.839 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_SECRET_KEY", 2024-02-18T11:53:23.839 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.839 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.839 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.839 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.839 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.839 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.839 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.839 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.839 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.839 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.839 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.839 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.839 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_SSL_VERIFY": { 2024-02-18T11:53:23.840 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_SSL_VERIFY", 2024-02-18T11:53:23.840 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.840 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.840 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.840 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.840 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.840 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.840 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.840 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.840 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.840 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.840 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.840 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.840 INFO:teuthology.orchestra.run.smithi012.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-02-18T11:53:23.841 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-02-18T11:53:23.841 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.841 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.841 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.841 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-02-18T11:53:23.841 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.841 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.841 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.841 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.841 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.841 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.841 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.841 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.841 INFO:teuthology.orchestra.run.smithi012.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-02-18T11:53:23.841 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-02-18T11:53:23.841 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.842 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.842 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.842 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-02-18T11:53:23.842 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.842 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.842 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.842 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.842 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.842 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.842 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.842 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.842 INFO:teuthology.orchestra.run.smithi012.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-02-18T11:53:23.842 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-02-18T11:53:23.842 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.842 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.843 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.843 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-02-18T11:53:23.843 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.843 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.843 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.843 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.843 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.843 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.843 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.843 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.843 INFO:teuthology.orchestra.run.smithi012.stdout: "cross_origin_url": { 2024-02-18T11:53:23.843 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cross_origin_url", 2024-02-18T11:53:23.843 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.843 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.843 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.844 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.844 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.844 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.844 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.844 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.844 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.844 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.844 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.844 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.844 INFO:teuthology.orchestra.run.smithi012.stdout: "crt_file": { 2024-02-18T11:53:23.844 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crt_file", 2024-02-18T11:53:23.844 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.844 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.844 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.844 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.845 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.845 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.845 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.845 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.845 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.845 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.845 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.845 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.845 INFO:teuthology.orchestra.run.smithi012.stdout: "debug": { 2024-02-18T11:53:23.845 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "debug", 2024-02-18T11:53:23.845 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.845 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.845 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.845 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.845 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.846 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.846 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.846 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Enable/disable debug options", 2024-02-18T11:53:23.846 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.846 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.846 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.846 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.846 INFO:teuthology.orchestra.run.smithi012.stdout: "jwt_token_ttl": { 2024-02-18T11:53:23.846 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "jwt_token_ttl", 2024-02-18T11:53:23.846 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.846 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.846 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.846 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "28800", 2024-02-18T11:53:23.846 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.846 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.846 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.847 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.847 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.847 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.847 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.847 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.847 INFO:teuthology.orchestra.run.smithi012.stdout: "key_file": { 2024-02-18T11:53:23.847 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "key_file", 2024-02-18T11:53:23.847 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.847 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.847 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.847 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.847 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.847 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.847 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.847 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.847 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.848 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.848 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.848 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.848 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.848 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.848 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.848 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.848 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.848 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.848 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.848 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.848 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.848 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.848 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.848 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.849 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.849 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.849 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.849 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.849 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.849 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.849 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.849 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.849 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.849 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.849 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.849 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.849 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.849 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.849 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.850 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.850 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.850 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.850 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.850 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.850 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.850 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.850 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.850 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.850 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.850 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.850 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.850 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.850 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.850 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.850 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.851 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.851 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.851 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.851 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.851 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.851 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.851 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.851 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.851 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.851 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.851 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.851 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.851 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.851 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.851 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.852 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.852 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.852 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.852 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.852 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.852 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.852 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.852 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.852 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.852 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.852 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.852 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.852 INFO:teuthology.orchestra.run.smithi012.stdout: "motd": { 2024-02-18T11:53:23.852 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "motd", 2024-02-18T11:53:23.852 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.853 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.853 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.853 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.853 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.853 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.853 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.853 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "The message of the day", 2024-02-18T11:53:23.853 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.853 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.853 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.853 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.853 INFO:teuthology.orchestra.run.smithi012.stdout: "redirect_resolve_ip_addr": { 2024-02-18T11:53:23.853 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "redirect_resolve_ip_addr", 2024-02-18T11:53:23.853 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.853 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.854 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.854 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.854 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.854 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.854 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.854 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.854 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.854 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.854 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.854 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.854 INFO:teuthology.orchestra.run.smithi012.stdout: "server_addr": { 2024-02-18T11:53:23.854 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_addr", 2024-02-18T11:53:23.854 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.854 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.854 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.855 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "::", 2024-02-18T11:53:23.855 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.855 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.855 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.855 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.855 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.855 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.855 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.855 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.855 INFO:teuthology.orchestra.run.smithi012.stdout: "server_port": { 2024-02-18T11:53:23.855 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_port", 2024-02-18T11:53:23.855 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.855 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.855 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.855 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8080", 2024-02-18T11:53:23.856 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.856 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.856 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.856 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.856 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.856 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.856 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.856 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.856 INFO:teuthology.orchestra.run.smithi012.stdout: "ssl": { 2024-02-18T11:53:23.856 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssl", 2024-02-18T11:53:23.856 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.856 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.856 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.856 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.856 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.857 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.857 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.857 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.857 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.857 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.857 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.857 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.857 INFO:teuthology.orchestra.run.smithi012.stdout: "ssl_server_port": { 2024-02-18T11:53:23.857 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssl_server_port", 2024-02-18T11:53:23.857 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.857 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.857 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.857 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8443", 2024-02-18T11:53:23.857 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.857 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.857 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.858 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.858 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.858 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.858 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.858 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.858 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_behaviour": { 2024-02-18T11:53:23.858 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_behaviour", 2024-02-18T11:53:23.858 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.858 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.858 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.858 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "redirect", 2024-02-18T11:53:23.858 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.858 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.858 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.859 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.859 INFO:teuthology.orchestra.run.smithi012.stdout: "redirect" 2024-02-18T11:53:23.859 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.859 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.859 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.859 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.859 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.859 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.859 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_error_status_code": { 2024-02-18T11:53:23.859 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_error_status_code", 2024-02-18T11:53:23.859 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.859 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.859 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.859 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "500", 2024-02-18T11:53:23.859 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "400", 2024-02-18T11:53:23.859 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "599", 2024-02-18T11:53:23.860 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.860 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.860 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.860 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.860 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.860 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.860 INFO:teuthology.orchestra.run.smithi012.stdout: "url_prefix": { 2024-02-18T11:53:23.860 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "url_prefix", 2024-02-18T11:53:23.860 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.860 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.860 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.860 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.860 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.860 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.860 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.861 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.861 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.861 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.861 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.861 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.861 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.861 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.861 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.861 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "devicehealth", 2024-02-18T11:53:23.861 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.861 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.861 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.861 INFO:teuthology.orchestra.run.smithi012.stdout: "enable_monitoring": { 2024-02-18T11:53:23.861 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enable_monitoring", 2024-02-18T11:53:23.861 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.862 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.862 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.862 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.862 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.862 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.862 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.862 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "monitor device health metrics", 2024-02-18T11:53:23.862 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.862 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.862 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.862 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.862 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.862 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.862 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.862 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.862 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.863 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.863 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.863 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.863 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.863 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.863 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.863 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.863 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.863 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.863 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.863 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.863 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.863 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.863 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.863 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.864 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.864 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.864 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.864 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.864 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.864 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.864 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.864 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.864 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.864 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.864 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.864 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.864 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.864 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.864 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.865 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.865 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.865 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.865 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.865 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.865 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.865 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.865 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.865 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.865 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.865 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.865 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.865 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.865 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.865 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.866 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.866 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.866 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.866 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.866 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.866 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.866 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.866 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.866 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.866 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.866 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.866 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.866 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.866 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.866 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.867 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.867 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.867 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.867 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.867 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.867 INFO:teuthology.orchestra.run.smithi012.stdout: "mark_out_threshold": { 2024-02-18T11:53:23.867 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mark_out_threshold", 2024-02-18T11:53:23.867 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-02-18T11:53:23.867 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.867 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.867 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "2419200", 2024-02-18T11:53:23.867 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.867 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.867 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.867 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-02-18T11:53:23.867 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.868 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.868 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.868 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.868 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_name": { 2024-02-18T11:53:23.868 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pool_name", 2024-02-18T11:53:23.868 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.868 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.868 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.868 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "device_health_metrics", 2024-02-18T11:53:23.868 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.868 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.868 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.868 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "name of pool in which to store device health metrics", 2024-02-18T11:53:23.869 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.869 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.869 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.869 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.869 INFO:teuthology.orchestra.run.smithi012.stdout: "retention_period": { 2024-02-18T11:53:23.869 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "retention_period", 2024-02-18T11:53:23.869 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-02-18T11:53:23.869 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.869 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.869 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "15552000", 2024-02-18T11:53:23.869 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.869 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.869 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.869 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how long to retain device health metrics", 2024-02-18T11:53:23.869 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.869 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.870 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.870 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.870 INFO:teuthology.orchestra.run.smithi012.stdout: "scrape_frequency": { 2024-02-18T11:53:23.870 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "scrape_frequency", 2024-02-18T11:53:23.870 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-02-18T11:53:23.870 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.870 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.870 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "86400", 2024-02-18T11:53:23.870 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.870 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.870 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.870 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to scrape device health metrics", 2024-02-18T11:53:23.870 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.870 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.870 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.871 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.871 INFO:teuthology.orchestra.run.smithi012.stdout: "self_heal": { 2024-02-18T11:53:23.871 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "self_heal", 2024-02-18T11:53:23.871 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.871 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.871 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.871 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.871 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.871 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.871 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.871 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-02-18T11:53:23.871 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.871 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.871 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.871 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.872 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-02-18T11:53:23.872 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-02-18T11:53:23.872 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-02-18T11:53:23.872 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.872 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.872 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-02-18T11:53:23.872 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.872 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.872 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.872 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to wake up and check device health", 2024-02-18T11:53:23.872 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.872 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.872 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.872 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.873 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_threshold": { 2024-02-18T11:53:23.873 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_threshold", 2024-02-18T11:53:23.873 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-02-18T11:53:23.873 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.873 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.873 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "7257600", 2024-02-18T11:53:23.873 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.873 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.873 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.873 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-02-18T11:53:23.873 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.873 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.873 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.873 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.873 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.874 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.874 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.874 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "diskprediction_local", 2024-02-18T11:53:23.874 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.874 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.874 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.874 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.874 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.874 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.874 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.874 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.874 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.874 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.874 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.875 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.875 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.875 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.875 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.875 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.875 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.875 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.875 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.875 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.875 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.875 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.875 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.875 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.875 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.875 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.876 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.876 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.876 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.876 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.876 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.876 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.876 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.876 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.876 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.876 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.876 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.876 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.876 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.876 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.876 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.877 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.877 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.877 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.877 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.877 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.877 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.877 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.877 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.877 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.877 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.877 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.877 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.877 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.877 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.877 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.877 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.878 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.878 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.878 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.878 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.878 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.878 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.878 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.878 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.878 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.878 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.878 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.878 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.878 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.878 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.878 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.879 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.879 INFO:teuthology.orchestra.run.smithi012.stdout: "predict_interval": { 2024-02-18T11:53:23.879 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "predict_interval", 2024-02-18T11:53:23.879 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.879 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.879 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.879 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "86400", 2024-02-18T11:53:23.879 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.879 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.879 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.879 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.879 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.879 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.879 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.879 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.879 INFO:teuthology.orchestra.run.smithi012.stdout: "predictor_model": { 2024-02-18T11:53:23.880 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "predictor_model", 2024-02-18T11:53:23.880 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.880 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.880 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.880 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "prophetstor", 2024-02-18T11:53:23.880 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.880 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.880 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.880 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.880 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.880 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.880 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.880 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.880 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-02-18T11:53:23.880 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-02-18T11:53:23.881 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.881 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.881 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.881 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-02-18T11:53:23.881 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.881 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.881 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.881 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.881 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.881 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.881 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.881 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.881 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.881 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.881 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.882 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "influx", 2024-02-18T11:53:23.882 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": false, 2024-02-18T11:53:23.882 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "influxdb python module not found", 2024-02-18T11:53:23.882 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.882 INFO:teuthology.orchestra.run.smithi012.stdout: "batch_size": { 2024-02-18T11:53:23.882 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "batch_size", 2024-02-18T11:53:23.882 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.882 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.882 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.882 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5000", 2024-02-18T11:53:23.882 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.882 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.882 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.882 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-02-18T11:53:23.882 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.882 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.883 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.883 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.883 INFO:teuthology.orchestra.run.smithi012.stdout: "database": { 2024-02-18T11:53:23.883 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "database", 2024-02-18T11:53:23.883 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.883 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.883 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.883 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "ceph", 2024-02-18T11:53:23.883 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.883 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.883 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.883 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-02-18T11:53:23.883 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.884 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.884 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.884 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.884 INFO:teuthology.orchestra.run.smithi012.stdout: "hostname": { 2024-02-18T11:53:23.884 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "hostname", 2024-02-18T11:53:23.884 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.884 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.884 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.884 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.884 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.884 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.884 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.884 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "InfluxDB server hostname", 2024-02-18T11:53:23.885 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.885 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.885 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.885 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.885 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-02-18T11:53:23.885 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-02-18T11:53:23.885 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-02-18T11:53:23.885 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.885 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.885 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "30", 2024-02-18T11:53:23.885 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "5", 2024-02-18T11:53:23.885 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.885 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.885 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-02-18T11:53:23.885 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.886 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.886 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.886 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.886 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.886 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.886 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.886 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.886 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.886 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.886 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.886 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.886 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.886 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.886 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.886 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.887 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.887 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.887 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.887 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.887 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.887 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.887 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.887 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.887 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.887 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.887 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.887 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.887 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.887 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.887 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.888 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.888 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.888 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.888 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.888 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.888 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.888 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.888 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.888 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.888 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.888 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.888 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.888 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.888 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.888 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.889 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.889 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.889 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.889 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.889 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.889 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.889 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.889 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.889 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.889 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.889 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.889 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.889 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.889 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.889 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.890 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.890 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.890 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.890 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.890 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.890 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.890 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.890 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.890 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.890 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.890 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.890 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.890 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.890 INFO:teuthology.orchestra.run.smithi012.stdout: "password": { 2024-02-18T11:53:23.890 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "password", 2024-02-18T11:53:23.890 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.891 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.891 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.891 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.891 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.891 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.891 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.891 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "password of InfluxDB server user", 2024-02-18T11:53:23.891 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.891 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.891 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.891 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.891 INFO:teuthology.orchestra.run.smithi012.stdout: "port": { 2024-02-18T11:53:23.891 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "port", 2024-02-18T11:53:23.891 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.891 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.892 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.892 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8086", 2024-02-18T11:53:23.892 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.892 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.892 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.892 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "InfluxDB server port", 2024-02-18T11:53:23.892 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.892 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.892 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.892 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.892 INFO:teuthology.orchestra.run.smithi012.stdout: "ssl": { 2024-02-18T11:53:23.892 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssl", 2024-02-18T11:53:23.892 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.892 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.892 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.893 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "false", 2024-02-18T11:53:23.893 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.893 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.893 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.893 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-02-18T11:53:23.893 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.893 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.893 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.893 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.893 INFO:teuthology.orchestra.run.smithi012.stdout: "threads": { 2024-02-18T11:53:23.893 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "threads", 2024-02-18T11:53:23.893 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.893 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.893 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.893 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-02-18T11:53:23.893 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-02-18T11:53:23.894 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "32", 2024-02-18T11:53:23.894 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.894 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-02-18T11:53:23.894 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.894 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.894 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.894 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.894 INFO:teuthology.orchestra.run.smithi012.stdout: "username": { 2024-02-18T11:53:23.894 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "username", 2024-02-18T11:53:23.894 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.894 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.894 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.894 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.894 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.894 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.895 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.895 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "username of InfluxDB server user", 2024-02-18T11:53:23.895 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.895 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.895 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.895 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.895 INFO:teuthology.orchestra.run.smithi012.stdout: "verify_ssl": { 2024-02-18T11:53:23.895 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "verify_ssl", 2024-02-18T11:53:23.895 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.895 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.895 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.895 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "true", 2024-02-18T11:53:23.895 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.895 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.895 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.896 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-02-18T11:53:23.896 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.896 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.896 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.896 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.896 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.896 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.896 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.896 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "insights", 2024-02-18T11:53:23.896 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.896 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.896 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.896 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.896 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.896 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.897 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.897 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.897 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.897 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.897 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.897 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.897 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.897 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.897 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.897 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.897 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.897 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.897 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.897 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.897 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.897 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.898 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.898 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.898 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.898 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.898 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.898 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.898 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.898 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.898 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.898 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.898 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.898 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.898 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.898 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.898 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.899 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.899 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.899 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.899 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.899 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.899 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.899 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.899 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.899 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.899 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.899 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.899 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.899 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.899 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.900 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.900 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.900 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.900 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.900 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.900 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.900 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.900 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.900 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.900 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.900 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.900 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.900 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.900 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.901 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.901 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.901 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.901 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.901 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.901 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.901 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.901 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.901 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.901 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.901 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.901 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "iostat", 2024-02-18T11:53:23.901 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.901 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.901 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.901 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.902 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.902 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.902 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.902 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.902 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.902 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.902 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.902 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.902 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.902 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.902 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.902 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.902 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.902 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.902 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.903 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.903 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.903 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.903 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.903 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.903 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.903 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.903 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.903 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.903 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.903 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.903 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.903 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.903 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.903 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.904 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.904 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.904 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.904 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.904 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.904 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.904 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.904 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.904 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.904 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.904 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.904 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.904 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.904 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.904 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.905 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.905 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.905 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.905 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.905 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.905 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.905 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.905 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.905 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.905 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.905 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.905 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.905 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.905 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.905 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.906 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.906 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.906 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.906 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.906 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.906 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.906 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.906 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.906 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.906 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.906 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.906 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.906 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "k8sevents", 2024-02-18T11:53:23.906 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.906 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.907 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.907 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_event_retention_days": { 2024-02-18T11:53:23.907 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ceph_event_retention_days", 2024-02-18T11:53:23.907 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.907 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.907 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.907 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "7", 2024-02-18T11:53:23.907 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.907 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.907 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.907 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Days to hold ceph event information within local cache", 2024-02-18T11:53:23.907 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.907 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.907 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.907 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.908 INFO:teuthology.orchestra.run.smithi012.stdout: "config_check_secs": { 2024-02-18T11:53:23.908 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "config_check_secs", 2024-02-18T11:53:23.908 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.908 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.908 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.908 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-02-18T11:53:23.908 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "10", 2024-02-18T11:53:23.908 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.908 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.908 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-02-18T11:53:23.908 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.908 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.908 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.908 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.909 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.909 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.909 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.909 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.909 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.909 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.909 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.909 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.909 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.909 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.909 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.909 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.909 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.909 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.909 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.909 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.910 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.910 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.910 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.910 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.910 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.910 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.910 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.910 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.910 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.910 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.910 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.910 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.910 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.910 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.910 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.911 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.911 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.911 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.911 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.911 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.911 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.911 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.911 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.911 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.911 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.911 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.911 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.911 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.911 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.911 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.911 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.912 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.912 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.912 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.912 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.912 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.912 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.912 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.912 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.912 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.912 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.912 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.912 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.912 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.912 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.912 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.913 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.913 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.913 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.913 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.913 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.913 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.913 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.913 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.913 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.913 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.913 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.913 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "localpool", 2024-02-18T11:53:23.913 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.913 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.913 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.913 INFO:teuthology.orchestra.run.smithi012.stdout: "failure_domain": { 2024-02-18T11:53:23.914 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "failure_domain", 2024-02-18T11:53:23.914 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.914 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.914 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.914 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "host", 2024-02-18T11:53:23.914 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.914 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.914 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.914 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "failure domain for any created local pool", 2024-02-18T11:53:23.914 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-02-18T11:53:23.914 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.914 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.914 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.915 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.915 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.915 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.915 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.915 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.915 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.915 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.915 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.915 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.915 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.915 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.915 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.915 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.915 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.915 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.916 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.916 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.916 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.916 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.916 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.916 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.916 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.916 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.916 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.916 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.916 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.916 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.916 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.916 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.917 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.917 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.917 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.917 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.917 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.917 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.917 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.917 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.917 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.917 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.917 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.917 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.917 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.917 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.917 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.918 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.918 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.918 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.918 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.918 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.918 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.918 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.918 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.918 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.918 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.918 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.918 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.918 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.918 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.918 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.919 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.919 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.919 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.919 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.919 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.919 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.919 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.919 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.919 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.919 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.919 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.919 INFO:teuthology.orchestra.run.smithi012.stdout: "min_size": { 2024-02-18T11:53:23.919 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "min_size", 2024-02-18T11:53:23.919 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.919 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.920 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.920 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.920 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.920 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.920 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.920 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "default min_size for any created local pool", 2024-02-18T11:53:23.920 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-02-18T11:53:23.920 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.920 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.920 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.920 INFO:teuthology.orchestra.run.smithi012.stdout: "num_rep": { 2024-02-18T11:53:23.920 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "num_rep", 2024-02-18T11:53:23.920 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.920 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.921 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.921 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "3", 2024-02-18T11:53:23.921 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.921 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.921 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.921 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "default replica count for any created local pool", 2024-02-18T11:53:23.921 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.921 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.921 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.921 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.921 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num": { 2024-02-18T11:53:23.921 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pg_num", 2024-02-18T11:53:23.921 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.921 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.921 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.922 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "128", 2024-02-18T11:53:23.922 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.922 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.922 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.922 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "default pg_num for any created local pool", 2024-02-18T11:53:23.922 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.922 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.922 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.922 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.922 INFO:teuthology.orchestra.run.smithi012.stdout: "prefix": { 2024-02-18T11:53:23.922 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "prefix", 2024-02-18T11:53:23.922 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.922 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.922 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.923 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.923 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.923 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.923 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.923 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "name prefix for any created local pool", 2024-02-18T11:53:23.923 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.923 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.923 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.923 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.923 INFO:teuthology.orchestra.run.smithi012.stdout: "subtree": { 2024-02-18T11:53:23.923 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "subtree", 2024-02-18T11:53:23.923 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.923 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.923 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.923 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "rack", 2024-02-18T11:53:23.923 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.924 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.924 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.924 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "CRUSH level for which to create a local pool", 2024-02-18T11:53:23.924 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-02-18T11:53:23.924 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.924 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.924 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.924 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.924 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.924 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.924 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mds_autoscaler", 2024-02-18T11:53:23.924 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.924 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.924 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.925 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.925 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.925 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.925 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.925 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.925 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.925 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.925 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.925 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.925 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.925 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.925 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.925 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.925 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.925 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.925 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.926 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.926 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.926 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.926 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.926 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.926 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.926 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.926 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.926 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.926 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.926 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.926 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.926 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.926 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.926 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.926 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.927 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.927 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.927 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.927 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.927 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.927 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.927 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.927 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.927 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.927 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.927 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.927 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.927 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.927 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.927 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.928 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.928 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.928 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.928 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.928 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.928 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.928 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.928 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.928 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.928 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.928 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.928 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.928 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.928 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.928 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.929 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.929 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.929 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.929 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.929 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.929 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.929 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.929 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.929 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.929 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.929 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.929 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mirroring", 2024-02-18T11:53:23.929 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.929 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.929 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.930 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.930 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.930 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.930 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.930 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.930 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.930 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.930 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.930 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.930 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.930 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.930 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.930 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.930 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.930 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.931 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.931 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.931 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.931 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.931 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.931 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.931 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.931 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.931 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.931 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.931 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.931 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.931 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.931 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.932 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.932 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.932 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.932 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.932 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.932 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.932 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.932 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.932 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.932 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.932 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.932 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.932 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.932 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.933 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.933 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.933 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.933 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.933 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.933 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.933 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.933 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.933 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.933 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.933 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.933 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.933 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.933 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.933 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.934 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.934 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.934 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.934 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.934 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.934 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.934 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.934 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.934 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.934 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.934 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.934 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.934 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.934 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.935 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.935 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "nfs", 2024-02-18T11:53:23.935 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.935 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.935 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.935 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.935 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.935 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.935 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.935 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.935 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.935 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.935 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.935 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.935 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.936 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.936 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.936 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.936 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.936 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.936 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.936 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.936 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.936 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.936 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.936 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.936 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.936 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.936 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.937 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.937 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.937 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.937 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.937 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.937 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.937 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.937 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.937 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.937 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.937 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.937 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.937 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.937 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.937 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.938 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.938 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.938 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.938 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.938 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.938 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.938 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.938 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.938 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.938 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.938 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.938 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.938 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.938 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.938 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.939 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.939 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.939 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.939 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.939 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.939 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.939 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.939 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.939 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.939 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.939 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.939 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.939 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.939 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.939 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.940 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.940 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.940 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.940 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.940 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "orchestrator", 2024-02-18T11:53:23.940 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.940 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.940 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.940 INFO:teuthology.orchestra.run.smithi012.stdout: "fail_fs": { 2024-02-18T11:53:23.940 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "fail_fs", 2024-02-18T11:53:23.940 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.940 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.940 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.940 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.941 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.941 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.941 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.941 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-02-18T11:53:23.941 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.941 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.941 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.941 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.941 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.941 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.941 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.941 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.941 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.941 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.941 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.942 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.942 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.942 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.942 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.942 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.942 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.942 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.942 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.942 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.942 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.942 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.942 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.942 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.942 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.942 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.942 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.943 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.943 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.943 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.943 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.943 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.943 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.943 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.943 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.943 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.943 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.943 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.943 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.943 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.943 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.943 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.944 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.944 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.944 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.944 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.944 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.944 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.944 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.944 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.944 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.944 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.944 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.944 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.944 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.944 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.945 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.945 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.945 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.945 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.945 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.945 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.945 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.945 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.945 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.945 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.945 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.945 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.945 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.945 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.945 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.946 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.946 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.946 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.946 INFO:teuthology.orchestra.run.smithi012.stdout: "orchestrator": { 2024-02-18T11:53:23.946 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "orchestrator", 2024-02-18T11:53:23.946 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.946 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.946 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.946 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.946 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.946 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.946 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.946 INFO:teuthology.orchestra.run.smithi012.stdout: "cephadm", 2024-02-18T11:53:23.946 INFO:teuthology.orchestra.run.smithi012.stdout: "rook", 2024-02-18T11:53:23.946 INFO:teuthology.orchestra.run.smithi012.stdout: "test_orchestrator" 2024-02-18T11:53:23.947 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.947 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Orchestrator backend", 2024-02-18T11:53:23.947 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.947 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.947 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.947 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.947 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.947 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.947 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.947 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "osd_perf_query", 2024-02-18T11:53:23.947 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.947 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.947 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.947 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.947 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.948 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.948 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.948 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.948 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.948 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.948 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.948 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.948 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.948 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.948 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.948 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.948 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.948 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.949 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.949 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.949 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.949 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.949 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.949 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.949 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.949 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.949 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.949 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.949 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.949 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.949 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.949 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.949 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.949 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.950 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.950 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.950 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.950 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.950 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.950 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.950 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.950 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.950 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.950 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.950 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.950 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.950 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.950 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.950 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.951 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.951 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.951 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.951 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.951 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.951 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.951 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.951 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.951 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.951 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.951 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.951 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.951 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.951 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.952 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.952 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.952 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.952 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.952 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.952 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.952 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.952 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.952 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.952 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.952 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.952 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.952 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.952 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "osd_support", 2024-02-18T11:53:23.953 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.953 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.953 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.953 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.953 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.953 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.953 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.953 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.953 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.953 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.953 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.953 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.953 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.953 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.953 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.954 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.954 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.954 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.954 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.954 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.954 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.954 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.954 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.954 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.954 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.954 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.954 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.954 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.955 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.955 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.955 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.955 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.955 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.955 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.955 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.955 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.955 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.955 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.955 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.955 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.955 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.955 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.956 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.956 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.956 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.956 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.956 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.956 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.956 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.956 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.956 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.956 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.956 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.956 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.956 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.956 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.957 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.957 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.957 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.957 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.957 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.957 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.957 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.957 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.957 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.957 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.957 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.957 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.957 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.957 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.958 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.958 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.958 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.958 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.958 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.958 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.958 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pg_autoscaler", 2024-02-18T11:53:23.958 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.958 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.958 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.958 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.958 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.958 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.958 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.958 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.959 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.959 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.959 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.959 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.959 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.959 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.959 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.959 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.959 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.959 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.959 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.959 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.959 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.959 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.959 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.960 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.960 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.960 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.960 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.960 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.960 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.960 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.960 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.960 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.960 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.960 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.960 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.960 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.960 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.961 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.961 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.961 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.961 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.961 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.961 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.961 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.961 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.961 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.961 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.961 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.961 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.961 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.961 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.961 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.962 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.962 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.962 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.962 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.962 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.962 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.962 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.962 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.962 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.962 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.962 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.962 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.962 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.962 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.962 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.963 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.963 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.963 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.963 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.963 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.963 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.963 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-02-18T11:53:23.963 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-02-18T11:53:23.963 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-02-18T11:53:23.963 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.963 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.963 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-02-18T11:53:23.963 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.963 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.963 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.964 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.964 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.964 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.964 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.964 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.964 INFO:teuthology.orchestra.run.smithi012.stdout: "threshold": { 2024-02-18T11:53:23.964 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "threshold", 2024-02-18T11:53:23.964 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-02-18T11:53:23.964 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.964 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.964 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "3.0", 2024-02-18T11:53:23.964 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1.0", 2024-02-18T11:53:23.964 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.964 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.965 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "scaling threshold", 2024-02-18T11:53:23.965 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-02-18T11:53:23.965 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.965 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.965 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.965 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.965 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.965 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.965 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "progress", 2024-02-18T11:53:23.965 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.965 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.965 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.965 INFO:teuthology.orchestra.run.smithi012.stdout: "allow_pg_recovery_event": { 2024-02-18T11:53:23.965 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "allow_pg_recovery_event", 2024-02-18T11:53:23.965 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.966 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.966 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.966 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.966 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.966 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.966 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.966 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "allow the module to show pg recovery progress", 2024-02-18T11:53:23.966 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.966 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.966 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.966 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.966 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": { 2024-02-18T11:53:23.966 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enabled", 2024-02-18T11:53:23.966 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.967 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.967 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.967 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.967 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.967 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.967 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.967 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.967 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.967 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.967 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.967 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.967 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.967 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.967 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.967 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.967 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.968 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.968 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.968 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.968 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.968 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.968 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.968 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.968 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.968 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.968 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.968 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.968 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.968 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.968 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.969 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.969 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.969 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.969 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.969 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.969 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.969 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.969 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.969 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.969 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.969 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.969 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.969 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.969 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.969 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.969 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.970 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.970 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.970 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.970 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.970 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.970 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.970 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.970 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.970 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.970 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.970 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.970 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.970 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.970 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.970 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.971 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.971 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.971 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.971 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.971 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.971 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.971 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.971 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.971 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.971 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.971 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.971 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.971 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.971 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.971 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.972 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.972 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.972 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.972 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.972 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.972 INFO:teuthology.orchestra.run.smithi012.stdout: "max_completed_events": { 2024-02-18T11:53:23.972 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_completed_events", 2024-02-18T11:53:23.972 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.972 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.972 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.972 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "50", 2024-02-18T11:53:23.972 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.972 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.972 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.972 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "number of past completed events to remember", 2024-02-18T11:53:23.973 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.973 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.973 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.973 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.973 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-02-18T11:53:23.973 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-02-18T11:53:23.973 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-02-18T11:53:23.973 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.973 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.973 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-02-18T11:53:23.973 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.973 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.973 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.973 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how long the module is going to sleep", 2024-02-18T11:53:23.973 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.974 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.974 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.974 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.974 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.974 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.974 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.974 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "prometheus", 2024-02-18T11:53:23.974 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.974 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.974 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.974 INFO:teuthology.orchestra.run.smithi012.stdout: "cache": { 2024-02-18T11:53:23.974 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cache", 2024-02-18T11:53:23.974 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.974 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.974 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.975 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.975 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.975 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.975 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.975 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.975 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.975 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.975 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.975 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.975 INFO:teuthology.orchestra.run.smithi012.stdout: "exclude_perf_counters": { 2024-02-18T11:53:23.975 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "exclude_perf_counters", 2024-02-18T11:53:23.975 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.975 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.975 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.975 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:23.976 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.976 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.976 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.976 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-02-18T11:53:23.976 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-02-18T11:53:23.976 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.976 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.976 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.976 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.976 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.976 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.976 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.976 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.976 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.976 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.977 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.977 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.977 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.977 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.977 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.977 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.977 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.977 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.977 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.977 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.977 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.977 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.977 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.977 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.978 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.978 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.978 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.978 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.978 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.978 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.978 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.978 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.978 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.978 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.978 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.978 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.978 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.978 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.978 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.978 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.979 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.979 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.979 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.979 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.979 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.979 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.979 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.979 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.979 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.979 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.979 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.979 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.979 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.979 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.980 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.980 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.980 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.980 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.980 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.980 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.980 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.980 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.980 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.980 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.980 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.984 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.984 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.984 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.984 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.984 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.984 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.984 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.984 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.984 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_stats_pools": { 2024-02-18T11:53:23.984 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rbd_stats_pools", 2024-02-18T11:53:23.984 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.984 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.984 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.985 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.985 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.985 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.985 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.985 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.985 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.985 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.985 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.985 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.985 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_stats_pools_refresh_interval": { 2024-02-18T11:53:23.985 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-02-18T11:53:23.985 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.985 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.985 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.985 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "300", 2024-02-18T11:53:23.985 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.986 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.986 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.986 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.986 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.986 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.986 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.986 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.986 INFO:teuthology.orchestra.run.smithi012.stdout: "scrape_interval": { 2024-02-18T11:53:23.986 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "scrape_interval", 2024-02-18T11:53:23.986 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-02-18T11:53:23.986 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.986 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.986 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "15.0", 2024-02-18T11:53:23.986 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.986 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.987 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.987 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.987 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.987 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.987 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.987 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.987 INFO:teuthology.orchestra.run.smithi012.stdout: "server_addr": { 2024-02-18T11:53:23.987 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_addr", 2024-02-18T11:53:23.987 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.987 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.987 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.987 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "::", 2024-02-18T11:53:23.987 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.987 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.987 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.988 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-02-18T11:53:23.988 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.988 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.988 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.988 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.988 INFO:teuthology.orchestra.run.smithi012.stdout: "server_port": { 2024-02-18T11:53:23.988 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_port", 2024-02-18T11:53:23.988 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.988 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.988 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.988 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "9283", 2024-02-18T11:53:23.988 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.988 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.988 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.988 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-02-18T11:53:23.989 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.989 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.989 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.989 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.989 INFO:teuthology.orchestra.run.smithi012.stdout: "stale_cache_strategy": { 2024-02-18T11:53:23.989 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "stale_cache_strategy", 2024-02-18T11:53:23.989 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.989 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.989 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.989 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "log", 2024-02-18T11:53:23.989 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.989 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.989 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.989 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.989 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.990 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.990 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.990 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.990 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_behaviour": { 2024-02-18T11:53:23.990 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_behaviour", 2024-02-18T11:53:23.990 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.990 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.990 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.990 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "default", 2024-02-18T11:53:23.990 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.990 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.990 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.990 INFO:teuthology.orchestra.run.smithi012.stdout: "default", 2024-02-18T11:53:23.990 INFO:teuthology.orchestra.run.smithi012.stdout: "error" 2024-02-18T11:53:23.990 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.990 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.991 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.991 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.991 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.991 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.991 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_error_status_code": { 2024-02-18T11:53:23.991 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_error_status_code", 2024-02-18T11:53:23.991 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.991 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.991 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.991 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "500", 2024-02-18T11:53:23.991 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "400", 2024-02-18T11:53:23.991 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "599", 2024-02-18T11:53:23.991 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.991 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.991 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.992 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.992 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.992 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.992 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:23.992 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.992 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:23.992 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rbd_support", 2024-02-18T11:53:23.992 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:23.992 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:23.992 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:23.992 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:23.992 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:23.992 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.992 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.992 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.993 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.993 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.993 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.993 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.993 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.993 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.993 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.993 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.993 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.993 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.993 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.993 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.993 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.993 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.993 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.993 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.994 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:23.994 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:23.994 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.994 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.994 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.994 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.994 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.994 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.994 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.994 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.994 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.994 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.994 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.994 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.994 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:23.995 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:23.995 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.995 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.995 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.995 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:23.995 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.995 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.995 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:23.995 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:23.995 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:23.995 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:23.995 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:23.995 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:23.995 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:23.995 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:23.995 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.996 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.996 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.996 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.996 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.996 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:23.996 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:23.996 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:23.996 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.996 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:23.996 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:23.996 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.996 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.996 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.996 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.996 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.997 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.997 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.997 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.997 INFO:teuthology.orchestra.run.smithi012.stdout: "max_concurrent_snap_create": { 2024-02-18T11:53:23.997 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_concurrent_snap_create", 2024-02-18T11:53:23.997 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:23.997 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.997 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.997 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-02-18T11:53:23.997 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.997 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.997 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.997 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.997 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.997 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.997 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.998 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.998 INFO:teuthology.orchestra.run.smithi012.stdout: "mirror_snapshot_schedule": { 2024-02-18T11:53:23.998 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mirror_snapshot_schedule", 2024-02-18T11:53:23.998 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.998 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.998 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.998 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.998 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.998 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.998 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.998 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.998 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.998 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.998 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.999 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:23.999 INFO:teuthology.orchestra.run.smithi012.stdout: "trash_purge_schedule": { 2024-02-18T11:53:23.999 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "trash_purge_schedule", 2024-02-18T11:53:23.999 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:23.999 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:23.999 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:23.999 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:23.999 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:23.999 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:23.999 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:23.999 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:23.999 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:23.999 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:23.999 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:23.999 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:24.000 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:24.000 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.000 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:24.000 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "restful", 2024-02-18T11:53:24.000 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:24.000 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:24.000 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:24.000 INFO:teuthology.orchestra.run.smithi012.stdout: "enable_auth": { 2024-02-18T11:53:24.000 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enable_auth", 2024-02-18T11:53:24.000 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:24.000 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.000 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:24.000 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:24.000 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.000 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.001 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:24.001 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.001 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.001 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.001 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.001 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.001 INFO:teuthology.orchestra.run.smithi012.stdout: "key_file": { 2024-02-18T11:53:24.001 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "key_file", 2024-02-18T11:53:24.001 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:24.001 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.001 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:24.001 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:24.001 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.001 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.001 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:24.002 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.002 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.002 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.002 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.002 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.002 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:24.002 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:24.002 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:24.002 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.002 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:24.002 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:24.002 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.002 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.002 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:24.002 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:24.003 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:24.003 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:24.003 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:24.003 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:24.003 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:24.003 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:24.003 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.003 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.003 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.003 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.003 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.003 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:24.003 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:24.003 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:24.003 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.004 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:24.004 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:24.004 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.004 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.004 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:24.004 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.004 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.004 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.004 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.004 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.004 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:24.004 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:24.004 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:24.004 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.005 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:24.005 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:24.005 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.005 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.005 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:24.005 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:24.005 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:24.005 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:24.005 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:24.005 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:24.005 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:24.005 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:24.005 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.005 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.005 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.005 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.006 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.006 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:24.006 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:24.006 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:24.006 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.006 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:24.006 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:24.006 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.006 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.006 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:24.006 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.006 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.006 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.006 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.006 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.007 INFO:teuthology.orchestra.run.smithi012.stdout: "server_addr": { 2024-02-18T11:53:24.007 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_addr", 2024-02-18T11:53:24.007 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:24.007 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.007 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:24.007 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:24.007 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.007 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.007 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:24.007 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.007 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.007 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.007 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.007 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.007 INFO:teuthology.orchestra.run.smithi012.stdout: "server_port": { 2024-02-18T11:53:24.007 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_port", 2024-02-18T11:53:24.008 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:24.008 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.008 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:24.008 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:24.008 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.008 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.008 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:24.008 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.008 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.008 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.008 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.008 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:24.008 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:24.008 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.008 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:24.009 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rgw", 2024-02-18T11:53:24.009 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:24.009 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:24.009 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:24.009 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:24.009 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:24.009 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:24.009 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.009 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:24.009 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:24.009 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.009 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.009 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:24.009 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:24.009 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:24.010 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:24.010 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:24.010 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:24.010 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:24.010 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:24.010 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.010 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.010 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.010 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.010 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.010 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:24.010 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:24.010 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:24.010 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.010 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:24.011 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:24.011 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.011 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.011 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:24.011 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.011 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.011 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.011 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.011 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.011 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:24.011 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:24.011 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:24.011 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.011 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:24.011 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:24.012 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.012 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.012 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:24.012 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:24.012 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:24.012 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:24.012 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:24.012 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:24.012 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:24.012 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:24.012 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.012 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.012 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.012 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.012 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.013 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:24.013 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:24.013 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:24.013 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.013 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:24.013 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:24.013 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.013 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.013 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:24.013 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.013 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.013 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.013 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.013 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:24.013 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:24.013 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.014 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:24.014 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rook", 2024-02-18T11:53:24.014 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:24.014 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:24.014 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:24.014 INFO:teuthology.orchestra.run.smithi012.stdout: "drive_group_interval": { 2024-02-18T11:53:24.014 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "drive_group_interval", 2024-02-18T11:53:24.014 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-02-18T11:53:24.014 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.014 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:24.014 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "300.0", 2024-02-18T11:53:24.014 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.014 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.014 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:24.015 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2024-02-18T11:53:24.015 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.015 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.015 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.015 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.015 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:24.015 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:24.015 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:24.015 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.015 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:24.015 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:24.015 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.015 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.015 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:24.015 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:24.016 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:24.016 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:24.016 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:24.016 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:24.016 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:24.016 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:24.016 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.016 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.016 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.016 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.016 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.016 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:24.016 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:24.016 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:24.016 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.017 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:24.017 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:24.017 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.017 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.017 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:24.017 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.017 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.017 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.017 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.017 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.017 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:24.017 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:24.017 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:24.017 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.017 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:24.018 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:24.018 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.018 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.018 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:24.018 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:24.018 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:24.018 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:24.018 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:24.018 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:24.018 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:24.018 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:24.018 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.018 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.018 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.018 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.019 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.019 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:24.019 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:24.019 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:24.019 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.019 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:24.019 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:24.019 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.019 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.019 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:24.019 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.019 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.019 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.019 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.019 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.020 INFO:teuthology.orchestra.run.smithi012.stdout: "storage_class": { 2024-02-18T11:53:24.020 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "storage_class", 2024-02-18T11:53:24.020 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:24.020 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.020 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:24.020 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "local", 2024-02-18T11:53:24.020 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.020 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.020 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:24.020 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-02-18T11:53:24.020 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.020 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.020 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.020 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:24.020 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:24.021 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.021 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:24.021 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "selftest", 2024-02-18T11:53:24.021 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:24.021 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:24.021 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:24.021 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:24.021 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:24.021 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:24.021 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.021 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:24.021 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:24.021 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.021 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.021 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:24.022 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:24.022 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:24.022 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:24.022 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:24.022 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:24.022 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:24.022 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:24.022 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.022 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.022 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.022 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.022 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.022 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:24.022 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:24.022 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:24.023 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.023 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:24.023 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:24.023 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.023 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.023 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:24.023 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.023 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.023 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.023 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.023 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.023 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:24.023 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:24.023 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:24.023 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.024 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:24.024 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:24.024 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.024 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.024 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:24.024 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:24.024 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:24.024 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:24.024 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:24.024 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:24.024 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:24.024 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:24.024 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.024 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.025 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.025 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.025 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.025 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:24.025 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:24.025 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:24.025 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.025 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:24.025 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:24.025 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.025 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.025 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:24.025 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.025 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.025 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.026 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.026 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.026 INFO:teuthology.orchestra.run.smithi012.stdout: "roption1": { 2024-02-18T11:53:24.026 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "roption1", 2024-02-18T11:53:24.026 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:24.026 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.026 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:24.026 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:24.026 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.026 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.026 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:24.026 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.026 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.026 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.026 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.026 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.027 INFO:teuthology.orchestra.run.smithi012.stdout: "roption2": { 2024-02-18T11:53:24.027 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "roption2", 2024-02-18T11:53:24.027 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:24.027 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.027 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:24.027 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "xyz", 2024-02-18T11:53:24.027 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.027 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.027 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:24.027 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.027 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.027 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.027 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.027 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.027 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption1": { 2024-02-18T11:53:24.028 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption1", 2024-02-18T11:53:24.028 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:24.028 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.028 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:24.028 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:24.028 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.028 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.028 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:24.028 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.028 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.028 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.028 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.028 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.028 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption2": { 2024-02-18T11:53:24.028 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption2", 2024-02-18T11:53:24.028 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:24.029 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.029 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:24.029 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:24.029 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.029 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.029 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:24.029 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.029 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.029 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.029 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.029 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.029 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption3": { 2024-02-18T11:53:24.029 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption3", 2024-02-18T11:53:24.029 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-02-18T11:53:24.029 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.030 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:24.030 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:24.030 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.030 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.030 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:24.030 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.030 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.030 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.030 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.030 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.030 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption4": { 2024-02-18T11:53:24.030 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption4", 2024-02-18T11:53:24.030 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:24.030 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.031 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:24.031 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:24.031 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.031 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.031 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:24.031 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.031 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.031 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.031 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.031 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.031 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption5": { 2024-02-18T11:53:24.031 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption5", 2024-02-18T11:53:24.031 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:24.031 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.031 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:24.032 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:24.032 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.032 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.032 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:24.032 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.032 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.032 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.032 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.032 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.032 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption6": { 2024-02-18T11:53:24.032 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption6", 2024-02-18T11:53:24.032 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:24.032 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.032 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:24.032 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:24.033 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.033 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.033 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:24.033 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.033 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.033 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.033 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.033 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.033 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption7": { 2024-02-18T11:53:24.033 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption7", 2024-02-18T11:53:24.033 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:24.033 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.033 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:24.033 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:24.033 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-02-18T11:53:24.033 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "42", 2024-02-18T11:53:24.034 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:24.034 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.034 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.034 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.034 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.034 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.034 INFO:teuthology.orchestra.run.smithi012.stdout: "testkey": { 2024-02-18T11:53:24.034 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "testkey", 2024-02-18T11:53:24.034 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:24.034 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.034 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:24.034 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:24.034 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.034 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.034 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:24.035 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.035 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.035 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.035 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.035 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.035 INFO:teuthology.orchestra.run.smithi012.stdout: "testlkey": { 2024-02-18T11:53:24.035 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "testlkey", 2024-02-18T11:53:24.035 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:24.035 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.035 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:24.035 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:24.035 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.035 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.035 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:24.035 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.036 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.036 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.036 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.036 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.036 INFO:teuthology.orchestra.run.smithi012.stdout: "testnewline": { 2024-02-18T11:53:24.036 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "testnewline", 2024-02-18T11:53:24.036 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:24.036 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.036 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:24.036 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:24.036 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.036 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.036 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:24.036 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.037 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.037 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.037 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.037 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:24.037 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:24.037 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.037 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:24.037 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "snap_schedule", 2024-02-18T11:53:24.037 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:24.037 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:24.037 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:24.037 INFO:teuthology.orchestra.run.smithi012.stdout: "allow_m_granularity": { 2024-02-18T11:53:24.037 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "allow_m_granularity", 2024-02-18T11:53:24.037 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:24.037 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.038 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:24.038 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:24.038 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.038 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.038 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:24.038 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "allow minute scheduled snapshots", 2024-02-18T11:53:24.038 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.038 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.038 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.038 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.038 INFO:teuthology.orchestra.run.smithi012.stdout: "dump_on_update": { 2024-02-18T11:53:24.038 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "dump_on_update", 2024-02-18T11:53:24.038 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:24.038 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.038 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:24.038 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:24.039 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.039 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.039 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:24.039 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "dump database to debug log on update", 2024-02-18T11:53:24.039 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.039 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.039 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.039 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.039 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:24.039 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:24.039 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:24.039 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.039 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:24.039 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:24.039 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.040 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.040 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:24.040 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:24.040 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:24.040 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:24.040 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:24.040 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:24.040 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:24.040 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:24.040 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.040 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.040 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.040 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.040 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.040 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:24.041 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:24.041 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:24.041 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.041 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:24.041 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:24.041 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.041 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.041 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:24.041 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.041 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.041 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.041 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.041 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.041 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:24.041 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:24.041 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:24.042 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.042 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:24.042 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:24.042 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.042 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.042 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:24.042 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:24.042 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:24.042 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:24.042 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:24.042 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:24.042 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:24.042 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:24.042 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.042 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.043 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.043 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.043 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.043 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:24.043 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:24.043 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:24.043 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.043 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:24.043 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:24.043 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.043 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.043 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:24.043 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.043 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.043 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.043 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.044 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:24.044 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:24.044 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.044 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:24.044 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "stats", 2024-02-18T11:53:24.044 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:24.044 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:24.044 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:24.044 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:24.044 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:24.044 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:24.044 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.044 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:24.044 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:24.045 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.045 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.045 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:24.045 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:24.045 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:24.045 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:24.045 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:24.045 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:24.045 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:24.045 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:24.045 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.045 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.045 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.045 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.045 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.045 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:24.046 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:24.046 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:24.046 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.046 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:24.046 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:24.046 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.046 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.046 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:24.046 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.046 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.046 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.046 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.046 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.046 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:24.047 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:24.047 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:24.047 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.047 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:24.047 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:24.047 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.047 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.047 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:24.047 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:24.047 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:24.047 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:24.047 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:24.047 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:24.047 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:24.047 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:24.048 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.048 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.048 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.048 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.048 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.048 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:24.048 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:24.048 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:24.048 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.048 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:24.048 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:24.048 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.048 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.048 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:24.048 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.049 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.049 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.049 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.049 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:24.049 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:24.049 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.049 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:24.049 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "status", 2024-02-18T11:53:24.049 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:24.049 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:24.049 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:24.049 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:24.049 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:24.049 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:24.049 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.049 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:24.050 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:24.050 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.050 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.050 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:24.050 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:24.050 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:24.050 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:24.050 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:24.050 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:24.050 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:24.050 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:24.050 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.050 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.050 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.051 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.051 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.051 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:24.051 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:24.051 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:24.051 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.051 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:24.051 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:24.051 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.051 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.051 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:24.051 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.051 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.051 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.051 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.052 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.052 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:24.052 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:24.052 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:24.052 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.052 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:24.052 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:24.052 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.052 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.052 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:24.052 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:24.052 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:24.052 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:24.052 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:24.052 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:24.053 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:24.053 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:24.053 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.053 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.053 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.053 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.053 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.053 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:24.053 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:24.053 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:24.053 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.053 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:24.053 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:24.053 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.053 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.054 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:24.054 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.054 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.054 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.054 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.054 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:24.054 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:24.054 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.054 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:24.054 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "telegraf", 2024-02-18T11:53:24.054 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:24.054 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:24.054 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:24.054 INFO:teuthology.orchestra.run.smithi012.stdout: "address": { 2024-02-18T11:53:24.054 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "address", 2024-02-18T11:53:24.054 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:24.055 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.055 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:24.055 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-02-18T11:53:24.055 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.055 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.055 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:24.055 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.055 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.055 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.055 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.055 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.055 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-02-18T11:53:24.055 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-02-18T11:53:24.055 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-02-18T11:53:24.055 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.056 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:24.056 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "15", 2024-02-18T11:53:24.056 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.056 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.056 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:24.056 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.056 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.056 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.056 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.056 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.056 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:24.056 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:24.056 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:24.056 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.056 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:24.056 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:24.057 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.057 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.057 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:24.057 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:24.057 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:24.057 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:24.057 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:24.057 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:24.057 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:24.057 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:24.057 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.057 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.057 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.057 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.057 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.058 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:24.058 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:24.058 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:24.058 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.058 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:24.058 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:24.058 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.058 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.058 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:24.058 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.058 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.058 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.058 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.058 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.058 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:24.058 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:24.059 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:24.059 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.059 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:24.059 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:24.059 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.059 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.059 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:24.059 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:24.059 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:24.059 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:24.059 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:24.059 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:24.059 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:24.059 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:24.059 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.059 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.060 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.060 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.060 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.060 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:24.060 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:24.060 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:24.060 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.060 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:24.060 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:24.060 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.060 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.060 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:24.060 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.060 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.060 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.061 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.061 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:24.061 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:24.061 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.061 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:24.061 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "telemetry", 2024-02-18T11:53:24.061 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:24.061 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:24.061 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:24.061 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_basic": { 2024-02-18T11:53:24.061 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_basic", 2024-02-18T11:53:24.061 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:24.061 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.061 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:24.061 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:24.062 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.062 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.062 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:24.062 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share basic cluster information (size, version)", 2024-02-18T11:53:24.062 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.062 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.062 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.062 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.062 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_crash": { 2024-02-18T11:53:24.062 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_crash", 2024-02-18T11:53:24.062 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:24.062 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.062 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:24.062 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:24.062 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.063 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.063 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:24.063 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-02-18T11:53:24.063 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.063 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.063 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.063 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.063 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_device": { 2024-02-18T11:53:24.063 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_device", 2024-02-18T11:53:24.063 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:24.063 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.063 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:24.063 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:24.063 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.063 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.064 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:24.064 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-02-18T11:53:24.064 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.064 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.064 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.064 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.064 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_ident": { 2024-02-18T11:53:24.064 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_ident", 2024-02-18T11:53:24.064 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:24.064 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.064 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:24.064 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:24.064 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.064 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.065 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:24.065 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-02-18T11:53:24.065 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.065 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.065 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.065 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.065 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_perf": { 2024-02-18T11:53:24.065 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_perf", 2024-02-18T11:53:24.065 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:24.065 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.065 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:24.065 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:24.065 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.065 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.065 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:24.066 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share various performance metrics of a cluster", 2024-02-18T11:53:24.066 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.066 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.066 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.066 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.066 INFO:teuthology.orchestra.run.smithi012.stdout: "contact": { 2024-02-18T11:53:24.066 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "contact", 2024-02-18T11:53:24.066 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:24.066 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.066 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:24.066 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:24.066 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.066 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.066 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:24.066 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.067 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.067 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.067 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.067 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.067 INFO:teuthology.orchestra.run.smithi012.stdout: "description": { 2024-02-18T11:53:24.067 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "description", 2024-02-18T11:53:24.067 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:24.067 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.067 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:24.067 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:24.067 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.067 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.067 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:24.067 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.067 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.068 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.068 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.068 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.068 INFO:teuthology.orchestra.run.smithi012.stdout: "device_url": { 2024-02-18T11:53:24.068 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "device_url", 2024-02-18T11:53:24.068 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:24.068 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.068 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:24.068 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-02-18T11:53:24.068 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.068 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.068 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:24.068 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.068 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.069 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.069 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.069 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.069 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": { 2024-02-18T11:53:24.069 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enabled", 2024-02-18T11:53:24.069 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:24.069 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.069 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:24.069 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:24.069 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.069 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.069 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:24.069 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.069 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.069 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.070 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.070 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.070 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-02-18T11:53:24.070 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-02-18T11:53:24.070 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:24.070 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.070 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:24.070 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "24", 2024-02-18T11:53:24.070 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "8", 2024-02-18T11:53:24.070 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.070 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:24.070 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.070 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.070 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.070 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.070 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.071 INFO:teuthology.orchestra.run.smithi012.stdout: "last_opt_revision": { 2024-02-18T11:53:24.071 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "last_opt_revision", 2024-02-18T11:53:24.071 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:24.071 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.071 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:24.071 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "1", 2024-02-18T11:53:24.071 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.071 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.071 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:24.071 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.071 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.071 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.071 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.071 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.071 INFO:teuthology.orchestra.run.smithi012.stdout: "leaderboard": { 2024-02-18T11:53:24.071 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "leaderboard", 2024-02-18T11:53:24.072 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:24.072 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.072 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:24.072 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:24.072 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.072 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.072 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:24.072 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.072 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.072 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.072 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.072 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.072 INFO:teuthology.orchestra.run.smithi012.stdout: "leaderboard_description": { 2024-02-18T11:53:24.072 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "leaderboard_description", 2024-02-18T11:53:24.072 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:24.073 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.073 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:24.073 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:24.073 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.073 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.073 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:24.073 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.073 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.073 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.073 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.073 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.073 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:24.073 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:24.073 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:24.073 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.074 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:24.074 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:24.074 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.074 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.074 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:24.074 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:24.074 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:24.074 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:24.074 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:24.074 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:24.074 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:24.074 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:24.074 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.074 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.074 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.075 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.075 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.075 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:24.075 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:24.075 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:24.075 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.075 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:24.075 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:24.075 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.075 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.075 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:24.075 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.075 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.075 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.075 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.075 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.076 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:24.076 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:24.076 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:24.076 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.076 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:24.076 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:24.076 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.076 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.076 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:24.076 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:24.076 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:24.076 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:24.076 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:24.076 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:24.077 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:24.077 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:24.077 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.077 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.077 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.077 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.077 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.077 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:24.077 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:24.077 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:24.077 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.077 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:24.077 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:24.077 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.077 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.078 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:24.078 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.078 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.078 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.078 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.078 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.078 INFO:teuthology.orchestra.run.smithi012.stdout: "organization": { 2024-02-18T11:53:24.078 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "organization", 2024-02-18T11:53:24.078 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:24.078 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.078 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:24.078 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:24.078 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.078 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.078 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:24.079 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.079 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.079 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.079 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.079 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.079 INFO:teuthology.orchestra.run.smithi012.stdout: "proxy": { 2024-02-18T11:53:24.079 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "proxy", 2024-02-18T11:53:24.079 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:24.079 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.079 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:24.079 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:24.079 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.079 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.079 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:24.079 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.080 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.080 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.080 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.080 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.080 INFO:teuthology.orchestra.run.smithi012.stdout: "url": { 2024-02-18T11:53:24.080 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "url", 2024-02-18T11:53:24.080 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:24.080 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.080 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:24.080 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-02-18T11:53:24.080 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.080 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.080 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:24.080 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.081 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.081 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.081 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.081 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:24.081 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:24.081 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.081 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:24.081 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "test_orchestrator", 2024-02-18T11:53:24.081 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:24.081 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:24.081 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:24.081 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:24.081 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:24.081 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:24.081 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.082 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:24.082 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:24.082 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.082 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.082 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:24.082 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:24.082 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:24.082 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:24.082 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:24.082 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:24.082 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:24.082 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:24.082 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.082 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.082 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.083 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.083 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.083 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:24.083 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:24.083 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:24.083 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.083 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:24.083 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:24.083 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.083 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.083 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:24.083 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.083 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.083 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.083 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.084 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.084 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:24.084 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:24.084 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:24.084 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.084 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:24.084 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:24.084 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.084 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.084 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:24.084 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:24.084 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:24.084 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:24.084 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:24.085 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:24.085 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:24.085 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:24.085 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.085 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.085 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.085 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.085 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.085 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:24.085 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:24.085 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:24.085 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.085 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:24.085 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:24.085 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.086 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.086 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:24.086 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.086 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.086 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.086 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.086 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:24.086 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:24.086 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.086 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:24.086 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "volumes", 2024-02-18T11:53:24.086 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:24.086 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:24.086 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:24.086 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:24.087 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:24.087 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:24.087 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.087 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:24.087 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:24.087 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.087 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.087 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:24.087 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:24.087 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:24.087 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:24.087 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:24.087 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:24.087 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:24.087 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:24.087 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.088 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.088 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.088 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.088 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.088 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:24.088 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:24.088 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:24.088 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.088 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:24.088 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:24.088 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.088 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.088 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:24.088 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.088 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.089 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.089 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.089 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.089 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:24.089 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:24.089 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:24.089 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.089 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:24.089 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:24.089 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.089 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.089 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:24.089 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:24.089 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:24.089 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:24.089 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:24.090 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:24.090 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:24.090 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:24.090 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.090 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.090 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.090 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.090 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.090 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:24.090 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:24.090 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:24.090 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.090 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:24.090 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:24.090 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.090 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.091 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:24.091 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.091 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.091 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.091 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.091 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.091 INFO:teuthology.orchestra.run.smithi012.stdout: "max_concurrent_clones": { 2024-02-18T11:53:24.091 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_concurrent_clones", 2024-02-18T11:53:24.091 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:24.091 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.091 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:24.091 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "4", 2024-02-18T11:53:24.091 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.091 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.091 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:24.092 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Number of asynchronous cloner threads", 2024-02-18T11:53:24.092 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.092 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.092 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.092 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.092 INFO:teuthology.orchestra.run.smithi012.stdout: "periodic_async_work": { 2024-02-18T11:53:24.092 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "periodic_async_work", 2024-02-18T11:53:24.092 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:24.092 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.092 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:24.092 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:24.092 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.092 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.092 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:24.092 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Periodically check for async work", 2024-02-18T11:53:24.093 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.093 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.093 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.093 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.093 INFO:teuthology.orchestra.run.smithi012.stdout: "snapshot_clone_delay": { 2024-02-18T11:53:24.093 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "snapshot_clone_delay", 2024-02-18T11:53:24.093 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:24.093 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.093 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:24.093 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-02-18T11:53:24.093 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.093 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.093 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:24.093 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-02-18T11:53:24.094 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.094 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.094 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.094 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.094 INFO:teuthology.orchestra.run.smithi012.stdout: "snapshot_clone_no_wait": { 2024-02-18T11:53:24.094 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "snapshot_clone_no_wait", 2024-02-18T11:53:24.094 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:24.094 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.094 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:24.094 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-02-18T11:53:24.094 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.094 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.094 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:24.094 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-02-18T11:53:24.094 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.095 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.095 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.095 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:24.095 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:24.095 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.095 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:24.095 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix", 2024-02-18T11:53:24.095 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-02-18T11:53:24.095 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-02-18T11:53:24.095 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-02-18T11:53:24.095 INFO:teuthology.orchestra.run.smithi012.stdout: "discovery_interval": { 2024-02-18T11:53:24.095 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "discovery_interval", 2024-02-18T11:53:24.095 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-02-18T11:53:24.095 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.095 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:24.096 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "100", 2024-02-18T11:53:24.096 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.096 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.096 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:24.096 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.096 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.096 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.096 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.096 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.096 INFO:teuthology.orchestra.run.smithi012.stdout: "identifier": { 2024-02-18T11:53:24.096 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "identifier", 2024-02-18T11:53:24.096 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:24.096 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.096 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:24.097 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:24.097 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.097 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.097 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:24.097 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.097 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.097 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.097 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.097 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.097 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-02-18T11:53:24.097 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-02-18T11:53:24.097 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-02-18T11:53:24.097 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.097 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:24.097 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-02-18T11:53:24.098 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.098 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.098 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:24.098 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.098 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.098 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.098 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.098 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.098 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-02-18T11:53:24.098 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-02-18T11:53:24.098 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:24.098 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.098 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:24.098 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:24.098 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.098 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.099 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:24.099 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:24.099 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:24.099 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:24.099 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:24.099 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:24.099 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:24.099 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:24.099 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.099 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.099 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.099 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.099 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.099 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-02-18T11:53:24.099 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-02-18T11:53:24.100 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:24.100 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.100 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:24.100 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:24.100 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.100 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.100 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:24.100 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.100 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.100 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.100 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.100 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.100 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-02-18T11:53:24.100 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-02-18T11:53:24.101 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:24.101 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.101 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:24.101 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-02-18T11:53:24.101 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.101 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.101 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-02-18T11:53:24.101 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-02-18T11:53:24.101 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-02-18T11:53:24.101 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-02-18T11:53:24.101 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-02-18T11:53:24.101 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-02-18T11:53:24.101 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-02-18T11:53:24.101 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:24.101 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.101 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.102 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.102 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.102 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.102 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-02-18T11:53:24.102 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-02-18T11:53:24.102 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-02-18T11:53:24.102 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.102 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-02-18T11:53:24.102 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-02-18T11:53:24.102 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.102 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.102 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:24.102 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.102 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.102 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.102 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.103 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.103 INFO:teuthology.orchestra.run.smithi012.stdout: "zabbix_host": { 2024-02-18T11:53:24.103 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix_host", 2024-02-18T11:53:24.103 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:24.103 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.103 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:24.103 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-02-18T11:53:24.103 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.103 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.103 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:24.103 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.103 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.103 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.103 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.103 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.104 INFO:teuthology.orchestra.run.smithi012.stdout: "zabbix_port": { 2024-02-18T11:53:24.104 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix_port", 2024-02-18T11:53:24.104 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-02-18T11:53:24.104 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.104 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:24.104 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10051", 2024-02-18T11:53:24.104 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.104 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.104 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:24.104 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.104 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.104 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.104 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.104 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.104 INFO:teuthology.orchestra.run.smithi012.stdout: "zabbix_sender": { 2024-02-18T11:53:24.105 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix_sender", 2024-02-18T11:53:24.105 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-02-18T11:53:24.105 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-02-18T11:53:24.105 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-02-18T11:53:24.105 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-02-18T11:53:24.105 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-02-18T11:53:24.105 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-02-18T11:53:24.105 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-02-18T11:53:24.105 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-02-18T11:53:24.105 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-02-18T11:53:24.105 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-02-18T11:53:24.105 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-02-18T11:53:24.105 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:24.105 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:24.105 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:24.105 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:24.106 INFO:teuthology.orchestra.run.smithi012.stdout: "services": { 2024-02-18T11:53:24.106 INFO:teuthology.orchestra.run.smithi012.stdout: "dashboard": "https://172.21.15.12:8443/" 2024-02-18T11:53:24.106 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.106 INFO:teuthology.orchestra.run.smithi012.stdout: "always_on_modules": { 2024-02-18T11:53:24.106 INFO:teuthology.orchestra.run.smithi012.stdout: "octopus": [ 2024-02-18T11:53:24.106 INFO:teuthology.orchestra.run.smithi012.stdout: "balancer", 2024-02-18T11:53:24.106 INFO:teuthology.orchestra.run.smithi012.stdout: "crash", 2024-02-18T11:53:24.106 INFO:teuthology.orchestra.run.smithi012.stdout: "devicehealth", 2024-02-18T11:53:24.106 INFO:teuthology.orchestra.run.smithi012.stdout: "orchestrator", 2024-02-18T11:53:24.106 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_autoscaler", 2024-02-18T11:53:24.106 INFO:teuthology.orchestra.run.smithi012.stdout: "progress", 2024-02-18T11:53:24.106 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_support", 2024-02-18T11:53:24.106 INFO:teuthology.orchestra.run.smithi012.stdout: "status", 2024-02-18T11:53:24.106 INFO:teuthology.orchestra.run.smithi012.stdout: "telemetry", 2024-02-18T11:53:24.106 INFO:teuthology.orchestra.run.smithi012.stdout: "volumes" 2024-02-18T11:53:24.107 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:24.107 INFO:teuthology.orchestra.run.smithi012.stdout: "pacific": [ 2024-02-18T11:53:24.107 INFO:teuthology.orchestra.run.smithi012.stdout: "balancer", 2024-02-18T11:53:24.107 INFO:teuthology.orchestra.run.smithi012.stdout: "crash", 2024-02-18T11:53:24.107 INFO:teuthology.orchestra.run.smithi012.stdout: "devicehealth", 2024-02-18T11:53:24.107 INFO:teuthology.orchestra.run.smithi012.stdout: "orchestrator", 2024-02-18T11:53:24.107 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_autoscaler", 2024-02-18T11:53:24.107 INFO:teuthology.orchestra.run.smithi012.stdout: "progress", 2024-02-18T11:53:24.107 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_support", 2024-02-18T11:53:24.107 INFO:teuthology.orchestra.run.smithi012.stdout: "status", 2024-02-18T11:53:24.107 INFO:teuthology.orchestra.run.smithi012.stdout: "telemetry", 2024-02-18T11:53:24.107 INFO:teuthology.orchestra.run.smithi012.stdout: "volumes" 2024-02-18T11:53:24.107 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:24.107 INFO:teuthology.orchestra.run.smithi012.stdout: "quincy": [ 2024-02-18T11:53:24.107 INFO:teuthology.orchestra.run.smithi012.stdout: "balancer", 2024-02-18T11:53:24.107 INFO:teuthology.orchestra.run.smithi012.stdout: "crash", 2024-02-18T11:53:24.108 INFO:teuthology.orchestra.run.smithi012.stdout: "devicehealth", 2024-02-18T11:53:24.108 INFO:teuthology.orchestra.run.smithi012.stdout: "orchestrator", 2024-02-18T11:53:24.108 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_autoscaler", 2024-02-18T11:53:24.108 INFO:teuthology.orchestra.run.smithi012.stdout: "progress", 2024-02-18T11:53:24.108 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_support", 2024-02-18T11:53:24.108 INFO:teuthology.orchestra.run.smithi012.stdout: "status", 2024-02-18T11:53:24.108 INFO:teuthology.orchestra.run.smithi012.stdout: "telemetry", 2024-02-18T11:53:24.108 INFO:teuthology.orchestra.run.smithi012.stdout: "volumes" 2024-02-18T11:53:24.108 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:24.108 INFO:teuthology.orchestra.run.smithi012.stdout: "reef": [ 2024-02-18T11:53:24.108 INFO:teuthology.orchestra.run.smithi012.stdout: "balancer", 2024-02-18T11:53:24.108 INFO:teuthology.orchestra.run.smithi012.stdout: "crash", 2024-02-18T11:53:24.108 INFO:teuthology.orchestra.run.smithi012.stdout: "devicehealth", 2024-02-18T11:53:24.108 INFO:teuthology.orchestra.run.smithi012.stdout: "orchestrator", 2024-02-18T11:53:24.108 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_autoscaler", 2024-02-18T11:53:24.109 INFO:teuthology.orchestra.run.smithi012.stdout: "progress", 2024-02-18T11:53:24.109 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_support", 2024-02-18T11:53:24.109 INFO:teuthology.orchestra.run.smithi012.stdout: "status", 2024-02-18T11:53:24.109 INFO:teuthology.orchestra.run.smithi012.stdout: "telemetry", 2024-02-18T11:53:24.109 INFO:teuthology.orchestra.run.smithi012.stdout: "volumes" 2024-02-18T11:53:24.109 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-02-18T11:53:24.109 INFO:teuthology.orchestra.run.smithi012.stdout: "squid": [ 2024-02-18T11:53:24.109 INFO:teuthology.orchestra.run.smithi012.stdout: "balancer", 2024-02-18T11:53:24.109 INFO:teuthology.orchestra.run.smithi012.stdout: "crash", 2024-02-18T11:53:24.109 INFO:teuthology.orchestra.run.smithi012.stdout: "devicehealth", 2024-02-18T11:53:24.109 INFO:teuthology.orchestra.run.smithi012.stdout: "orchestrator", 2024-02-18T11:53:24.109 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_autoscaler", 2024-02-18T11:53:24.109 INFO:teuthology.orchestra.run.smithi012.stdout: "progress", 2024-02-18T11:53:24.109 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_support", 2024-02-18T11:53:24.109 INFO:teuthology.orchestra.run.smithi012.stdout: "status", 2024-02-18T11:53:24.109 INFO:teuthology.orchestra.run.smithi012.stdout: "telemetry", 2024-02-18T11:53:24.110 INFO:teuthology.orchestra.run.smithi012.stdout: "volumes" 2024-02-18T11:53:24.110 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-02-18T11:53:24.110 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.110 INFO:teuthology.orchestra.run.smithi012.stdout: "last_failure_osd_epoch": 3, 2024-02-18T11:53:24.110 INFO:teuthology.orchestra.run.smithi012.stdout: "active_clients": [ 2024-02-18T11:53:24.110 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:24.110 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "devicehealth", 2024-02-18T11:53:24.110 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-02-18T11:53:24.110 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:24.110 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-02-18T11:53:24.110 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:0", 2024-02-18T11:53:24.110 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3660507355 2024-02-18T11:53:24.110 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:24.110 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-02-18T11:53:24.110 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.111 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:24.111 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "libcephsqlite", 2024-02-18T11:53:24.111 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-02-18T11:53:24.111 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:24.111 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-02-18T11:53:24.111 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:0", 2024-02-18T11:53:24.111 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3219103450 2024-02-18T11:53:24.111 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:24.111 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-02-18T11:53:24.111 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.111 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:24.111 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rbd_support", 2024-02-18T11:53:24.111 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-02-18T11:53:24.111 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:24.111 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-02-18T11:53:24.111 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:0", 2024-02-18T11:53:24.112 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3670903020 2024-02-18T11:53:24.112 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:24.112 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-02-18T11:53:24.112 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:24.112 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:24.112 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "volumes", 2024-02-18T11:53:24.112 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-02-18T11:53:24.112 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-02-18T11:53:24.112 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-02-18T11:53:24.112 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:0", 2024-02-18T11:53:24.112 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3429222323 2024-02-18T11:53:24.112 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:24.112 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-02-18T11:53:24.112 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:24.113 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-02-18T11:53:24.113 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-02-18T11:53:24.114 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:53:23 smithi158 ceph-mon[36626]: pgmap v724: 97 pgs: 97 active+clean; 10 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 93 MiB/s wr, 429 op/s 2024-02-18T11:53:24.114 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:53:23 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/1038782812' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-18T11:53:24.114 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:53:23 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/3252204354' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-18T11:53:24.114 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:53:23 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/564011142' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-02-18T11:53:24.114 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:53:23 smithi188 ceph-mon[36925]: pgmap v724: 97 pgs: 97 active+clean; 10 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 93 MiB/s wr, 429 op/s 2024-02-18T11:53:24.114 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:53:23 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/1038782812' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-18T11:53:24.114 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:53:23 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/3252204354' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-18T11:53:24.114 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:53:23 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/564011142' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-02-18T11:53:24.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.903+0000 7f227178a640 1 -- 172.21.15.12:0/564011142 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f22540a2750 msgr2=0x7f22540a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:53:24.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.903+0000 7f227178a640 1 --2- 172.21.15.12:0/564011142 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f22540a2750 0x7f22540a4c10 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f226c0fff30 tx=0x7f226000a4c0 comp rx=0 tx=0).stop 2024-02-18T11:53:24.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.903+0000 7f227178a640 1 -- 172.21.15.12:0/564011142 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f226c100920 msgr2=0x7f226c1001c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:53:24.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.903+0000 7f227178a640 1 --2- 172.21.15.12:0/564011142 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f226c100920 0x7f226c1001c0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f225c00c970 tx=0x7f225c00ce30 comp rx=0 tx=0).stop 2024-02-18T11:53:24.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.903+0000 7f227178a640 1 -- 172.21.15.12:0/564011142 shutdown_connections 2024-02-18T11:53:24.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.903+0000 7f227178a640 1 --2- 172.21.15.12:0/564011142 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f22540a2750 0x7f22540a4c10 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:53:24.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.903+0000 7f227178a640 1 --2- 172.21.15.12:0/564011142 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f226c105d50 0x7f226c19e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:53:24.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.903+0000 7f227178a640 1 --2- 172.21.15.12:0/564011142 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f226c103250 0x7f226c19bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:53:24.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.903+0000 7f227178a640 1 --2- 172.21.15.12:0/564011142 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f226c100920 0x7f226c1001c0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:53:24.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.903+0000 7f227178a640 1 -- 172.21.15.12:0/564011142 >> 172.21.15.12:0/564011142 conn(0x7f226c0fa9c0 msgr2=0x7f226c101260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:53:24.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.904+0000 7f227178a640 1 -- 172.21.15.12:0/564011142 shutdown_connections 2024-02-18T11:53:24.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:53:22.904+0000 7f227178a640 1 -- 172.21.15.12:0/564011142 wait complete. 2024-02-18T11:53:24.121 DEBUG:tasks.check_counter:Getting stats from a 2024-02-18T11:53:24.122 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph daemon mds.a perf dump 2024-02-18T11:53:24.122 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:53:23 smithi012 ceph-mon[32460]: pgmap v724: 97 pgs: 97 active+clean; 10 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 93 MiB/s wr, 429 op/s 2024-02-18T11:53:24.122 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:53:23 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/1038782812' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-18T11:53:24.122 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:53:23 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/3252204354' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-18T11:53:24.122 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:53:23 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/564011142' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-02-18T11:53:24.379 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T11:53:25.172 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:53:25 smithi012 ceph-mon[32460]: pgmap v725: 97 pgs: 97 active+clean; 10 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 92 MiB/s wr, 536 op/s 2024-02-18T11:53:25.352 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:53:25 smithi188 ceph-mon[36925]: pgmap v725: 97 pgs: 97 active+clean; 10 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 92 MiB/s wr, 536 op/s 2024-02-18T11:53:25.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:53:25 smithi158 ceph-mon[36626]: pgmap v725: 97 pgs: 97 active+clean; 10 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 92 MiB/s wr, 536 op/s 2024-02-18T11:53:26.181 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-02-18T11:53:26.181 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-0": { 2024-02-18T11:53:26.182 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 0, 2024-02-18T11:53:26.182 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 2, 2024-02-18T11:53:26.182 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 0, 2024-02-18T11:53:26.182 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 0, 2024-02-18T11:53:26.182 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 4, 2024-02-18T11:53:26.182 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 0, 2024-02-18T11:53:26.182 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.001270977, 2024-02-18T11:53:26.182 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.000000000, 2024-02-18T11:53:26.182 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.000300379, 2024-02-18T11:53:26.182 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-18T11:53:26.183 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-02-18T11:53:26.183 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-02-18T11:53:26.183 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-02-18T11:53:26.183 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-02-18T11:53:26.183 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:26.183 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-02-18T11:53:26.183 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-02-18T11:53:26.183 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-02-18T11:53:26.183 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-02-18T11:53:26.183 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:26.183 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 0, 2024-02-18T11:53:26.184 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 0 2024-02-18T11:53:26.184 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:26.184 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-1": { 2024-02-18T11:53:26.184 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 242, 2024-02-18T11:53:26.184 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 237, 2024-02-18T11:53:26.184 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 756960, 2024-02-18T11:53:26.184 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 142416, 2024-02-18T11:53:26.184 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 2, 2024-02-18T11:53:26.184 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 1, 2024-02-18T11:53:26.184 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.051635274, 2024-02-18T11:53:26.184 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.019661676, 2024-02-18T11:53:26.185 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.022660700, 2024-02-18T11:53:26.185 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.009436696, 2024-02-18T11:53:26.185 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-02-18T11:53:26.185 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 236, 2024-02-18T11:53:26.185 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.059646032, 2024-02-18T11:53:26.185 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000252737 2024-02-18T11:53:26.185 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:26.185 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-02-18T11:53:26.185 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-02-18T11:53:26.186 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-02-18T11:53:26.186 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-02-18T11:53:26.186 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:26.186 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 756960, 2024-02-18T11:53:26.186 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 142416 2024-02-18T11:53:26.188 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:26.188 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-2": { 2024-02-18T11:53:26.188 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 5, 2024-02-18T11:53:26.189 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 183, 2024-02-18T11:53:26.189 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 3184, 2024-02-18T11:53:26.189 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 109664, 2024-02-18T11:53:26.189 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 3, 2024-02-18T11:53:26.189 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 2, 2024-02-18T11:53:26.189 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.016153315, 2024-02-18T11:53:26.189 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.012416827, 2024-02-18T11:53:26.189 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.001508596, 2024-02-18T11:53:26.189 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-18T11:53:26.189 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-02-18T11:53:26.190 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 182, 2024-02-18T11:53:26.190 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.059710803, 2024-02-18T11:53:26.190 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000328081 2024-02-18T11:53:26.190 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:26.190 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-02-18T11:53:26.190 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-02-18T11:53:26.190 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-02-18T11:53:26.190 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-02-18T11:53:26.191 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:26.191 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-02-18T11:53:26.191 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 109664 2024-02-18T11:53:26.191 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:26.191 INFO:teuthology.orchestra.run.smithi012.stdout: "cct": { 2024-02-18T11:53:26.191 INFO:teuthology.orchestra.run.smithi012.stdout: "total_workers": 0, 2024-02-18T11:53:26.191 INFO:teuthology.orchestra.run.smithi012.stdout: "unhealthy_workers": 0 2024-02-18T11:53:26.191 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:26.191 INFO:teuthology.orchestra.run.smithi012.stdout: "mempool": { 2024-02-18T11:53:26.191 INFO:teuthology.orchestra.run.smithi012.stdout: "bloom_filter_bytes": 0, 2024-02-18T11:53:26.191 INFO:teuthology.orchestra.run.smithi012.stdout: "bloom_filter_items": 0, 2024-02-18T11:53:26.191 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_alloc_bytes": 0, 2024-02-18T11:53:26.192 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_alloc_items": 0, 2024-02-18T11:53:26.192 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_data_bytes": 0, 2024-02-18T11:53:26.192 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_data_items": 0, 2024-02-18T11:53:26.192 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-18T11:53:26.192 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_onode_items": 0, 2024-02-18T11:53:26.192 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-18T11:53:26.192 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_meta_items": 0, 2024-02-18T11:53:26.192 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_other_bytes": 0, 2024-02-18T11:53:26.193 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_other_items": 0, 2024-02-18T11:53:26.193 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-18T11:53:26.193 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_buffer_items": 0, 2024-02-18T11:53:26.194 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_extent_bytes": 0, 2024-02-18T11:53:26.194 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_extent_items": 0, 2024-02-18T11:53:26.194 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_blob_bytes": 0, 2024-02-18T11:53:26.194 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_blob_items": 0, 2024-02-18T11:53:26.194 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-18T11:53:26.194 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_shared_blob_items": 0, 2024-02-18T11:53:26.195 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-18T11:53:26.195 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_inline_bl_items": 0, 2024-02-18T11:53:26.195 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_bytes": 0, 2024-02-18T11:53:26.196 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_items": 0, 2024-02-18T11:53:26.196 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_txc_bytes": 0, 2024-02-18T11:53:26.196 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_txc_items": 0, 2024-02-18T11:53:26.196 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-18T11:53:26.196 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_deferred_items": 0, 2024-02-18T11:53:26.196 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_bytes": 0, 2024-02-18T11:53:26.196 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_items": 0, 2024-02-18T11:53:26.197 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_bytes": 0, 2024-02-18T11:53:26.197 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_items": 0, 2024-02-18T11:53:26.197 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_reader_bytes": 0, 2024-02-18T11:53:26.197 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_reader_items": 0, 2024-02-18T11:53:26.197 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_writer_bytes": 0, 2024-02-18T11:53:26.198 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_writer_items": 0, 2024-02-18T11:53:26.198 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_anon_bytes": 148967, 2024-02-18T11:53:26.198 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_anon_items": 29, 2024-02-18T11:53:26.198 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_meta_bytes": 0, 2024-02-18T11:53:26.198 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_meta_items": 0, 2024-02-18T11:53:26.198 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_bytes": 0, 2024-02-18T11:53:26.199 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_items": 0, 2024-02-18T11:53:26.199 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mapbl_bytes": 0, 2024-02-18T11:53:26.199 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mapbl_items": 0, 2024-02-18T11:53:26.199 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pglog_bytes": 0, 2024-02-18T11:53:26.199 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pglog_items": 0, 2024-02-18T11:53:26.199 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_bytes": 0, 2024-02-18T11:53:26.199 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_items": 0, 2024-02-18T11:53:26.200 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_mapping_bytes": 0, 2024-02-18T11:53:26.200 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_mapping_items": 0, 2024-02-18T11:53:26.200 INFO:teuthology.orchestra.run.smithi012.stdout: "pgmap_bytes": 0, 2024-02-18T11:53:26.200 INFO:teuthology.orchestra.run.smithi012.stdout: "pgmap_items": 0, 2024-02-18T11:53:26.200 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_co_bytes": 928, 2024-02-18T11:53:26.200 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_co_items": 2, 2024-02-18T11:53:26.200 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_1_bytes": 0, 2024-02-18T11:53:26.201 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_1_items": 0, 2024-02-18T11:53:26.201 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_2_bytes": 0, 2024-02-18T11:53:26.201 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_2_items": 0 2024-02-18T11:53:26.201 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:26.201 INFO:teuthology.orchestra.run.smithi012.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-18T11:53:26.201 INFO:teuthology.orchestra.run.smithi012.stdout: "val": 0, 2024-02-18T11:53:26.202 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 104857600, 2024-02-18T11:53:26.202 INFO:teuthology.orchestra.run.smithi012.stdout: "get_started": 0, 2024-02-18T11:53:26.202 INFO:teuthology.orchestra.run.smithi012.stdout: "get": 243, 2024-02-18T11:53:26.202 INFO:teuthology.orchestra.run.smithi012.stdout: "get_sum": 722788, 2024-02-18T11:53:26.202 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_fail": 0, 2024-02-18T11:53:26.202 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_success": 243, 2024-02-18T11:53:26.202 INFO:teuthology.orchestra.run.smithi012.stdout: "take": 0, 2024-02-18T11:53:26.203 INFO:teuthology.orchestra.run.smithi012.stdout: "take_sum": 0, 2024-02-18T11:53:26.203 INFO:teuthology.orchestra.run.smithi012.stdout: "put": 243, 2024-02-18T11:53:26.203 INFO:teuthology.orchestra.run.smithi012.stdout: "put_sum": 722788, 2024-02-18T11:53:26.204 INFO:teuthology.orchestra.run.smithi012.stdout: "wait": { 2024-02-18T11:53:26.204 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-02-18T11:53:26.204 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-02-18T11:53:26.204 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-02-18T11:53:26.204 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:26.205 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:26.205 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-02-18T11:53:27.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:53:27 smithi012 ceph-mon[32460]: pgmap v726: 97 pgs: 97 active+clean; 10 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 76 MiB/s wr, 506 op/s 2024-02-18T11:53:27.191 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-02-18T11:53:27.191 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-02-18T11:53:27.191 DEBUG:tasks.check_counter:Getting stats from d 2024-02-18T11:53:27.192 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph daemon mds.d perf dump 2024-02-18T11:53:27.352 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:53:27 smithi188 ceph-mon[36925]: pgmap v726: 97 pgs: 97 active+clean; 10 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 76 MiB/s wr, 506 op/s 2024-02-18T11:53:27.445 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T11:53:27.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:53:27 smithi158 ceph-mon[36626]: pgmap v726: 97 pgs: 97 active+clean; 10 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 76 MiB/s wr, 506 op/s 2024-02-18T11:53:29.125 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:53:29 smithi012 ceph-mon[32460]: pgmap v727: 97 pgs: 97 active+clean; 9.0 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 69 MiB/s wr, 515 op/s 2024-02-18T11:53:29.252 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-02-18T11:53:29.252 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-0": { 2024-02-18T11:53:29.252 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 253, 2024-02-18T11:53:29.252 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 246, 2024-02-18T11:53:29.252 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 762512, 2024-02-18T11:53:29.253 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 146944, 2024-02-18T11:53:29.253 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 4, 2024-02-18T11:53:29.253 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 0, 2024-02-18T11:53:29.253 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.054742825, 2024-02-18T11:53:29.253 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.018914998, 2024-02-18T11:53:29.253 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.028355864, 2024-02-18T11:53:29.253 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.008573248, 2024-02-18T11:53:29.253 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-02-18T11:53:29.253 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 246, 2024-02-18T11:53:29.253 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.034064312, 2024-02-18T11:53:29.253 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000138472 2024-02-18T11:53:29.253 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:29.253 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-02-18T11:53:29.253 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-02-18T11:53:29.253 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-02-18T11:53:29.254 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-02-18T11:53:29.254 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:29.254 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 762512, 2024-02-18T11:53:29.254 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 146944 2024-02-18T11:53:29.254 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:29.254 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-1": { 2024-02-18T11:53:29.254 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 2, 2024-02-18T11:53:29.254 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 186, 2024-02-18T11:53:29.254 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 320, 2024-02-18T11:53:29.254 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 187808, 2024-02-18T11:53:29.254 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 4, 2024-02-18T11:53:29.254 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 2, 2024-02-18T11:53:29.254 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.021794080, 2024-02-18T11:53:29.254 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.013715175, 2024-02-18T11:53:29.255 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.002102769, 2024-02-18T11:53:29.255 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-18T11:53:29.255 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-02-18T11:53:29.255 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 184, 2024-02-18T11:53:29.255 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.034185629, 2024-02-18T11:53:29.255 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000185791 2024-02-18T11:53:29.255 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:29.255 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-02-18T11:53:29.255 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-02-18T11:53:29.255 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-02-18T11:53:29.255 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-02-18T11:53:29.255 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:29.255 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 320, 2024-02-18T11:53:29.255 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 187808 2024-02-18T11:53:29.255 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:29.256 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-2": { 2024-02-18T11:53:29.256 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 0, 2024-02-18T11:53:29.256 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 2, 2024-02-18T11:53:29.256 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 0, 2024-02-18T11:53:29.256 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 0, 2024-02-18T11:53:29.256 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 2, 2024-02-18T11:53:29.256 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 1, 2024-02-18T11:53:29.256 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.000328423, 2024-02-18T11:53:29.256 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.000000000, 2024-02-18T11:53:29.256 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.000250293, 2024-02-18T11:53:29.256 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-18T11:53:29.256 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-02-18T11:53:29.256 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-02-18T11:53:29.256 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-02-18T11:53:29.256 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-02-18T11:53:29.257 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:29.257 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-02-18T11:53:29.257 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-02-18T11:53:29.257 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-02-18T11:53:29.257 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-02-18T11:53:29.257 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:29.257 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 0, 2024-02-18T11:53:29.257 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 0 2024-02-18T11:53:29.257 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:29.257 INFO:teuthology.orchestra.run.smithi012.stdout: "cct": { 2024-02-18T11:53:29.257 INFO:teuthology.orchestra.run.smithi012.stdout: "total_workers": 0, 2024-02-18T11:53:29.257 INFO:teuthology.orchestra.run.smithi012.stdout: "unhealthy_workers": 0 2024-02-18T11:53:29.258 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:29.258 INFO:teuthology.orchestra.run.smithi012.stdout: "mempool": { 2024-02-18T11:53:29.258 INFO:teuthology.orchestra.run.smithi012.stdout: "bloom_filter_bytes": 0, 2024-02-18T11:53:29.258 INFO:teuthology.orchestra.run.smithi012.stdout: "bloom_filter_items": 0, 2024-02-18T11:53:29.258 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_alloc_bytes": 0, 2024-02-18T11:53:29.258 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_alloc_items": 0, 2024-02-18T11:53:29.258 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_data_bytes": 0, 2024-02-18T11:53:29.258 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_data_items": 0, 2024-02-18T11:53:29.258 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-18T11:53:29.258 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_onode_items": 0, 2024-02-18T11:53:29.258 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-18T11:53:29.259 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_meta_items": 0, 2024-02-18T11:53:29.259 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_other_bytes": 0, 2024-02-18T11:53:29.259 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_other_items": 0, 2024-02-18T11:53:29.259 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-18T11:53:29.259 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_buffer_items": 0, 2024-02-18T11:53:29.259 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_extent_bytes": 0, 2024-02-18T11:53:29.259 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_extent_items": 0, 2024-02-18T11:53:29.259 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_blob_bytes": 0, 2024-02-18T11:53:29.259 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_blob_items": 0, 2024-02-18T11:53:29.259 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-18T11:53:29.259 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_shared_blob_items": 0, 2024-02-18T11:53:29.259 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-18T11:53:29.259 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_inline_bl_items": 0, 2024-02-18T11:53:29.259 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_bytes": 0, 2024-02-18T11:53:29.259 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_items": 0, 2024-02-18T11:53:29.260 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_txc_bytes": 0, 2024-02-18T11:53:29.260 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_txc_items": 0, 2024-02-18T11:53:29.260 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-18T11:53:29.260 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_deferred_items": 0, 2024-02-18T11:53:29.260 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_bytes": 0, 2024-02-18T11:53:29.260 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_items": 0, 2024-02-18T11:53:29.260 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_bytes": 0, 2024-02-18T11:53:29.260 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_items": 0, 2024-02-18T11:53:29.260 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_reader_bytes": 0, 2024-02-18T11:53:29.260 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_reader_items": 0, 2024-02-18T11:53:29.260 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_writer_bytes": 0, 2024-02-18T11:53:29.260 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_writer_items": 0, 2024-02-18T11:53:29.260 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_anon_bytes": 148721, 2024-02-18T11:53:29.260 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_anon_items": 30, 2024-02-18T11:53:29.260 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_meta_bytes": 0, 2024-02-18T11:53:29.261 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_meta_items": 0, 2024-02-18T11:53:29.261 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_bytes": 0, 2024-02-18T11:53:29.261 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_items": 0, 2024-02-18T11:53:29.261 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mapbl_bytes": 0, 2024-02-18T11:53:29.261 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mapbl_items": 0, 2024-02-18T11:53:29.261 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pglog_bytes": 0, 2024-02-18T11:53:29.261 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pglog_items": 0, 2024-02-18T11:53:29.261 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_bytes": 0, 2024-02-18T11:53:29.261 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_items": 0, 2024-02-18T11:53:29.261 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_mapping_bytes": 0, 2024-02-18T11:53:29.261 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_mapping_items": 0, 2024-02-18T11:53:29.261 INFO:teuthology.orchestra.run.smithi012.stdout: "pgmap_bytes": 0, 2024-02-18T11:53:29.261 INFO:teuthology.orchestra.run.smithi012.stdout: "pgmap_items": 0, 2024-02-18T11:53:29.261 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_co_bytes": 928, 2024-02-18T11:53:29.261 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_co_items": 2, 2024-02-18T11:53:29.262 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_1_bytes": 0, 2024-02-18T11:53:29.262 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_1_items": 0, 2024-02-18T11:53:29.262 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_2_bytes": 0, 2024-02-18T11:53:29.262 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_2_items": 0 2024-02-18T11:53:29.262 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:29.262 INFO:teuthology.orchestra.run.smithi012.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-18T11:53:29.262 INFO:teuthology.orchestra.run.smithi012.stdout: "val": 0, 2024-02-18T11:53:29.262 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 104857600, 2024-02-18T11:53:29.262 INFO:teuthology.orchestra.run.smithi012.stdout: "get_started": 0, 2024-02-18T11:53:29.262 INFO:teuthology.orchestra.run.smithi012.stdout: "get": 251, 2024-02-18T11:53:29.262 INFO:teuthology.orchestra.run.smithi012.stdout: "get_sum": 724362, 2024-02-18T11:53:29.262 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_fail": 0, 2024-02-18T11:53:29.262 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_success": 251, 2024-02-18T11:53:29.262 INFO:teuthology.orchestra.run.smithi012.stdout: "take": 0, 2024-02-18T11:53:29.263 INFO:teuthology.orchestra.run.smithi012.stdout: "take_sum": 0, 2024-02-18T11:53:29.263 INFO:teuthology.orchestra.run.smithi012.stdout: "put": 251, 2024-02-18T11:53:29.263 INFO:teuthology.orchestra.run.smithi012.stdout: "put_sum": 724362, 2024-02-18T11:53:29.263 INFO:teuthology.orchestra.run.smithi012.stdout: "wait": { 2024-02-18T11:53:29.263 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-02-18T11:53:29.263 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-02-18T11:53:29.263 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-02-18T11:53:29.263 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:29.263 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:29.263 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-02-18T11:53:29.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:53:29 smithi188 ceph-mon[36925]: pgmap v727: 97 pgs: 97 active+clean; 9.0 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 69 MiB/s wr, 515 op/s 2024-02-18T11:53:29.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:53:29 smithi158 ceph-mon[36626]: pgmap v727: 97 pgs: 97 active+clean; 9.0 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 69 MiB/s wr, 515 op/s 2024-02-18T11:53:29.916 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-02-18T11:53:29.917 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-02-18T11:53:29.917 DEBUG:tasks.check_counter:Getting stats from g 2024-02-18T11:53:29.917 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph daemon mds.g perf dump 2024-02-18T11:53:30.164 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T11:53:31.186 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:53:31 smithi012 ceph-mon[32460]: pgmap v728: 97 pgs: 97 active+clean; 8.2 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 49 MiB/s wr, 457 op/s 2024-02-18T11:53:31.352 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:53:31 smithi188 ceph-mon[36925]: pgmap v728: 97 pgs: 97 active+clean; 8.2 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 49 MiB/s wr, 457 op/s 2024-02-18T11:53:31.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:53:31 smithi158 ceph-mon[36626]: pgmap v728: 97 pgs: 97 active+clean; 8.2 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 49 MiB/s wr, 457 op/s 2024-02-18T11:53:31.795 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-02-18T11:53:31.796 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-0": { 2024-02-18T11:53:31.796 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 244, 2024-02-18T11:53:31.796 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 239, 2024-02-18T11:53:31.796 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 757504, 2024-02-18T11:53:31.796 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 143664, 2024-02-18T11:53:31.796 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 4, 2024-02-18T11:53:31.796 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 0, 2024-02-18T11:53:31.796 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.051156523, 2024-02-18T11:53:31.796 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.018553032, 2024-02-18T11:53:31.796 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.021887049, 2024-02-18T11:53:31.796 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.008918239, 2024-02-18T11:53:31.796 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-02-18T11:53:31.797 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 238, 2024-02-18T11:53:31.797 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.030260140, 2024-02-18T11:53:31.797 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000127143 2024-02-18T11:53:31.797 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:31.797 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-02-18T11:53:31.797 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-02-18T11:53:31.797 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-02-18T11:53:31.797 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-02-18T11:53:31.797 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:31.797 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 757504, 2024-02-18T11:53:31.797 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 143664 2024-02-18T11:53:31.797 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:31.797 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-1": { 2024-02-18T11:53:31.798 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 1, 2024-02-18T11:53:31.798 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 183, 2024-02-18T11:53:31.798 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 160, 2024-02-18T11:53:31.798 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 109568, 2024-02-18T11:53:31.798 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 3, 2024-02-18T11:53:31.798 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 2, 2024-02-18T11:53:31.798 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.017726336, 2024-02-18T11:53:31.798 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.013191053, 2024-02-18T11:53:31.798 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.001571710, 2024-02-18T11:53:31.798 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-18T11:53:31.798 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-02-18T11:53:31.798 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 181, 2024-02-18T11:53:31.798 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.032994025, 2024-02-18T11:53:31.798 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000182287 2024-02-18T11:53:31.799 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:31.799 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-02-18T11:53:31.799 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-02-18T11:53:31.799 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-02-18T11:53:31.799 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-02-18T11:53:31.799 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:31.799 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 160, 2024-02-18T11:53:31.799 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 109568 2024-02-18T11:53:31.799 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:31.799 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-2": { 2024-02-18T11:53:31.799 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 4, 2024-02-18T11:53:31.799 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 4, 2024-02-18T11:53:31.799 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 3024, 2024-02-18T11:53:31.799 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 448, 2024-02-18T11:53:31.800 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 2, 2024-02-18T11:53:31.800 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 1, 2024-02-18T11:53:31.800 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.000658253, 2024-02-18T11:53:31.800 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.000032290, 2024-02-18T11:53:31.800 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.000574724, 2024-02-18T11:53:31.800 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-18T11:53:31.800 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-02-18T11:53:31.800 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 3, 2024-02-18T11:53:31.800 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000789039, 2024-02-18T11:53:31.800 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000263013 2024-02-18T11:53:31.800 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:31.800 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-02-18T11:53:31.800 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-02-18T11:53:31.801 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-02-18T11:53:31.801 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-02-18T11:53:31.801 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:31.801 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-02-18T11:53:31.801 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 448 2024-02-18T11:53:31.801 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:31.801 INFO:teuthology.orchestra.run.smithi012.stdout: "cct": { 2024-02-18T11:53:31.801 INFO:teuthology.orchestra.run.smithi012.stdout: "total_workers": 0, 2024-02-18T11:53:31.801 INFO:teuthology.orchestra.run.smithi012.stdout: "unhealthy_workers": 0 2024-02-18T11:53:31.801 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:31.801 INFO:teuthology.orchestra.run.smithi012.stdout: "mempool": { 2024-02-18T11:53:31.801 INFO:teuthology.orchestra.run.smithi012.stdout: "bloom_filter_bytes": 0, 2024-02-18T11:53:31.801 INFO:teuthology.orchestra.run.smithi012.stdout: "bloom_filter_items": 0, 2024-02-18T11:53:31.801 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_alloc_bytes": 0, 2024-02-18T11:53:31.802 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_alloc_items": 0, 2024-02-18T11:53:31.802 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_data_bytes": 0, 2024-02-18T11:53:31.802 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_data_items": 0, 2024-02-18T11:53:31.802 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-18T11:53:31.802 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_onode_items": 0, 2024-02-18T11:53:31.802 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-18T11:53:31.802 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_meta_items": 0, 2024-02-18T11:53:31.802 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_other_bytes": 0, 2024-02-18T11:53:31.802 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_other_items": 0, 2024-02-18T11:53:31.802 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-18T11:53:31.802 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_buffer_items": 0, 2024-02-18T11:53:31.802 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_extent_bytes": 0, 2024-02-18T11:53:31.802 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_extent_items": 0, 2024-02-18T11:53:31.802 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_blob_bytes": 0, 2024-02-18T11:53:31.803 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_blob_items": 0, 2024-02-18T11:53:31.803 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-18T11:53:31.803 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_shared_blob_items": 0, 2024-02-18T11:53:31.803 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-18T11:53:31.803 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_inline_bl_items": 0, 2024-02-18T11:53:31.803 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_bytes": 0, 2024-02-18T11:53:31.803 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_items": 0, 2024-02-18T11:53:31.803 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_txc_bytes": 0, 2024-02-18T11:53:31.803 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_txc_items": 0, 2024-02-18T11:53:31.804 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-18T11:53:31.804 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_deferred_items": 0, 2024-02-18T11:53:31.804 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_bytes": 0, 2024-02-18T11:53:31.804 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_items": 0, 2024-02-18T11:53:31.804 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_bytes": 0, 2024-02-18T11:53:31.804 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_items": 0, 2024-02-18T11:53:31.804 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_reader_bytes": 0, 2024-02-18T11:53:31.804 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_reader_items": 0, 2024-02-18T11:53:31.804 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_writer_bytes": 0, 2024-02-18T11:53:31.804 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_writer_items": 0, 2024-02-18T11:53:31.804 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_anon_bytes": 148967, 2024-02-18T11:53:31.804 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_anon_items": 29, 2024-02-18T11:53:31.804 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_meta_bytes": 0, 2024-02-18T11:53:31.804 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_meta_items": 0, 2024-02-18T11:53:31.805 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_bytes": 0, 2024-02-18T11:53:31.805 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_items": 0, 2024-02-18T11:53:31.805 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mapbl_bytes": 0, 2024-02-18T11:53:31.805 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mapbl_items": 0, 2024-02-18T11:53:31.805 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pglog_bytes": 0, 2024-02-18T11:53:31.805 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pglog_items": 0, 2024-02-18T11:53:31.805 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_bytes": 0, 2024-02-18T11:53:31.805 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_items": 0, 2024-02-18T11:53:31.805 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_mapping_bytes": 0, 2024-02-18T11:53:31.805 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_mapping_items": 0, 2024-02-18T11:53:31.805 INFO:teuthology.orchestra.run.smithi012.stdout: "pgmap_bytes": 0, 2024-02-18T11:53:31.805 INFO:teuthology.orchestra.run.smithi012.stdout: "pgmap_items": 0, 2024-02-18T11:53:31.805 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_co_bytes": 928, 2024-02-18T11:53:31.805 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_co_items": 2, 2024-02-18T11:53:31.805 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_1_bytes": 0, 2024-02-18T11:53:31.806 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_1_items": 0, 2024-02-18T11:53:31.806 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_2_bytes": 0, 2024-02-18T11:53:31.806 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_2_items": 0 2024-02-18T11:53:31.806 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:31.806 INFO:teuthology.orchestra.run.smithi012.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-18T11:53:31.806 INFO:teuthology.orchestra.run.smithi012.stdout: "val": 0, 2024-02-18T11:53:31.806 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 104857600, 2024-02-18T11:53:31.806 INFO:teuthology.orchestra.run.smithi012.stdout: "get_started": 0, 2024-02-18T11:53:31.806 INFO:teuthology.orchestra.run.smithi012.stdout: "get": 245, 2024-02-18T11:53:31.806 INFO:teuthology.orchestra.run.smithi012.stdout: "get_sum": 723048, 2024-02-18T11:53:31.806 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_fail": 0, 2024-02-18T11:53:31.806 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_success": 245, 2024-02-18T11:53:31.806 INFO:teuthology.orchestra.run.smithi012.stdout: "take": 0, 2024-02-18T11:53:31.806 INFO:teuthology.orchestra.run.smithi012.stdout: "take_sum": 0, 2024-02-18T11:53:31.806 INFO:teuthology.orchestra.run.smithi012.stdout: "put": 245, 2024-02-18T11:53:31.807 INFO:teuthology.orchestra.run.smithi012.stdout: "put_sum": 723048, 2024-02-18T11:53:31.807 INFO:teuthology.orchestra.run.smithi012.stdout: "wait": { 2024-02-18T11:53:31.807 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-02-18T11:53:31.807 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-02-18T11:53:31.807 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-02-18T11:53:31.807 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:31.807 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:31.807 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-02-18T11:53:32.545 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-02-18T11:53:32.545 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-02-18T11:53:32.545 DEBUG:tasks.check_counter:Getting stats from j 2024-02-18T11:53:32.546 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph daemon mds.j perf dump 2024-02-18T11:53:32.800 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T11:53:33.172 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:53:33 smithi012 ceph-mon[32460]: pgmap v729: 97 pgs: 97 active+clean; 8.2 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 995 KiB/s rd, 34 MiB/s wr, 404 op/s 2024-02-18T11:53:33.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:53:33 smithi188 ceph-mon[36925]: pgmap v729: 97 pgs: 97 active+clean; 8.2 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 995 KiB/s rd, 34 MiB/s wr, 404 op/s 2024-02-18T11:53:33.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:53:33 smithi158 ceph-mon[36626]: pgmap v729: 97 pgs: 97 active+clean; 8.2 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 995 KiB/s rd, 34 MiB/s wr, 404 op/s 2024-02-18T11:53:34.517 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-02-18T11:53:34.517 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-0": { 2024-02-18T11:53:34.517 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 4, 2024-02-18T11:53:34.517 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 4, 2024-02-18T11:53:34.517 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 3024, 2024-02-18T11:53:34.517 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 448, 2024-02-18T11:53:34.517 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 4, 2024-02-18T11:53:34.518 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 0, 2024-02-18T11:53:34.518 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.001454910, 2024-02-18T11:53:34.518 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.000043705, 2024-02-18T11:53:34.518 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.000574895, 2024-02-18T11:53:34.518 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-18T11:53:34.518 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-02-18T11:53:34.518 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 3, 2024-02-18T11:53:34.518 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000771237, 2024-02-18T11:53:34.518 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000257079 2024-02-18T11:53:34.518 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:34.518 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-02-18T11:53:34.518 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-02-18T11:53:34.518 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-02-18T11:53:34.518 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-02-18T11:53:34.519 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:34.519 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-02-18T11:53:34.519 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 448 2024-02-18T11:53:34.519 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:34.519 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-1": { 2024-02-18T11:53:34.519 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 251, 2024-02-18T11:53:34.519 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 246, 2024-02-18T11:53:34.519 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 760032, 2024-02-18T11:53:34.519 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 147744, 2024-02-18T11:53:34.519 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 2, 2024-02-18T11:53:34.519 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 1, 2024-02-18T11:53:34.519 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.053096781, 2024-02-18T11:53:34.519 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.019350921, 2024-02-18T11:53:34.519 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.026023645, 2024-02-18T11:53:34.519 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.008840404, 2024-02-18T11:53:34.519 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-02-18T11:53:34.520 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 245, 2024-02-18T11:53:34.520 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.028139328, 2024-02-18T11:53:34.520 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000114854 2024-02-18T11:53:34.520 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:34.520 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-02-18T11:53:34.520 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-02-18T11:53:34.520 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-02-18T11:53:34.520 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-02-18T11:53:34.520 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:34.520 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 760032, 2024-02-18T11:53:34.520 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 147744 2024-02-18T11:53:34.520 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:34.520 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-2": { 2024-02-18T11:53:34.521 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 2, 2024-02-18T11:53:34.521 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 188, 2024-02-18T11:53:34.521 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 320, 2024-02-18T11:53:34.521 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 188160, 2024-02-18T11:53:34.521 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 4, 2024-02-18T11:53:34.521 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 2, 2024-02-18T11:53:34.521 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.018421693, 2024-02-18T11:53:34.521 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.013814943, 2024-02-18T11:53:34.521 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.001618091, 2024-02-18T11:53:34.521 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-18T11:53:34.521 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-02-18T11:53:34.521 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 186, 2024-02-18T11:53:34.521 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.030240658, 2024-02-18T11:53:34.521 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000162584 2024-02-18T11:53:34.521 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:34.522 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-02-18T11:53:34.522 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-02-18T11:53:34.522 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-02-18T11:53:34.522 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-02-18T11:53:34.522 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:34.522 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 320, 2024-02-18T11:53:34.522 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 188160 2024-02-18T11:53:34.522 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:34.522 INFO:teuthology.orchestra.run.smithi012.stdout: "cct": { 2024-02-18T11:53:34.522 INFO:teuthology.orchestra.run.smithi012.stdout: "total_workers": 0, 2024-02-18T11:53:34.522 INFO:teuthology.orchestra.run.smithi012.stdout: "unhealthy_workers": 0 2024-02-18T11:53:34.522 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:34.522 INFO:teuthology.orchestra.run.smithi012.stdout: "mempool": { 2024-02-18T11:53:34.522 INFO:teuthology.orchestra.run.smithi012.stdout: "bloom_filter_bytes": 0, 2024-02-18T11:53:34.522 INFO:teuthology.orchestra.run.smithi012.stdout: "bloom_filter_items": 0, 2024-02-18T11:53:34.523 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_alloc_bytes": 0, 2024-02-18T11:53:34.523 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_alloc_items": 0, 2024-02-18T11:53:34.523 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_data_bytes": 0, 2024-02-18T11:53:34.523 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_data_items": 0, 2024-02-18T11:53:34.523 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-18T11:53:34.523 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_onode_items": 0, 2024-02-18T11:53:34.523 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-18T11:53:34.523 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_meta_items": 0, 2024-02-18T11:53:34.523 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_other_bytes": 0, 2024-02-18T11:53:34.523 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_other_items": 0, 2024-02-18T11:53:34.523 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-18T11:53:34.524 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_buffer_items": 0, 2024-02-18T11:53:34.524 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_extent_bytes": 0, 2024-02-18T11:53:34.524 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_extent_items": 0, 2024-02-18T11:53:34.524 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_blob_bytes": 0, 2024-02-18T11:53:34.524 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_blob_items": 0, 2024-02-18T11:53:34.524 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-18T11:53:34.524 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_shared_blob_items": 0, 2024-02-18T11:53:34.524 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-18T11:53:34.524 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_inline_bl_items": 0, 2024-02-18T11:53:34.524 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_bytes": 0, 2024-02-18T11:53:34.524 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_items": 0, 2024-02-18T11:53:34.524 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_txc_bytes": 0, 2024-02-18T11:53:34.524 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_txc_items": 0, 2024-02-18T11:53:34.524 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-18T11:53:34.525 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_deferred_items": 0, 2024-02-18T11:53:34.525 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_bytes": 0, 2024-02-18T11:53:34.525 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_items": 0, 2024-02-18T11:53:34.525 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_bytes": 0, 2024-02-18T11:53:34.525 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_items": 0, 2024-02-18T11:53:34.525 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_reader_bytes": 0, 2024-02-18T11:53:34.525 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_reader_items": 0, 2024-02-18T11:53:34.525 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_writer_bytes": 0, 2024-02-18T11:53:34.525 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_writer_items": 0, 2024-02-18T11:53:34.525 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_anon_bytes": 148761, 2024-02-18T11:53:34.525 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_anon_items": 31, 2024-02-18T11:53:34.525 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_meta_bytes": 0, 2024-02-18T11:53:34.525 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_meta_items": 0, 2024-02-18T11:53:34.525 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_bytes": 0, 2024-02-18T11:53:34.525 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_items": 0, 2024-02-18T11:53:34.526 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mapbl_bytes": 0, 2024-02-18T11:53:34.526 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mapbl_items": 0, 2024-02-18T11:53:34.526 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pglog_bytes": 0, 2024-02-18T11:53:34.526 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pglog_items": 0, 2024-02-18T11:53:34.526 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_bytes": 0, 2024-02-18T11:53:34.526 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_items": 0, 2024-02-18T11:53:34.526 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_mapping_bytes": 0, 2024-02-18T11:53:34.526 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_mapping_items": 0, 2024-02-18T11:53:34.526 INFO:teuthology.orchestra.run.smithi012.stdout: "pgmap_bytes": 0, 2024-02-18T11:53:34.526 INFO:teuthology.orchestra.run.smithi012.stdout: "pgmap_items": 0, 2024-02-18T11:53:34.526 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_co_bytes": 928, 2024-02-18T11:53:34.526 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_co_items": 2, 2024-02-18T11:53:34.526 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_1_bytes": 0, 2024-02-18T11:53:34.526 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_1_items": 0, 2024-02-18T11:53:34.526 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_2_bytes": 0, 2024-02-18T11:53:34.527 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_2_items": 0 2024-02-18T11:53:34.527 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-02-18T11:53:34.527 INFO:teuthology.orchestra.run.smithi012.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-18T11:53:34.527 INFO:teuthology.orchestra.run.smithi012.stdout: "val": 0, 2024-02-18T11:53:34.527 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 104857600, 2024-02-18T11:53:34.527 INFO:teuthology.orchestra.run.smithi012.stdout: "get_started": 0, 2024-02-18T11:53:34.527 INFO:teuthology.orchestra.run.smithi012.stdout: "get": 253, 2024-02-18T11:53:34.527 INFO:teuthology.orchestra.run.smithi012.stdout: "get_sum": 724622, 2024-02-18T11:53:34.527 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_fail": 0, 2024-02-18T11:53:34.527 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_success": 253, 2024-02-18T11:53:34.527 INFO:teuthology.orchestra.run.smithi012.stdout: "take": 0, 2024-02-18T11:53:34.527 INFO:teuthology.orchestra.run.smithi012.stdout: "take_sum": 0, 2024-02-18T11:53:34.527 INFO:teuthology.orchestra.run.smithi012.stdout: "put": 253, 2024-02-18T11:53:34.527 INFO:teuthology.orchestra.run.smithi012.stdout: "put_sum": 724622, 2024-02-18T11:53:34.527 INFO:teuthology.orchestra.run.smithi012.stdout: "wait": { 2024-02-18T11:53:34.528 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-02-18T11:53:34.528 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-02-18T11:53:34.528 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-02-18T11:53:34.528 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:34.528 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-02-18T11:53:34.528 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-02-18T11:53:35.219 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:53:35 smithi012 ceph-mon[32460]: pgmap v730: 97 pgs: 97 active+clean; 8.2 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 21 MiB/s wr, 269 op/s 2024-02-18T11:53:35.220 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-02-18T11:53:35.220 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-02-18T11:53:35.220 DEBUG:tasks.check_counter:Getting stats from b 2024-02-18T11:53:35.221 DEBUG:teuthology.orchestra.run.smithi158:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph daemon mds.b perf dump 2024-02-18T11:53:35.352 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:53:35 smithi188 ceph-mon[36925]: pgmap v730: 97 pgs: 97 active+clean; 8.2 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 21 MiB/s wr, 269 op/s 2024-02-18T11:53:35.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:53:35 smithi158 ceph-mon[36626]: pgmap v730: 97 pgs: 97 active+clean; 8.2 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 21 MiB/s wr, 269 op/s 2024-02-18T11:53:36.728 INFO:teuthology.orchestra.run.smithi158.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T11:53:37.242 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:53:37 smithi158 ceph-mon[36626]: pgmap v731: 97 pgs: 97 active+clean; 8.2 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 4.2 MiB/s wr, 96 op/s 2024-02-18T11:53:37.243 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:53:37 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:53:37.352 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:53:37 smithi188 ceph-mon[36925]: pgmap v731: 97 pgs: 97 active+clean; 8.2 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 4.2 MiB/s wr, 96 op/s 2024-02-18T11:53:37.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:53:37 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:53:37.421 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:53:37 smithi012 ceph-mon[32460]: pgmap v731: 97 pgs: 97 active+clean; 8.2 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 4.2 MiB/s wr, 96 op/s 2024-02-18T11:53:37.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:53:37 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T11:53:38.352 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:53:38 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:53:38.352 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:53:38 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:53:38.352 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:53:38 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:53:38.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:53:38 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:53:38.421 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:53:38 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:53:38.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:53:38 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:53:38.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:53:38 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:53:38.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:53:38 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:53:38.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:53:38 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:53:38.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:53:38 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:53:38.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:53:38 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:53:38.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:53:38 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:53:39.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:53:39 smithi188 ceph-mon[36925]: pgmap v732: 97 pgs: 97 active+clean; 8.2 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 4.0 KiB/s wr, 56 op/s 2024-02-18T11:53:39.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:53:39 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:53:39.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:53:39 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:53:39.353 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:53:39 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:53:39.421 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:53:39 smithi012 ceph-mon[32460]: pgmap v732: 97 pgs: 97 active+clean; 8.2 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 4.0 KiB/s wr, 56 op/s 2024-02-18T11:53:39.421 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:53:39 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:53:39.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:53:39 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:53:39.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:53:39 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:53:39.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:53:39 smithi158 ceph-mon[36626]: pgmap v732: 97 pgs: 97 active+clean; 8.2 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 4.0 KiB/s wr, 56 op/s 2024-02-18T11:53:39.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:53:39 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T11:53:39.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:53:39 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T11:53:39.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:53:39 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' 2024-02-18T11:53:40.877 INFO:teuthology.orchestra.run.smithi158.stdout:{ 2024-02-18T11:53:40.878 INFO:teuthology.orchestra.run.smithi158.stdout: "AsyncMessenger::Worker-0": { 2024-02-18T11:53:40.878 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_recv_messages": 5220, 2024-02-18T11:53:40.878 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_messages": 5020, 2024-02-18T11:53:40.878 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_recv_bytes": 1391807, 2024-02-18T11:53:40.878 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_bytes": 622177688, 2024-02-18T11:53:40.878 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_created_connections": 11, 2024-02-18T11:53:40.878 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_active_connections": 3, 2024-02-18T11:53:40.878 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_total_time": 9.489263658, 2024-02-18T11:53:40.878 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_send_time": 2.441772170, 2024-02-18T11:53:40.878 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_recv_time": 0.132895699, 2024-02-18T11:53:40.878 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_fast_dispatch_time": 6.869353790, 2024-02-18T11:53:40.879 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_messages_queue_lat": { 2024-02-18T11:53:40.879 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 5018, 2024-02-18T11:53:40.879 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 1.848753557, 2024-02-18T11:53:40.879 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000368424 2024-02-18T11:53:40.879 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:40.879 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_handle_ack_lat": { 2024-02-18T11:53:40.879 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 961, 2024-02-18T11:53:40.879 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000246228, 2024-02-18T11:53:40.879 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000256 2024-02-18T11:53:40.879 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:40.879 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_recv_encrypted_bytes": 33744, 2024-02-18T11:53:40.879 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_encrypted_bytes": 1456 2024-02-18T11:53:40.880 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:40.880 INFO:teuthology.orchestra.run.smithi158.stdout: "AsyncMessenger::Worker-1": { 2024-02-18T11:53:40.880 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_recv_messages": 710046, 2024-02-18T11:53:40.880 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_messages": 712612, 2024-02-18T11:53:40.880 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_recv_bytes": 220690430, 2024-02-18T11:53:40.880 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_bytes": 672966078, 2024-02-18T11:53:40.880 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_created_connections": 11, 2024-02-18T11:53:40.880 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_active_connections": 5, 2024-02-18T11:53:40.880 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_total_time": 44.999431576, 2024-02-18T11:53:40.880 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_send_time": 17.331675435, 2024-02-18T11:53:40.880 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_recv_time": 66.863672965, 2024-02-18T11:53:40.880 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_fast_dispatch_time": 6.625091973, 2024-02-18T11:53:40.880 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_messages_queue_lat": { 2024-02-18T11:53:40.881 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 712612, 2024-02-18T11:53:40.881 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 169.499143687, 2024-02-18T11:53:40.881 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000237856 2024-02-18T11:53:40.881 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:40.881 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_handle_ack_lat": { 2024-02-18T11:53:40.881 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 532234, 2024-02-18T11:53:40.881 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.328875644, 2024-02-18T11:53:40.881 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000617 2024-02-18T11:53:40.881 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:40.881 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_recv_encrypted_bytes": 852048, 2024-02-18T11:53:40.881 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_encrypted_bytes": 180944 2024-02-18T11:53:40.881 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:40.882 INFO:teuthology.orchestra.run.smithi158.stdout: "AsyncMessenger::Worker-2": { 2024-02-18T11:53:40.882 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_recv_messages": 2485, 2024-02-18T11:53:40.882 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_messages": 2754, 2024-02-18T11:53:40.882 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_recv_bytes": 724103, 2024-02-18T11:53:40.882 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_bytes": 204591095, 2024-02-18T11:53:40.882 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_created_connections": 10, 2024-02-18T11:53:40.882 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_active_connections": 6, 2024-02-18T11:53:40.882 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_total_time": 6.062930517, 2024-02-18T11:53:40.882 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_send_time": 0.895321366, 2024-02-18T11:53:40.882 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_recv_time": 0.087975107, 2024-02-18T11:53:40.882 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_fast_dispatch_time": 5.059378786, 2024-02-18T11:53:40.882 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_messages_queue_lat": { 2024-02-18T11:53:40.882 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 2752, 2024-02-18T11:53:40.883 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 1.384081286, 2024-02-18T11:53:40.883 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000502936 2024-02-18T11:53:40.883 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:40.883 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_handle_ack_lat": { 2024-02-18T11:53:40.883 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 1280, 2024-02-18T11:53:40.883 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000429952, 2024-02-18T11:53:40.883 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000335 2024-02-18T11:53:40.883 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:40.883 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_recv_encrypted_bytes": 160, 2024-02-18T11:53:40.883 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_encrypted_bytes": 406320 2024-02-18T11:53:40.883 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:40.883 INFO:teuthology.orchestra.run.smithi158.stdout: "cct": { 2024-02-18T11:53:40.883 INFO:teuthology.orchestra.run.smithi158.stdout: "total_workers": 1, 2024-02-18T11:53:40.884 INFO:teuthology.orchestra.run.smithi158.stdout: "unhealthy_workers": 0 2024-02-18T11:53:40.884 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:40.884 INFO:teuthology.orchestra.run.smithi158.stdout: "finisher-MDSRank": { 2024-02-18T11:53:40.884 INFO:teuthology.orchestra.run.smithi158.stdout: "queue_len": 0, 2024-02-18T11:53:40.884 INFO:teuthology.orchestra.run.smithi158.stdout: "complete_latency": { 2024-02-18T11:53:40.884 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 2359, 2024-02-18T11:53:40.884 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 640.191270894, 2024-02-18T11:53:40.884 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.271382480 2024-02-18T11:53:40.884 INFO:teuthology.orchestra.run.smithi158.stdout: } 2024-02-18T11:53:40.884 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:40.884 INFO:teuthology.orchestra.run.smithi158.stdout: "finisher-PurgeQueue": { 2024-02-18T11:53:40.884 INFO:teuthology.orchestra.run.smithi158.stdout: "queue_len": 0, 2024-02-18T11:53:40.885 INFO:teuthology.orchestra.run.smithi158.stdout: "complete_latency": { 2024-02-18T11:53:40.885 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 521, 2024-02-18T11:53:40.885 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.017635990, 2024-02-18T11:53:40.885 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000033850 2024-02-18T11:53:40.885 INFO:teuthology.orchestra.run.smithi158.stdout: } 2024-02-18T11:53:40.885 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:40.885 INFO:teuthology.orchestra.run.smithi158.stdout: "mds": { 2024-02-18T11:53:40.885 INFO:teuthology.orchestra.run.smithi158.stdout: "request": 103, 2024-02-18T11:53:40.885 INFO:teuthology.orchestra.run.smithi158.stdout: "reply": 103, 2024-02-18T11:53:40.885 INFO:teuthology.orchestra.run.smithi158.stdout: "reply_latency": { 2024-02-18T11:53:40.885 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 103, 2024-02-18T11:53:40.885 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 31.603654778, 2024-02-18T11:53:40.886 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.306831599 2024-02-18T11:53:40.886 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:40.886 INFO:teuthology.orchestra.run.smithi158.stdout: "slow_reply": 0, 2024-02-18T11:53:40.886 INFO:teuthology.orchestra.run.smithi158.stdout: "forward": 0, 2024-02-18T11:53:40.886 INFO:teuthology.orchestra.run.smithi158.stdout: "dir_fetch_complete": 0, 2024-02-18T11:53:40.886 INFO:teuthology.orchestra.run.smithi158.stdout: "dir_fetch_keys": 0, 2024-02-18T11:53:40.886 INFO:teuthology.orchestra.run.smithi158.stdout: "dir_commit": 33, 2024-02-18T11:53:40.886 INFO:teuthology.orchestra.run.smithi158.stdout: "dir_split": 1, 2024-02-18T11:53:40.886 INFO:teuthology.orchestra.run.smithi158.stdout: "dir_merge": 0, 2024-02-18T11:53:40.886 INFO:teuthology.orchestra.run.smithi158.stdout: "inodes": 27, 2024-02-18T11:53:40.886 INFO:teuthology.orchestra.run.smithi158.stdout: "inodes_top": 4, 2024-02-18T11:53:40.886 INFO:teuthology.orchestra.run.smithi158.stdout: "inodes_bottom": 23, 2024-02-18T11:53:40.886 INFO:teuthology.orchestra.run.smithi158.stdout: "inodes_pin_tail": 0, 2024-02-18T11:53:40.887 INFO:teuthology.orchestra.run.smithi158.stdout: "inodes_pinned": 20, 2024-02-18T11:53:40.887 INFO:teuthology.orchestra.run.smithi158.stdout: "inodes_expired": 0, 2024-02-18T11:53:40.887 INFO:teuthology.orchestra.run.smithi158.stdout: "inodes_with_caps": 6, 2024-02-18T11:53:40.887 INFO:teuthology.orchestra.run.smithi158.stdout: "caps": 6, 2024-02-18T11:53:40.887 INFO:teuthology.orchestra.run.smithi158.stdout: "subtrees": 2, 2024-02-18T11:53:40.887 INFO:teuthology.orchestra.run.smithi158.stdout: "traverse": 135, 2024-02-18T11:53:40.887 INFO:teuthology.orchestra.run.smithi158.stdout: "traverse_hit": 127, 2024-02-18T11:53:40.887 INFO:teuthology.orchestra.run.smithi158.stdout: "traverse_forward": 0, 2024-02-18T11:53:40.887 INFO:teuthology.orchestra.run.smithi158.stdout: "traverse_discover": 0, 2024-02-18T11:53:40.887 INFO:teuthology.orchestra.run.smithi158.stdout: "traverse_dir_fetch": 0, 2024-02-18T11:53:40.887 INFO:teuthology.orchestra.run.smithi158.stdout: "traverse_remote_ino": 0, 2024-02-18T11:53:40.887 INFO:teuthology.orchestra.run.smithi158.stdout: "traverse_lock": 0, 2024-02-18T11:53:40.887 INFO:teuthology.orchestra.run.smithi158.stdout: "load_cent": 0, 2024-02-18T11:53:40.888 INFO:teuthology.orchestra.run.smithi158.stdout: "q": 0, 2024-02-18T11:53:40.888 INFO:teuthology.orchestra.run.smithi158.stdout: "exported": 0, 2024-02-18T11:53:40.888 INFO:teuthology.orchestra.run.smithi158.stdout: "exported_inodes": 0, 2024-02-18T11:53:40.888 INFO:teuthology.orchestra.run.smithi158.stdout: "imported": 0, 2024-02-18T11:53:40.888 INFO:teuthology.orchestra.run.smithi158.stdout: "imported_inodes": 0, 2024-02-18T11:53:40.888 INFO:teuthology.orchestra.run.smithi158.stdout: "openino_dir_fetch": 0, 2024-02-18T11:53:40.888 INFO:teuthology.orchestra.run.smithi158.stdout: "openino_backtrace_fetch": 0, 2024-02-18T11:53:40.888 INFO:teuthology.orchestra.run.smithi158.stdout: "openino_peer_discover": 0, 2024-02-18T11:53:40.888 INFO:teuthology.orchestra.run.smithi158.stdout: "root_rfiles": 4, 2024-02-18T11:53:40.888 INFO:teuthology.orchestra.run.smithi158.stdout: "root_rbytes": 234, 2024-02-18T11:53:40.888 INFO:teuthology.orchestra.run.smithi158.stdout: "root_rsnaps": 0, 2024-02-18T11:53:40.888 INFO:teuthology.orchestra.run.smithi158.stdout: "scrub_backtrace_fetch": 0, 2024-02-18T11:53:40.888 INFO:teuthology.orchestra.run.smithi158.stdout: "scrub_set_tag": 0, 2024-02-18T11:53:40.888 INFO:teuthology.orchestra.run.smithi158.stdout: "scrub_backtrace_repaired": 0, 2024-02-18T11:53:40.889 INFO:teuthology.orchestra.run.smithi158.stdout: "scrub_inotable_repaired": 0, 2024-02-18T11:53:40.889 INFO:teuthology.orchestra.run.smithi158.stdout: "scrub_dir_inodes": 0, 2024-02-18T11:53:40.889 INFO:teuthology.orchestra.run.smithi158.stdout: "scrub_dir_base_inodes": 0, 2024-02-18T11:53:40.889 INFO:teuthology.orchestra.run.smithi158.stdout: "scrub_dirfrag_rstats": 0, 2024-02-18T11:53:40.889 INFO:teuthology.orchestra.run.smithi158.stdout: "scrub_file_inodes": 0, 2024-02-18T11:53:40.889 INFO:teuthology.orchestra.run.smithi158.stdout: "handle_inode_file_caps": 0, 2024-02-18T11:53:40.889 INFO:teuthology.orchestra.run.smithi158.stdout: "ceph_cap_op_revoke": 21, 2024-02-18T11:53:40.889 INFO:teuthology.orchestra.run.smithi158.stdout: "ceph_cap_op_grant": 3416, 2024-02-18T11:53:40.889 INFO:teuthology.orchestra.run.smithi158.stdout: "ceph_cap_op_trunc": 0, 2024-02-18T11:53:40.889 INFO:teuthology.orchestra.run.smithi158.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-02-18T11:53:40.889 INFO:teuthology.orchestra.run.smithi158.stdout: "ceph_cap_op_flush_ack": 0, 2024-02-18T11:53:40.889 INFO:teuthology.orchestra.run.smithi158.stdout: "handle_client_caps": 705367, 2024-02-18T11:53:40.889 INFO:teuthology.orchestra.run.smithi158.stdout: "handle_client_caps_dirty": 705315, 2024-02-18T11:53:40.890 INFO:teuthology.orchestra.run.smithi158.stdout: "handle_client_cap_release": 2, 2024-02-18T11:53:40.890 INFO:teuthology.orchestra.run.smithi158.stdout: "process_request_cap_release": 48 2024-02-18T11:53:40.890 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:40.890 INFO:teuthology.orchestra.run.smithi158.stdout: "mds_cache": { 2024-02-18T11:53:40.890 INFO:teuthology.orchestra.run.smithi158.stdout: "dir_update": 0, 2024-02-18T11:53:40.890 INFO:teuthology.orchestra.run.smithi158.stdout: "dir_update_receipt": 0, 2024-02-18T11:53:40.890 INFO:teuthology.orchestra.run.smithi158.stdout: "dir_try_discover": 0, 2024-02-18T11:53:40.890 INFO:teuthology.orchestra.run.smithi158.stdout: "dir_send_discover": 0, 2024-02-18T11:53:40.890 INFO:teuthology.orchestra.run.smithi158.stdout: "dir_handle_discover": 4, 2024-02-18T11:53:40.890 INFO:teuthology.orchestra.run.smithi158.stdout: "num_strays": 4, 2024-02-18T11:53:40.890 INFO:teuthology.orchestra.run.smithi158.stdout: "num_strays_delayed": 0, 2024-02-18T11:53:40.890 INFO:teuthology.orchestra.run.smithi158.stdout: "num_strays_enqueuing": 0, 2024-02-18T11:53:40.890 INFO:teuthology.orchestra.run.smithi158.stdout: "strays_created": 22, 2024-02-18T11:53:40.890 INFO:teuthology.orchestra.run.smithi158.stdout: "strays_enqueued": 18, 2024-02-18T11:53:40.890 INFO:teuthology.orchestra.run.smithi158.stdout: "strays_reintegrated": 0, 2024-02-18T11:53:40.890 INFO:teuthology.orchestra.run.smithi158.stdout: "strays_migrated": 0, 2024-02-18T11:53:40.891 INFO:teuthology.orchestra.run.smithi158.stdout: "num_recovering_processing": 0, 2024-02-18T11:53:40.891 INFO:teuthology.orchestra.run.smithi158.stdout: "num_recovering_enqueued": 0, 2024-02-18T11:53:40.891 INFO:teuthology.orchestra.run.smithi158.stdout: "num_recovering_prioritized": 0, 2024-02-18T11:53:40.891 INFO:teuthology.orchestra.run.smithi158.stdout: "recovery_started": 0, 2024-02-18T11:53:40.891 INFO:teuthology.orchestra.run.smithi158.stdout: "recovery_completed": 0, 2024-02-18T11:53:40.891 INFO:teuthology.orchestra.run.smithi158.stdout: "ireq_enqueue_scrub": 0, 2024-02-18T11:53:40.891 INFO:teuthology.orchestra.run.smithi158.stdout: "ireq_exportdir": 0, 2024-02-18T11:53:40.891 INFO:teuthology.orchestra.run.smithi158.stdout: "ireq_flush": 0, 2024-02-18T11:53:40.891 INFO:teuthology.orchestra.run.smithi158.stdout: "ireq_fragmentdir": 1, 2024-02-18T11:53:40.892 INFO:teuthology.orchestra.run.smithi158.stdout: "ireq_fragstats": 0, 2024-02-18T11:53:40.892 INFO:teuthology.orchestra.run.smithi158.stdout: "ireq_inodestats": 0 2024-02-18T11:53:40.892 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:40.892 INFO:teuthology.orchestra.run.smithi158.stdout: "mds_log": { 2024-02-18T11:53:40.892 INFO:teuthology.orchestra.run.smithi158.stdout: "evlrg": 0, 2024-02-18T11:53:40.892 INFO:teuthology.orchestra.run.smithi158.stdout: "evadd": 707101, 2024-02-18T11:53:40.892 INFO:teuthology.orchestra.run.smithi158.stdout: "evex": 597324, 2024-02-18T11:53:40.892 INFO:teuthology.orchestra.run.smithi158.stdout: "evtrm": 589921, 2024-02-18T11:53:40.892 INFO:teuthology.orchestra.run.smithi158.stdout: "ev": 117180, 2024-02-18T11:53:40.892 INFO:teuthology.orchestra.run.smithi158.stdout: "evexg": 0, 2024-02-18T11:53:40.892 INFO:teuthology.orchestra.run.smithi158.stdout: "evexd": 7403, 2024-02-18T11:53:40.892 INFO:teuthology.orchestra.run.smithi158.stdout: "segadd": 821, 2024-02-18T11:53:40.892 INFO:teuthology.orchestra.run.smithi158.stdout: "segex": 693, 2024-02-18T11:53:40.892 INFO:teuthology.orchestra.run.smithi158.stdout: "segtrm": 685, 2024-02-18T11:53:40.893 INFO:teuthology.orchestra.run.smithi158.stdout: "seg": 136, 2024-02-18T11:53:40.893 INFO:teuthology.orchestra.run.smithi158.stdout: "segmjr": 10, 2024-02-18T11:53:40.893 INFO:teuthology.orchestra.run.smithi158.stdout: "segexg": 0, 2024-02-18T11:53:40.893 INFO:teuthology.orchestra.run.smithi158.stdout: "segexd": 8, 2024-02-18T11:53:40.893 INFO:teuthology.orchestra.run.smithi158.stdout: "expos": 1015262913, 2024-02-18T11:53:40.893 INFO:teuthology.orchestra.run.smithi158.stdout: "wrpos": 1230575980, 2024-02-18T11:53:40.893 INFO:teuthology.orchestra.run.smithi158.stdout: "rdpos": 0, 2024-02-18T11:53:40.893 INFO:teuthology.orchestra.run.smithi158.stdout: "jlat": { 2024-02-18T11:53:40.893 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 7069, 2024-02-18T11:53:40.893 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 2408.806878671, 2024-02-18T11:53:40.893 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.340756384 2024-02-18T11:53:40.893 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:40.893 INFO:teuthology.orchestra.run.smithi158.stdout: "replayed": 0 2024-02-18T11:53:40.893 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:40.894 INFO:teuthology.orchestra.run.smithi158.stdout: "mds_mem": { 2024-02-18T11:53:40.894 INFO:teuthology.orchestra.run.smithi158.stdout: "ino": 27, 2024-02-18T11:53:40.894 INFO:teuthology.orchestra.run.smithi158.stdout: "ino+": 45, 2024-02-18T11:53:40.894 INFO:teuthology.orchestra.run.smithi158.stdout: "ino-": 18, 2024-02-18T11:53:40.894 INFO:teuthology.orchestra.run.smithi158.stdout: "dir": 18, 2024-02-18T11:53:40.894 INFO:teuthology.orchestra.run.smithi158.stdout: "dir+": 30, 2024-02-18T11:53:40.894 INFO:teuthology.orchestra.run.smithi158.stdout: "dir-": 12, 2024-02-18T11:53:40.894 INFO:teuthology.orchestra.run.smithi158.stdout: "dn": 27, 2024-02-18T11:53:40.894 INFO:teuthology.orchestra.run.smithi158.stdout: "dn+": 66, 2024-02-18T11:53:40.894 INFO:teuthology.orchestra.run.smithi158.stdout: "dn-": 39, 2024-02-18T11:53:40.894 INFO:teuthology.orchestra.run.smithi158.stdout: "cap": 6, 2024-02-18T11:53:40.894 INFO:teuthology.orchestra.run.smithi158.stdout: "cap+": 36, 2024-02-18T11:53:40.894 INFO:teuthology.orchestra.run.smithi158.stdout: "cap-": 30, 2024-02-18T11:53:40.894 INFO:teuthology.orchestra.run.smithi158.stdout: "rss": 92400, 2024-02-18T11:53:40.895 INFO:teuthology.orchestra.run.smithi158.stdout: "heap": 215420 2024-02-18T11:53:40.895 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:40.895 INFO:teuthology.orchestra.run.smithi158.stdout: "mds_server": { 2024-02-18T11:53:40.895 INFO:teuthology.orchestra.run.smithi158.stdout: "dispatch_client_request": 130, 2024-02-18T11:53:40.895 INFO:teuthology.orchestra.run.smithi158.stdout: "dispatch_server_request": 1, 2024-02-18T11:53:40.895 INFO:teuthology.orchestra.run.smithi158.stdout: "handle_client_request": 103, 2024-02-18T11:53:40.895 INFO:teuthology.orchestra.run.smithi158.stdout: "handle_client_session": 21, 2024-02-18T11:53:40.895 INFO:teuthology.orchestra.run.smithi158.stdout: "handle_peer_request": 2, 2024-02-18T11:53:40.895 INFO:teuthology.orchestra.run.smithi158.stdout: "req_create_latency": { 2024-02-18T11:53:40.895 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 22, 2024-02-18T11:53:40.895 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.020988741, 2024-02-18T11:53:40.895 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000954033 2024-02-18T11:53:40.895 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:40.895 INFO:teuthology.orchestra.run.smithi158.stdout: "req_getattr_latency": { 2024-02-18T11:53:40.896 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 16, 2024-02-18T11:53:40.896 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.003774383, 2024-02-18T11:53:40.896 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000235898 2024-02-18T11:53:40.896 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:40.896 INFO:teuthology.orchestra.run.smithi158.stdout: "req_getfilelock_latency": { 2024-02-18T11:53:40.896 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:40.896 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:40.896 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:40.896 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:40.896 INFO:teuthology.orchestra.run.smithi158.stdout: "req_link_latency": { 2024-02-18T11:53:40.896 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:40.896 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:40.896 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:40.897 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:40.897 INFO:teuthology.orchestra.run.smithi158.stdout: "req_lookup_latency": { 2024-02-18T11:53:40.897 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 16, 2024-02-18T11:53:40.897 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.005855008, 2024-02-18T11:53:40.897 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000365938 2024-02-18T11:53:40.897 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:40.897 INFO:teuthology.orchestra.run.smithi158.stdout: "req_lookuphash_latency": { 2024-02-18T11:53:40.897 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:40.897 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:40.897 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:40.897 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:40.897 INFO:teuthology.orchestra.run.smithi158.stdout: "req_lookupino_latency": { 2024-02-18T11:53:40.897 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 1, 2024-02-18T11:53:40.897 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000177057, 2024-02-18T11:53:40.898 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000177057 2024-02-18T11:53:40.898 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:40.898 INFO:teuthology.orchestra.run.smithi158.stdout: "req_lookupname_latency": { 2024-02-18T11:53:40.898 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:40.898 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:40.898 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:40.898 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:40.898 INFO:teuthology.orchestra.run.smithi158.stdout: "req_lookupparent_latency": { 2024-02-18T11:53:40.898 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:40.898 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:40.898 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:40.898 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:40.898 INFO:teuthology.orchestra.run.smithi158.stdout: "req_lookupsnap_latency": { 2024-02-18T11:53:40.899 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:40.899 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:40.899 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:40.899 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:40.899 INFO:teuthology.orchestra.run.smithi158.stdout: "req_lssnap_latency": { 2024-02-18T11:53:40.899 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:40.899 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:40.899 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:40.899 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:40.899 INFO:teuthology.orchestra.run.smithi158.stdout: "req_mkdir_latency": { 2024-02-18T11:53:40.899 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 10, 2024-02-18T11:53:40.899 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.013829756, 2024-02-18T11:53:40.899 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.001382975 2024-02-18T11:53:40.900 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:40.900 INFO:teuthology.orchestra.run.smithi158.stdout: "req_mknod_latency": { 2024-02-18T11:53:40.900 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:40.900 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:40.900 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:40.900 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:40.900 INFO:teuthology.orchestra.run.smithi158.stdout: "req_mksnap_latency": { 2024-02-18T11:53:40.900 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:40.900 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:40.900 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:40.900 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:40.900 INFO:teuthology.orchestra.run.smithi158.stdout: "req_open_latency": { 2024-02-18T11:53:40.900 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:40.900 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:40.901 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:40.901 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:40.901 INFO:teuthology.orchestra.run.smithi158.stdout: "req_readdir_latency": { 2024-02-18T11:53:40.901 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 5, 2024-02-18T11:53:40.901 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 31.013233848, 2024-02-18T11:53:40.901 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 6.202646769 2024-02-18T11:53:40.901 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:40.901 INFO:teuthology.orchestra.run.smithi158.stdout: "req_rename_latency": { 2024-02-18T11:53:40.901 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 2, 2024-02-18T11:53:40.901 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.002210743, 2024-02-18T11:53:40.901 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.001105371 2024-02-18T11:53:40.901 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:40.901 INFO:teuthology.orchestra.run.smithi158.stdout: "req_renamesnap_latency": { 2024-02-18T11:53:40.902 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:40.902 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:40.902 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:40.902 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:40.902 INFO:teuthology.orchestra.run.smithi158.stdout: "req_snapdiff_latency": { 2024-02-18T11:53:40.902 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:40.902 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:40.902 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:40.902 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:40.902 INFO:teuthology.orchestra.run.smithi158.stdout: "req_rmdir_latency": { 2024-02-18T11:53:40.902 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 4, 2024-02-18T11:53:40.902 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.142489973, 2024-02-18T11:53:40.902 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.035622493 2024-02-18T11:53:40.902 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:40.903 INFO:teuthology.orchestra.run.smithi158.stdout: "req_rmsnap_latency": { 2024-02-18T11:53:40.903 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:40.903 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:40.903 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:40.903 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:40.903 INFO:teuthology.orchestra.run.smithi158.stdout: "req_rmxattr_latency": { 2024-02-18T11:53:40.903 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:40.903 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:40.903 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:40.903 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:40.903 INFO:teuthology.orchestra.run.smithi158.stdout: "req_setattr_latency": { 2024-02-18T11:53:40.903 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 1, 2024-02-18T11:53:40.903 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.001669244, 2024-02-18T11:53:40.904 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.001669244 2024-02-18T11:53:40.904 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:40.904 INFO:teuthology.orchestra.run.smithi158.stdout: "req_setdirlayout_latency": { 2024-02-18T11:53:40.904 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:40.904 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:40.904 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:40.904 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:40.904 INFO:teuthology.orchestra.run.smithi158.stdout: "req_setfilelock_latency": { 2024-02-18T11:53:40.904 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:40.904 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:40.904 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:40.904 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:40.904 INFO:teuthology.orchestra.run.smithi158.stdout: "req_setlayout_latency": { 2024-02-18T11:53:40.905 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:40.905 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:40.905 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:40.905 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:40.905 INFO:teuthology.orchestra.run.smithi158.stdout: "req_setxattr_latency": { 2024-02-18T11:53:40.905 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 8, 2024-02-18T11:53:40.905 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.020087816, 2024-02-18T11:53:40.905 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.002510977 2024-02-18T11:53:40.905 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:40.905 INFO:teuthology.orchestra.run.smithi158.stdout: "req_symlink_latency": { 2024-02-18T11:53:40.905 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:40.905 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:40.905 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:40.906 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:40.906 INFO:teuthology.orchestra.run.smithi158.stdout: "req_unlink_latency": { 2024-02-18T11:53:40.906 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 18, 2024-02-18T11:53:40.906 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.379338209, 2024-02-18T11:53:40.906 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.021074344 2024-02-18T11:53:40.906 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:40.906 INFO:teuthology.orchestra.run.smithi158.stdout: "cap_revoke_eviction": 0, 2024-02-18T11:53:40.906 INFO:teuthology.orchestra.run.smithi158.stdout: "cap_acquisition_throttle": 0, 2024-02-18T11:53:40.906 INFO:teuthology.orchestra.run.smithi158.stdout: "req_getvxattr_latency": { 2024-02-18T11:53:40.906 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:40.906 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:40.906 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:40.906 INFO:teuthology.orchestra.run.smithi158.stdout: } 2024-02-18T11:53:40.906 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:40.907 INFO:teuthology.orchestra.run.smithi158.stdout: "mds_sessions": { 2024-02-18T11:53:40.907 INFO:teuthology.orchestra.run.smithi158.stdout: "session_count": 1, 2024-02-18T11:53:40.907 INFO:teuthology.orchestra.run.smithi158.stdout: "session_add": 3, 2024-02-18T11:53:40.907 INFO:teuthology.orchestra.run.smithi158.stdout: "session_remove": 2, 2024-02-18T11:53:40.907 INFO:teuthology.orchestra.run.smithi158.stdout: "sessions_open": 1, 2024-02-18T11:53:40.907 INFO:teuthology.orchestra.run.smithi158.stdout: "sessions_stale": 0, 2024-02-18T11:53:40.907 INFO:teuthology.orchestra.run.smithi158.stdout: "total_load": 41, 2024-02-18T11:53:40.907 INFO:teuthology.orchestra.run.smithi158.stdout: "average_load": 41, 2024-02-18T11:53:40.907 INFO:teuthology.orchestra.run.smithi158.stdout: "avg_session_uptime": 781, 2024-02-18T11:53:40.907 INFO:teuthology.orchestra.run.smithi158.stdout: "mdthresh_evicted": 0 2024-02-18T11:53:40.907 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:40.907 INFO:teuthology.orchestra.run.smithi158.stdout: "mempool": { 2024-02-18T11:53:40.908 INFO:teuthology.orchestra.run.smithi158.stdout: "bloom_filter_bytes": 0, 2024-02-18T11:53:40.908 INFO:teuthology.orchestra.run.smithi158.stdout: "bloom_filter_items": 0, 2024-02-18T11:53:40.908 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_alloc_bytes": 0, 2024-02-18T11:53:40.908 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_alloc_items": 0, 2024-02-18T11:53:40.908 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_data_bytes": 0, 2024-02-18T11:53:40.908 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_data_items": 0, 2024-02-18T11:53:40.908 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-18T11:53:40.908 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_onode_items": 0, 2024-02-18T11:53:40.908 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-18T11:53:40.908 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_meta_items": 0, 2024-02-18T11:53:40.908 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_other_bytes": 0, 2024-02-18T11:53:40.908 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_other_items": 0, 2024-02-18T11:53:40.908 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-18T11:53:40.909 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_buffer_items": 0, 2024-02-18T11:53:40.909 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_extent_bytes": 0, 2024-02-18T11:53:40.909 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_extent_items": 0, 2024-02-18T11:53:40.909 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_blob_bytes": 0, 2024-02-18T11:53:40.909 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_blob_items": 0, 2024-02-18T11:53:40.909 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-18T11:53:40.909 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_shared_blob_items": 0, 2024-02-18T11:53:40.909 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-18T11:53:40.909 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_inline_bl_items": 0, 2024-02-18T11:53:40.909 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_fsck_bytes": 0, 2024-02-18T11:53:40.909 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_fsck_items": 0, 2024-02-18T11:53:40.909 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_txc_bytes": 0, 2024-02-18T11:53:40.909 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_txc_items": 0, 2024-02-18T11:53:40.909 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-18T11:53:40.910 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_writing_deferred_items": 0, 2024-02-18T11:53:40.910 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_writing_bytes": 0, 2024-02-18T11:53:40.910 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_writing_items": 0, 2024-02-18T11:53:40.910 INFO:teuthology.orchestra.run.smithi158.stdout: "bluefs_bytes": 0, 2024-02-18T11:53:40.910 INFO:teuthology.orchestra.run.smithi158.stdout: "bluefs_items": 0, 2024-02-18T11:53:40.910 INFO:teuthology.orchestra.run.smithi158.stdout: "bluefs_file_reader_bytes": 0, 2024-02-18T11:53:40.910 INFO:teuthology.orchestra.run.smithi158.stdout: "bluefs_file_reader_items": 0, 2024-02-18T11:53:40.910 INFO:teuthology.orchestra.run.smithi158.stdout: "bluefs_file_writer_bytes": 0, 2024-02-18T11:53:40.910 INFO:teuthology.orchestra.run.smithi158.stdout: "bluefs_file_writer_items": 0, 2024-02-18T11:53:40.910 INFO:teuthology.orchestra.run.smithi158.stdout: "buffer_anon_bytes": 306443, 2024-02-18T11:53:40.910 INFO:teuthology.orchestra.run.smithi158.stdout: "buffer_anon_items": 121, 2024-02-18T11:53:40.910 INFO:teuthology.orchestra.run.smithi158.stdout: "buffer_meta_bytes": 0, 2024-02-18T11:53:40.910 INFO:teuthology.orchestra.run.smithi158.stdout: "buffer_meta_items": 0, 2024-02-18T11:53:40.911 INFO:teuthology.orchestra.run.smithi158.stdout: "osd_bytes": 0, 2024-02-18T11:53:40.911 INFO:teuthology.orchestra.run.smithi158.stdout: "osd_items": 0, 2024-02-18T11:53:40.911 INFO:teuthology.orchestra.run.smithi158.stdout: "osd_mapbl_bytes": 0, 2024-02-18T11:53:40.911 INFO:teuthology.orchestra.run.smithi158.stdout: "osd_mapbl_items": 0, 2024-02-18T11:53:40.911 INFO:teuthology.orchestra.run.smithi158.stdout: "osd_pglog_bytes": 0, 2024-02-18T11:53:40.911 INFO:teuthology.orchestra.run.smithi158.stdout: "osd_pglog_items": 0, 2024-02-18T11:53:40.911 INFO:teuthology.orchestra.run.smithi158.stdout: "osdmap_bytes": 7528, 2024-02-18T11:53:40.911 INFO:teuthology.orchestra.run.smithi158.stdout: "osdmap_items": 158, 2024-02-18T11:53:40.911 INFO:teuthology.orchestra.run.smithi158.stdout: "osdmap_mapping_bytes": 0, 2024-02-18T11:53:40.911 INFO:teuthology.orchestra.run.smithi158.stdout: "osdmap_mapping_items": 0, 2024-02-18T11:53:40.911 INFO:teuthology.orchestra.run.smithi158.stdout: "pgmap_bytes": 0, 2024-02-18T11:53:40.911 INFO:teuthology.orchestra.run.smithi158.stdout: "pgmap_items": 0, 2024-02-18T11:53:40.911 INFO:teuthology.orchestra.run.smithi158.stdout: "mds_co_bytes": 105766, 2024-02-18T11:53:40.911 INFO:teuthology.orchestra.run.smithi158.stdout: "mds_co_items": 1115, 2024-02-18T11:53:40.912 INFO:teuthology.orchestra.run.smithi158.stdout: "unittest_1_bytes": 0, 2024-02-18T11:53:40.912 INFO:teuthology.orchestra.run.smithi158.stdout: "unittest_1_items": 0, 2024-02-18T11:53:40.912 INFO:teuthology.orchestra.run.smithi158.stdout: "unittest_2_bytes": 0, 2024-02-18T11:53:40.912 INFO:teuthology.orchestra.run.smithi158.stdout: "unittest_2_items": 0 2024-02-18T11:53:40.912 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:40.912 INFO:teuthology.orchestra.run.smithi158.stdout: "objecter": { 2024-02-18T11:53:40.912 INFO:teuthology.orchestra.run.smithi158.stdout: "op_active": 0, 2024-02-18T11:53:40.912 INFO:teuthology.orchestra.run.smithi158.stdout: "op_laggy": 0, 2024-02-18T11:53:40.912 INFO:teuthology.orchestra.run.smithi158.stdout: "op_send": 9878, 2024-02-18T11:53:40.912 INFO:teuthology.orchestra.run.smithi158.stdout: "op_send_bytes": 1322003147, 2024-02-18T11:53:40.912 INFO:teuthology.orchestra.run.smithi158.stdout: "op_resend": 31, 2024-02-18T11:53:40.912 INFO:teuthology.orchestra.run.smithi158.stdout: "op_reply": 9878, 2024-02-18T11:53:40.912 INFO:teuthology.orchestra.run.smithi158.stdout: "op_latency": { 2024-02-18T11:53:40.913 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 9878, 2024-02-18T11:53:40.913 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 757.394874340, 2024-02-18T11:53:40.913 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.076674921 2024-02-18T11:53:40.913 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:40.913 INFO:teuthology.orchestra.run.smithi158.stdout: "op_inflight": 0, 2024-02-18T11:53:40.913 INFO:teuthology.orchestra.run.smithi158.stdout: "oplen_avg": { 2024-02-18T11:53:40.913 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 9878, 2024-02-18T11:53:40.913 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 10014 2024-02-18T11:53:40.913 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:40.913 INFO:teuthology.orchestra.run.smithi158.stdout: "op": 9878, 2024-02-18T11:53:40.913 INFO:teuthology.orchestra.run.smithi158.stdout: "op_r": 2, 2024-02-18T11:53:40.913 INFO:teuthology.orchestra.run.smithi158.stdout: "op_w": 9876, 2024-02-18T11:53:40.913 INFO:teuthology.orchestra.run.smithi158.stdout: "op_rmw": 0, 2024-02-18T11:53:40.913 INFO:teuthology.orchestra.run.smithi158.stdout: "op_pg": 0, 2024-02-18T11:53:40.913 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_stat": 3, 2024-02-18T11:53:40.914 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_create": 44, 2024-02-18T11:53:40.914 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_read": 2, 2024-02-18T11:53:40.914 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_write": 7071, 2024-02-18T11:53:40.914 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_writefull": 75, 2024-02-18T11:53:40.914 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_writesame": 0, 2024-02-18T11:53:40.914 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_append": 0, 2024-02-18T11:53:40.914 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_zero": 0, 2024-02-18T11:53:40.914 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_truncate": 1, 2024-02-18T11:53:40.914 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_delete": 1583, 2024-02-18T11:53:40.914 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_mapext": 0, 2024-02-18T11:53:40.914 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_sparse_read": 0, 2024-02-18T11:53:40.914 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_clonerange": 0, 2024-02-18T11:53:40.914 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_getxattr": 0, 2024-02-18T11:53:40.915 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_setxattr": 88, 2024-02-18T11:53:40.915 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_cmpxattr": 0, 2024-02-18T11:53:40.915 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_rmxattr": 0, 2024-02-18T11:53:40.915 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_resetxattrs": 0, 2024-02-18T11:53:40.915 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_call": 0, 2024-02-18T11:53:40.915 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_watch": 0, 2024-02-18T11:53:40.915 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_notify": 0, 2024-02-18T11:53:40.915 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_src_cmpxattr": 0, 2024-02-18T11:53:40.915 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_pgls": 0, 2024-02-18T11:53:40.915 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_pgls_filter": 0, 2024-02-18T11:53:40.915 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_other": 0, 2024-02-18T11:53:40.915 INFO:teuthology.orchestra.run.smithi158.stdout: "linger_active": 0, 2024-02-18T11:53:40.915 INFO:teuthology.orchestra.run.smithi158.stdout: "linger_send": 0, 2024-02-18T11:53:40.916 INFO:teuthology.orchestra.run.smithi158.stdout: "linger_resend": 0, 2024-02-18T11:53:40.916 INFO:teuthology.orchestra.run.smithi158.stdout: "linger_ping": 0, 2024-02-18T11:53:40.916 INFO:teuthology.orchestra.run.smithi158.stdout: "poolop_active": 0, 2024-02-18T11:53:40.916 INFO:teuthology.orchestra.run.smithi158.stdout: "poolop_send": 0, 2024-02-18T11:53:40.916 INFO:teuthology.orchestra.run.smithi158.stdout: "poolop_resend": 0, 2024-02-18T11:53:40.916 INFO:teuthology.orchestra.run.smithi158.stdout: "poolstat_active": 0, 2024-02-18T11:53:40.916 INFO:teuthology.orchestra.run.smithi158.stdout: "poolstat_send": 0, 2024-02-18T11:53:40.916 INFO:teuthology.orchestra.run.smithi158.stdout: "poolstat_resend": 0, 2024-02-18T11:53:40.916 INFO:teuthology.orchestra.run.smithi158.stdout: "statfs_active": 0, 2024-02-18T11:53:40.916 INFO:teuthology.orchestra.run.smithi158.stdout: "statfs_send": 0, 2024-02-18T11:53:40.916 INFO:teuthology.orchestra.run.smithi158.stdout: "statfs_resend": 0, 2024-02-18T11:53:40.916 INFO:teuthology.orchestra.run.smithi158.stdout: "command_active": 0, 2024-02-18T11:53:40.916 INFO:teuthology.orchestra.run.smithi158.stdout: "command_send": 0, 2024-02-18T11:53:40.916 INFO:teuthology.orchestra.run.smithi158.stdout: "command_resend": 0, 2024-02-18T11:53:40.917 INFO:teuthology.orchestra.run.smithi158.stdout: "map_epoch": 89, 2024-02-18T11:53:40.917 INFO:teuthology.orchestra.run.smithi158.stdout: "map_full": 0, 2024-02-18T11:53:40.917 INFO:teuthology.orchestra.run.smithi158.stdout: "map_inc": 3, 2024-02-18T11:53:40.917 INFO:teuthology.orchestra.run.smithi158.stdout: "osd_sessions": 12, 2024-02-18T11:53:40.917 INFO:teuthology.orchestra.run.smithi158.stdout: "osd_session_open": 12, 2024-02-18T11:53:40.917 INFO:teuthology.orchestra.run.smithi158.stdout: "osd_session_close": 0, 2024-02-18T11:53:40.917 INFO:teuthology.orchestra.run.smithi158.stdout: "osd_laggy": 0, 2024-02-18T11:53:40.917 INFO:teuthology.orchestra.run.smithi158.stdout: "omap_wr": 1134, 2024-02-18T11:53:40.917 INFO:teuthology.orchestra.run.smithi158.stdout: "omap_rd": 0, 2024-02-18T11:53:40.917 INFO:teuthology.orchestra.run.smithi158.stdout: "omap_del": 13 2024-02-18T11:53:40.917 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:40.917 INFO:teuthology.orchestra.run.smithi158.stdout: "oft": { 2024-02-18T11:53:40.917 INFO:teuthology.orchestra.run.smithi158.stdout: "omap_total_objs": 1, 2024-02-18T11:53:40.918 INFO:teuthology.orchestra.run.smithi158.stdout: "omap_total_kv_pairs": 0, 2024-02-18T11:53:40.918 INFO:teuthology.orchestra.run.smithi158.stdout: "omap_total_updates": 0, 2024-02-18T11:53:40.918 INFO:teuthology.orchestra.run.smithi158.stdout: "omap_total_removes": 0 2024-02-18T11:53:40.918 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:40.918 INFO:teuthology.orchestra.run.smithi158.stdout: "osd-slow-ops": { 2024-02-18T11:53:40.918 INFO:teuthology.orchestra.run.smithi158.stdout: "slow_ops_count": 2 2024-02-18T11:53:40.918 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:40.918 INFO:teuthology.orchestra.run.smithi158.stdout: "purge_queue": { 2024-02-18T11:53:40.918 INFO:teuthology.orchestra.run.smithi158.stdout: "pq_executing_ops": 0, 2024-02-18T11:53:40.918 INFO:teuthology.orchestra.run.smithi158.stdout: "pq_executing_ops_high_water": 522, 2024-02-18T11:53:40.918 INFO:teuthology.orchestra.run.smithi158.stdout: "pq_executing": 0, 2024-02-18T11:53:40.918 INFO:teuthology.orchestra.run.smithi158.stdout: "pq_executing_high_water": 10, 2024-02-18T11:53:40.918 INFO:teuthology.orchestra.run.smithi158.stdout: "pq_executed_ops": 538, 2024-02-18T11:53:40.918 INFO:teuthology.orchestra.run.smithi158.stdout: "pq_executed": 18, 2024-02-18T11:53:40.919 INFO:teuthology.orchestra.run.smithi158.stdout: "pq_item_in_journal": 0 2024-02-18T11:53:40.919 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:40.919 INFO:teuthology.orchestra.run.smithi158.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-18T11:53:40.919 INFO:teuthology.orchestra.run.smithi158.stdout: "val": 0, 2024-02-18T11:53:40.919 INFO:teuthology.orchestra.run.smithi158.stdout: "max": 104857600, 2024-02-18T11:53:40.919 INFO:teuthology.orchestra.run.smithi158.stdout: "get_started": 0, 2024-02-18T11:53:40.919 INFO:teuthology.orchestra.run.smithi158.stdout: "get": 717747, 2024-02-18T11:53:40.919 INFO:teuthology.orchestra.run.smithi158.stdout: "get_sum": 169480958, 2024-02-18T11:53:40.919 INFO:teuthology.orchestra.run.smithi158.stdout: "get_or_fail_fail": 0, 2024-02-18T11:53:40.919 INFO:teuthology.orchestra.run.smithi158.stdout: "get_or_fail_success": 717747, 2024-02-18T11:53:40.919 INFO:teuthology.orchestra.run.smithi158.stdout: "take": 0, 2024-02-18T11:53:40.919 INFO:teuthology.orchestra.run.smithi158.stdout: "take_sum": 0, 2024-02-18T11:53:40.919 INFO:teuthology.orchestra.run.smithi158.stdout: "put": 717747, 2024-02-18T11:53:40.920 INFO:teuthology.orchestra.run.smithi158.stdout: "put_sum": 169480958, 2024-02-18T11:53:40.920 INFO:teuthology.orchestra.run.smithi158.stdout: "wait": { 2024-02-18T11:53:40.920 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:40.920 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:40.920 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:40.920 INFO:teuthology.orchestra.run.smithi158.stdout: } 2024-02-18T11:53:40.920 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:40.920 INFO:teuthology.orchestra.run.smithi158.stdout: "throttle-objecter_bytes": { 2024-02-18T11:53:40.920 INFO:teuthology.orchestra.run.smithi158.stdout: "val": 0, 2024-02-18T11:53:40.920 INFO:teuthology.orchestra.run.smithi158.stdout: "max": 104857600, 2024-02-18T11:53:40.920 INFO:teuthology.orchestra.run.smithi158.stdout: "get_started": 0, 2024-02-18T11:53:40.920 INFO:teuthology.orchestra.run.smithi158.stdout: "get": 0, 2024-02-18T11:53:40.920 INFO:teuthology.orchestra.run.smithi158.stdout: "get_sum": 0, 2024-02-18T11:53:40.920 INFO:teuthology.orchestra.run.smithi158.stdout: "get_or_fail_fail": 0, 2024-02-18T11:53:40.921 INFO:teuthology.orchestra.run.smithi158.stdout: "get_or_fail_success": 0, 2024-02-18T11:53:40.921 INFO:teuthology.orchestra.run.smithi158.stdout: "take": 9878, 2024-02-18T11:53:40.921 INFO:teuthology.orchestra.run.smithi158.stdout: "take_sum": 1322004973, 2024-02-18T11:53:40.921 INFO:teuthology.orchestra.run.smithi158.stdout: "put": 9878, 2024-02-18T11:53:40.921 INFO:teuthology.orchestra.run.smithi158.stdout: "put_sum": 1322004973, 2024-02-18T11:53:40.921 INFO:teuthology.orchestra.run.smithi158.stdout: "wait": { 2024-02-18T11:53:40.921 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:40.921 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:40.921 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:40.921 INFO:teuthology.orchestra.run.smithi158.stdout: } 2024-02-18T11:53:40.921 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:40.921 INFO:teuthology.orchestra.run.smithi158.stdout: "throttle-objecter_ops": { 2024-02-18T11:53:40.922 INFO:teuthology.orchestra.run.smithi158.stdout: "val": 0, 2024-02-18T11:53:40.922 INFO:teuthology.orchestra.run.smithi158.stdout: "max": 1024, 2024-02-18T11:53:40.922 INFO:teuthology.orchestra.run.smithi158.stdout: "get_started": 0, 2024-02-18T11:53:40.922 INFO:teuthology.orchestra.run.smithi158.stdout: "get": 0, 2024-02-18T11:53:40.922 INFO:teuthology.orchestra.run.smithi158.stdout: "get_sum": 0, 2024-02-18T11:53:40.922 INFO:teuthology.orchestra.run.smithi158.stdout: "get_or_fail_fail": 0, 2024-02-18T11:53:40.922 INFO:teuthology.orchestra.run.smithi158.stdout: "get_or_fail_success": 0, 2024-02-18T11:53:40.922 INFO:teuthology.orchestra.run.smithi158.stdout: "take": 9878, 2024-02-18T11:53:40.922 INFO:teuthology.orchestra.run.smithi158.stdout: "take_sum": 9878, 2024-02-18T11:53:40.922 INFO:teuthology.orchestra.run.smithi158.stdout: "put": 9878, 2024-02-18T11:53:40.922 INFO:teuthology.orchestra.run.smithi158.stdout: "put_sum": 9878, 2024-02-18T11:53:40.922 INFO:teuthology.orchestra.run.smithi158.stdout: "wait": { 2024-02-18T11:53:40.922 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:40.922 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:40.923 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:40.923 INFO:teuthology.orchestra.run.smithi158.stdout: } 2024-02-18T11:53:40.923 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:40.923 INFO:teuthology.orchestra.run.smithi158.stdout: "throttle-write_buf_throttle": { 2024-02-18T11:53:40.923 INFO:teuthology.orchestra.run.smithi158.stdout: "val": 0, 2024-02-18T11:53:40.923 INFO:teuthology.orchestra.run.smithi158.stdout: "max": 3758096384, 2024-02-18T11:53:40.923 INFO:teuthology.orchestra.run.smithi158.stdout: "get_started": 0, 2024-02-18T11:53:40.923 INFO:teuthology.orchestra.run.smithi158.stdout: "get": 18, 2024-02-18T11:53:40.923 INFO:teuthology.orchestra.run.smithi158.stdout: "get_sum": 1826, 2024-02-18T11:53:40.923 INFO:teuthology.orchestra.run.smithi158.stdout: "get_or_fail_fail": 0, 2024-02-18T11:53:40.923 INFO:teuthology.orchestra.run.smithi158.stdout: "get_or_fail_success": 18, 2024-02-18T11:53:40.923 INFO:teuthology.orchestra.run.smithi158.stdout: "take": 0, 2024-02-18T11:53:40.923 INFO:teuthology.orchestra.run.smithi158.stdout: "take_sum": 0, 2024-02-18T11:53:40.924 INFO:teuthology.orchestra.run.smithi158.stdout: "put": 2, 2024-02-18T11:53:40.924 INFO:teuthology.orchestra.run.smithi158.stdout: "put_sum": 1826, 2024-02-18T11:53:40.924 INFO:teuthology.orchestra.run.smithi158.stdout: "wait": { 2024-02-18T11:53:40.924 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:40.924 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:40.924 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:40.924 INFO:teuthology.orchestra.run.smithi158.stdout: } 2024-02-18T11:53:40.924 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:40.924 INFO:teuthology.orchestra.run.smithi158.stdout: "throttle-write_buf_throttle-0x55eabbb8e6e0": { 2024-02-18T11:53:40.924 INFO:teuthology.orchestra.run.smithi158.stdout: "val": 0, 2024-02-18T11:53:40.924 INFO:teuthology.orchestra.run.smithi158.stdout: "max": 3758096384, 2024-02-18T11:53:40.924 INFO:teuthology.orchestra.run.smithi158.stdout: "get_started": 0, 2024-02-18T11:53:40.924 INFO:teuthology.orchestra.run.smithi158.stdout: "get": 707101, 2024-02-18T11:53:40.925 INFO:teuthology.orchestra.run.smithi158.stdout: "get_sum": 1226381676, 2024-02-18T11:53:40.925 INFO:teuthology.orchestra.run.smithi158.stdout: "get_or_fail_fail": 0, 2024-02-18T11:53:40.925 INFO:teuthology.orchestra.run.smithi158.stdout: "get_or_fail_success": 707101, 2024-02-18T11:53:40.925 INFO:teuthology.orchestra.run.smithi158.stdout: "take": 0, 2024-02-18T11:53:40.925 INFO:teuthology.orchestra.run.smithi158.stdout: "take_sum": 0, 2024-02-18T11:53:40.925 INFO:teuthology.orchestra.run.smithi158.stdout: "put": 7069, 2024-02-18T11:53:40.925 INFO:teuthology.orchestra.run.smithi158.stdout: "put_sum": 1226381676, 2024-02-18T11:53:40.925 INFO:teuthology.orchestra.run.smithi158.stdout: "wait": { 2024-02-18T11:53:40.925 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:40.925 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:40.925 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:40.925 INFO:teuthology.orchestra.run.smithi158.stdout: } 2024-02-18T11:53:40.925 INFO:teuthology.orchestra.run.smithi158.stdout: } 2024-02-18T11:53:40.926 INFO:teuthology.orchestra.run.smithi158.stdout:} 2024-02-18T11:53:41.352 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:53:41 smithi188 ceph-mon[36925]: pgmap v733: 97 pgs: 97 active+clean; 8.2 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 938 B/s wr, 27 op/s 2024-02-18T11:53:41.382 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:53:41 smithi158 ceph-mon[36626]: pgmap v733: 97 pgs: 97 active+clean; 8.2 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 938 B/s wr, 27 op/s 2024-02-18T11:53:41.421 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:53:41 smithi012 ceph-mon[32460]: pgmap v733: 97 pgs: 97 active+clean; 8.2 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 938 B/s wr, 27 op/s 2024-02-18T11:53:41.752 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-02-18T11:53:41.752 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-02-18T11:53:41.752 DEBUG:tasks.check_counter:Getting stats from e 2024-02-18T11:53:41.752 DEBUG:teuthology.orchestra.run.smithi158:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph daemon mds.e perf dump 2024-02-18T11:53:42.009 INFO:teuthology.orchestra.run.smithi158.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T11:53:43.333 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:53:43 smithi158 ceph-mon[36626]: pgmap v734: 97 pgs: 97 active+clean; 8.2 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 13 MiB/s wr, 58 op/s 2024-02-18T11:53:43.352 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:53:43 smithi188 ceph-mon[36925]: pgmap v734: 97 pgs: 97 active+clean; 8.2 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 13 MiB/s wr, 58 op/s 2024-02-18T11:53:43.421 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:53:43 smithi012 ceph-mon[32460]: pgmap v734: 97 pgs: 97 active+clean; 8.2 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 13 MiB/s wr, 58 op/s 2024-02-18T11:53:43.931 INFO:teuthology.orchestra.run.smithi158.stdout:{ 2024-02-18T11:53:43.931 INFO:teuthology.orchestra.run.smithi158.stdout: "AsyncMessenger::Worker-0": { 2024-02-18T11:53:43.931 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_recv_messages": 864, 2024-02-18T11:53:43.931 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_messages": 864, 2024-02-18T11:53:43.931 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_recv_bytes": 230347, 2024-02-18T11:53:43.931 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_bytes": 267972, 2024-02-18T11:53:43.931 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_created_connections": 6, 2024-02-18T11:53:43.931 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_active_connections": 2, 2024-02-18T11:53:43.931 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_total_time": 0.090310462, 2024-02-18T11:53:43.931 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_send_time": 0.030026861, 2024-02-18T11:53:43.931 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_recv_time": 0.022763285, 2024-02-18T11:53:43.932 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_fast_dispatch_time": 0.028265798, 2024-02-18T11:53:43.932 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_messages_queue_lat": { 2024-02-18T11:53:43.932 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 863, 2024-02-18T11:53:43.932 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.043399473, 2024-02-18T11:53:43.932 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000050289 2024-02-18T11:53:43.932 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:43.932 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_handle_ack_lat": { 2024-02-18T11:53:43.932 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:43.932 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:43.932 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:43.932 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:43.932 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-02-18T11:53:43.932 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_encrypted_bytes": 448 2024-02-18T11:53:43.933 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:43.933 INFO:teuthology.orchestra.run.smithi158.stdout: "AsyncMessenger::Worker-1": { 2024-02-18T11:53:43.933 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_recv_messages": 747, 2024-02-18T11:53:43.933 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_messages": 934, 2024-02-18T11:53:43.933 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_recv_bytes": 183698, 2024-02-18T11:53:43.933 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_bytes": 637963, 2024-02-18T11:53:43.933 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_created_connections": 5, 2024-02-18T11:53:43.933 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_active_connections": 4, 2024-02-18T11:53:43.933 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_total_time": 0.100632002, 2024-02-18T11:53:43.933 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_send_time": 0.025768883, 2024-02-18T11:53:43.933 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_recv_time": 0.022978561, 2024-02-18T11:53:43.933 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_fast_dispatch_time": 0.041529201, 2024-02-18T11:53:43.933 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_messages_queue_lat": { 2024-02-18T11:53:43.933 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 932, 2024-02-18T11:53:43.933 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.050689020, 2024-02-18T11:53:43.934 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000054387 2024-02-18T11:53:43.934 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:43.934 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_handle_ack_lat": { 2024-02-18T11:53:43.934 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:43.934 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:43.934 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:43.934 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:43.934 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_recv_encrypted_bytes": 160, 2024-02-18T11:53:43.934 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_encrypted_bytes": 405952 2024-02-18T11:53:43.934 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:43.934 INFO:teuthology.orchestra.run.smithi158.stdout: "AsyncMessenger::Worker-2": { 2024-02-18T11:53:43.934 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_recv_messages": 1113, 2024-02-18T11:53:43.934 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_messages": 1107, 2024-02-18T11:53:43.934 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_recv_bytes": 1064816, 2024-02-18T11:53:43.934 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_bytes": 414835, 2024-02-18T11:53:43.935 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_created_connections": 3, 2024-02-18T11:53:43.935 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_active_connections": 2, 2024-02-18T11:53:43.935 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_total_time": 0.152070660, 2024-02-18T11:53:43.935 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_send_time": 0.043994110, 2024-02-18T11:53:43.935 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_recv_time": 0.053136298, 2024-02-18T11:53:43.935 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_fast_dispatch_time": 0.048610927, 2024-02-18T11:53:43.935 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_messages_queue_lat": { 2024-02-18T11:53:43.935 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 1106, 2024-02-18T11:53:43.935 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.075720047, 2024-02-18T11:53:43.935 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000068462 2024-02-18T11:53:43.935 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:43.935 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_handle_ack_lat": { 2024-02-18T11:53:43.935 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:43.935 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:43.936 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:43.936 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:43.936 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_recv_encrypted_bytes": 778208, 2024-02-18T11:53:43.936 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_encrypted_bytes": 149552 2024-02-18T11:53:43.936 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:43.936 INFO:teuthology.orchestra.run.smithi158.stdout: "cct": { 2024-02-18T11:53:43.936 INFO:teuthology.orchestra.run.smithi158.stdout: "total_workers": 1, 2024-02-18T11:53:43.936 INFO:teuthology.orchestra.run.smithi158.stdout: "unhealthy_workers": 0 2024-02-18T11:53:43.936 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:43.936 INFO:teuthology.orchestra.run.smithi158.stdout: "finisher-MDSRank": { 2024-02-18T11:53:43.936 INFO:teuthology.orchestra.run.smithi158.stdout: "queue_len": 0, 2024-02-18T11:53:43.936 INFO:teuthology.orchestra.run.smithi158.stdout: "complete_latency": { 2024-02-18T11:53:43.936 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 4126, 2024-02-18T11:53:43.936 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.157021628, 2024-02-18T11:53:43.937 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000038056 2024-02-18T11:53:43.937 INFO:teuthology.orchestra.run.smithi158.stdout: } 2024-02-18T11:53:43.937 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:43.937 INFO:teuthology.orchestra.run.smithi158.stdout: "finisher-PurgeQueue": { 2024-02-18T11:53:43.937 INFO:teuthology.orchestra.run.smithi158.stdout: "queue_len": 0, 2024-02-18T11:53:43.937 INFO:teuthology.orchestra.run.smithi158.stdout: "complete_latency": { 2024-02-18T11:53:43.937 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:43.937 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:43.937 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:43.937 INFO:teuthology.orchestra.run.smithi158.stdout: } 2024-02-18T11:53:43.937 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:43.937 INFO:teuthology.orchestra.run.smithi158.stdout: "mds": { 2024-02-18T11:53:43.937 INFO:teuthology.orchestra.run.smithi158.stdout: "request": 0, 2024-02-18T11:53:43.937 INFO:teuthology.orchestra.run.smithi158.stdout: "reply": 0, 2024-02-18T11:53:43.938 INFO:teuthology.orchestra.run.smithi158.stdout: "reply_latency": { 2024-02-18T11:53:43.938 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:43.938 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:43.938 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:43.938 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:43.938 INFO:teuthology.orchestra.run.smithi158.stdout: "slow_reply": 0, 2024-02-18T11:53:43.938 INFO:teuthology.orchestra.run.smithi158.stdout: "forward": 0, 2024-02-18T11:53:43.938 INFO:teuthology.orchestra.run.smithi158.stdout: "dir_fetch_complete": 0, 2024-02-18T11:53:43.938 INFO:teuthology.orchestra.run.smithi158.stdout: "dir_fetch_keys": 0, 2024-02-18T11:53:43.938 INFO:teuthology.orchestra.run.smithi158.stdout: "dir_commit": 0, 2024-02-18T11:53:43.938 INFO:teuthology.orchestra.run.smithi158.stdout: "dir_split": 0, 2024-02-18T11:53:43.938 INFO:teuthology.orchestra.run.smithi158.stdout: "dir_merge": 0, 2024-02-18T11:53:43.938 INFO:teuthology.orchestra.run.smithi158.stdout: "inodes": 0, 2024-02-18T11:53:43.938 INFO:teuthology.orchestra.run.smithi158.stdout: "inodes_top": 0, 2024-02-18T11:53:43.938 INFO:teuthology.orchestra.run.smithi158.stdout: "inodes_bottom": 0, 2024-02-18T11:53:43.939 INFO:teuthology.orchestra.run.smithi158.stdout: "inodes_pin_tail": 0, 2024-02-18T11:53:43.939 INFO:teuthology.orchestra.run.smithi158.stdout: "inodes_pinned": 0, 2024-02-18T11:53:43.939 INFO:teuthology.orchestra.run.smithi158.stdout: "inodes_expired": 0, 2024-02-18T11:53:43.939 INFO:teuthology.orchestra.run.smithi158.stdout: "inodes_with_caps": 0, 2024-02-18T11:53:43.939 INFO:teuthology.orchestra.run.smithi158.stdout: "caps": 0, 2024-02-18T11:53:43.939 INFO:teuthology.orchestra.run.smithi158.stdout: "subtrees": 1, 2024-02-18T11:53:43.939 INFO:teuthology.orchestra.run.smithi158.stdout: "traverse": 0, 2024-02-18T11:53:43.939 INFO:teuthology.orchestra.run.smithi158.stdout: "traverse_hit": 0, 2024-02-18T11:53:43.939 INFO:teuthology.orchestra.run.smithi158.stdout: "traverse_forward": 0, 2024-02-18T11:53:43.939 INFO:teuthology.orchestra.run.smithi158.stdout: "traverse_discover": 0, 2024-02-18T11:53:43.939 INFO:teuthology.orchestra.run.smithi158.stdout: "traverse_dir_fetch": 0, 2024-02-18T11:53:43.939 INFO:teuthology.orchestra.run.smithi158.stdout: "traverse_remote_ino": 0, 2024-02-18T11:53:43.939 INFO:teuthology.orchestra.run.smithi158.stdout: "traverse_lock": 0, 2024-02-18T11:53:43.939 INFO:teuthology.orchestra.run.smithi158.stdout: "load_cent": 0, 2024-02-18T11:53:43.940 INFO:teuthology.orchestra.run.smithi158.stdout: "q": 0, 2024-02-18T11:53:43.940 INFO:teuthology.orchestra.run.smithi158.stdout: "exported": 0, 2024-02-18T11:53:43.940 INFO:teuthology.orchestra.run.smithi158.stdout: "exported_inodes": 0, 2024-02-18T11:53:43.940 INFO:teuthology.orchestra.run.smithi158.stdout: "imported": 0, 2024-02-18T11:53:43.940 INFO:teuthology.orchestra.run.smithi158.stdout: "imported_inodes": 0, 2024-02-18T11:53:43.940 INFO:teuthology.orchestra.run.smithi158.stdout: "openino_dir_fetch": 0, 2024-02-18T11:53:43.940 INFO:teuthology.orchestra.run.smithi158.stdout: "openino_backtrace_fetch": 0, 2024-02-18T11:53:43.940 INFO:teuthology.orchestra.run.smithi158.stdout: "openino_peer_discover": 0, 2024-02-18T11:53:43.940 INFO:teuthology.orchestra.run.smithi158.stdout: "root_rfiles": 0, 2024-02-18T11:53:43.940 INFO:teuthology.orchestra.run.smithi158.stdout: "root_rbytes": 0, 2024-02-18T11:53:43.940 INFO:teuthology.orchestra.run.smithi158.stdout: "root_rsnaps": 0, 2024-02-18T11:53:43.940 INFO:teuthology.orchestra.run.smithi158.stdout: "scrub_backtrace_fetch": 0, 2024-02-18T11:53:43.940 INFO:teuthology.orchestra.run.smithi158.stdout: "scrub_set_tag": 0, 2024-02-18T11:53:43.940 INFO:teuthology.orchestra.run.smithi158.stdout: "scrub_backtrace_repaired": 0, 2024-02-18T11:53:43.941 INFO:teuthology.orchestra.run.smithi158.stdout: "scrub_inotable_repaired": 0, 2024-02-18T11:53:43.941 INFO:teuthology.orchestra.run.smithi158.stdout: "scrub_dir_inodes": 0, 2024-02-18T11:53:43.941 INFO:teuthology.orchestra.run.smithi158.stdout: "scrub_dir_base_inodes": 0, 2024-02-18T11:53:43.941 INFO:teuthology.orchestra.run.smithi158.stdout: "scrub_dirfrag_rstats": 0, 2024-02-18T11:53:43.941 INFO:teuthology.orchestra.run.smithi158.stdout: "scrub_file_inodes": 0, 2024-02-18T11:53:43.941 INFO:teuthology.orchestra.run.smithi158.stdout: "handle_inode_file_caps": 0, 2024-02-18T11:53:43.941 INFO:teuthology.orchestra.run.smithi158.stdout: "ceph_cap_op_revoke": 0, 2024-02-18T11:53:43.941 INFO:teuthology.orchestra.run.smithi158.stdout: "ceph_cap_op_grant": 0, 2024-02-18T11:53:43.941 INFO:teuthology.orchestra.run.smithi158.stdout: "ceph_cap_op_trunc": 0, 2024-02-18T11:53:43.941 INFO:teuthology.orchestra.run.smithi158.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-02-18T11:53:43.941 INFO:teuthology.orchestra.run.smithi158.stdout: "ceph_cap_op_flush_ack": 0, 2024-02-18T11:53:43.941 INFO:teuthology.orchestra.run.smithi158.stdout: "handle_client_caps": 0, 2024-02-18T11:53:43.941 INFO:teuthology.orchestra.run.smithi158.stdout: "handle_client_caps_dirty": 0, 2024-02-18T11:53:43.941 INFO:teuthology.orchestra.run.smithi158.stdout: "handle_client_cap_release": 0, 2024-02-18T11:53:43.941 INFO:teuthology.orchestra.run.smithi158.stdout: "process_request_cap_release": 0 2024-02-18T11:53:43.942 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:43.942 INFO:teuthology.orchestra.run.smithi158.stdout: "mds_cache": { 2024-02-18T11:53:43.942 INFO:teuthology.orchestra.run.smithi158.stdout: "dir_update": 0, 2024-02-18T11:53:43.942 INFO:teuthology.orchestra.run.smithi158.stdout: "dir_update_receipt": 0, 2024-02-18T11:53:43.942 INFO:teuthology.orchestra.run.smithi158.stdout: "dir_try_discover": 0, 2024-02-18T11:53:43.942 INFO:teuthology.orchestra.run.smithi158.stdout: "dir_send_discover": 0, 2024-02-18T11:53:43.942 INFO:teuthology.orchestra.run.smithi158.stdout: "dir_handle_discover": 0, 2024-02-18T11:53:43.942 INFO:teuthology.orchestra.run.smithi158.stdout: "num_strays": 0, 2024-02-18T11:53:43.942 INFO:teuthology.orchestra.run.smithi158.stdout: "num_strays_delayed": 0, 2024-02-18T11:53:43.942 INFO:teuthology.orchestra.run.smithi158.stdout: "num_strays_enqueuing": 0, 2024-02-18T11:53:43.942 INFO:teuthology.orchestra.run.smithi158.stdout: "strays_created": 0, 2024-02-18T11:53:43.942 INFO:teuthology.orchestra.run.smithi158.stdout: "strays_enqueued": 0, 2024-02-18T11:53:43.942 INFO:teuthology.orchestra.run.smithi158.stdout: "strays_reintegrated": 0, 2024-02-18T11:53:43.942 INFO:teuthology.orchestra.run.smithi158.stdout: "strays_migrated": 0, 2024-02-18T11:53:43.943 INFO:teuthology.orchestra.run.smithi158.stdout: "num_recovering_processing": 0, 2024-02-18T11:53:43.943 INFO:teuthology.orchestra.run.smithi158.stdout: "num_recovering_enqueued": 0, 2024-02-18T11:53:43.943 INFO:teuthology.orchestra.run.smithi158.stdout: "num_recovering_prioritized": 0, 2024-02-18T11:53:43.943 INFO:teuthology.orchestra.run.smithi158.stdout: "recovery_started": 0, 2024-02-18T11:53:43.943 INFO:teuthology.orchestra.run.smithi158.stdout: "recovery_completed": 0, 2024-02-18T11:53:43.943 INFO:teuthology.orchestra.run.smithi158.stdout: "ireq_enqueue_scrub": 0, 2024-02-18T11:53:43.943 INFO:teuthology.orchestra.run.smithi158.stdout: "ireq_exportdir": 0, 2024-02-18T11:53:43.943 INFO:teuthology.orchestra.run.smithi158.stdout: "ireq_flush": 0, 2024-02-18T11:53:43.943 INFO:teuthology.orchestra.run.smithi158.stdout: "ireq_fragmentdir": 0, 2024-02-18T11:53:43.943 INFO:teuthology.orchestra.run.smithi158.stdout: "ireq_fragstats": 0, 2024-02-18T11:53:43.943 INFO:teuthology.orchestra.run.smithi158.stdout: "ireq_inodestats": 0 2024-02-18T11:53:43.943 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:43.943 INFO:teuthology.orchestra.run.smithi158.stdout: "mds_log": { 2024-02-18T11:53:43.943 INFO:teuthology.orchestra.run.smithi158.stdout: "evlrg": 0, 2024-02-18T11:53:43.943 INFO:teuthology.orchestra.run.smithi158.stdout: "evadd": 0, 2024-02-18T11:53:43.944 INFO:teuthology.orchestra.run.smithi158.stdout: "evex": 0, 2024-02-18T11:53:43.944 INFO:teuthology.orchestra.run.smithi158.stdout: "evtrm": 0, 2024-02-18T11:53:43.944 INFO:teuthology.orchestra.run.smithi158.stdout: "ev": 5, 2024-02-18T11:53:43.944 INFO:teuthology.orchestra.run.smithi158.stdout: "evexg": 0, 2024-02-18T11:53:43.944 INFO:teuthology.orchestra.run.smithi158.stdout: "evexd": 0, 2024-02-18T11:53:43.944 INFO:teuthology.orchestra.run.smithi158.stdout: "segadd": 0, 2024-02-18T11:53:43.944 INFO:teuthology.orchestra.run.smithi158.stdout: "segex": 0, 2024-02-18T11:53:43.944 INFO:teuthology.orchestra.run.smithi158.stdout: "segtrm": 0, 2024-02-18T11:53:43.944 INFO:teuthology.orchestra.run.smithi158.stdout: "seg": 2, 2024-02-18T11:53:43.944 INFO:teuthology.orchestra.run.smithi158.stdout: "segmjr": 2, 2024-02-18T11:53:43.944 INFO:teuthology.orchestra.run.smithi158.stdout: "segexg": 0, 2024-02-18T11:53:43.944 INFO:teuthology.orchestra.run.smithi158.stdout: "segexd": 0, 2024-02-18T11:53:43.944 INFO:teuthology.orchestra.run.smithi158.stdout: "expos": 4194304, 2024-02-18T11:53:43.944 INFO:teuthology.orchestra.run.smithi158.stdout: "wrpos": 4195732, 2024-02-18T11:53:43.944 INFO:teuthology.orchestra.run.smithi158.stdout: "rdpos": 4195591, 2024-02-18T11:53:43.945 INFO:teuthology.orchestra.run.smithi158.stdout: "jlat": { 2024-02-18T11:53:43.945 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:43.945 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:43.945 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:43.945 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:43.945 INFO:teuthology.orchestra.run.smithi158.stdout: "replayed": 5 2024-02-18T11:53:43.945 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:43.945 INFO:teuthology.orchestra.run.smithi158.stdout: "mds_mem": { 2024-02-18T11:53:43.945 INFO:teuthology.orchestra.run.smithi158.stdout: "ino": 3, 2024-02-18T11:53:43.945 INFO:teuthology.orchestra.run.smithi158.stdout: "ino+": 3, 2024-02-18T11:53:43.945 INFO:teuthology.orchestra.run.smithi158.stdout: "ino-": 0, 2024-02-18T11:53:43.945 INFO:teuthology.orchestra.run.smithi158.stdout: "dir": 1, 2024-02-18T11:53:43.945 INFO:teuthology.orchestra.run.smithi158.stdout: "dir+": 1, 2024-02-18T11:53:43.945 INFO:teuthology.orchestra.run.smithi158.stdout: "dir-": 0, 2024-02-18T11:53:43.946 INFO:teuthology.orchestra.run.smithi158.stdout: "dn": 0, 2024-02-18T11:53:43.946 INFO:teuthology.orchestra.run.smithi158.stdout: "dn+": 0, 2024-02-18T11:53:43.946 INFO:teuthology.orchestra.run.smithi158.stdout: "dn-": 0, 2024-02-18T11:53:43.946 INFO:teuthology.orchestra.run.smithi158.stdout: "cap": 0, 2024-02-18T11:53:43.946 INFO:teuthology.orchestra.run.smithi158.stdout: "cap+": 0, 2024-02-18T11:53:43.946 INFO:teuthology.orchestra.run.smithi158.stdout: "cap-": 0, 2024-02-18T11:53:43.946 INFO:teuthology.orchestra.run.smithi158.stdout: "rss": 46824, 2024-02-18T11:53:43.946 INFO:teuthology.orchestra.run.smithi158.stdout: "heap": 215420 2024-02-18T11:53:43.946 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:43.946 INFO:teuthology.orchestra.run.smithi158.stdout: "mds_server": { 2024-02-18T11:53:43.946 INFO:teuthology.orchestra.run.smithi158.stdout: "dispatch_client_request": 0, 2024-02-18T11:53:43.946 INFO:teuthology.orchestra.run.smithi158.stdout: "dispatch_server_request": 0, 2024-02-18T11:53:43.946 INFO:teuthology.orchestra.run.smithi158.stdout: "handle_client_request": 0, 2024-02-18T11:53:43.946 INFO:teuthology.orchestra.run.smithi158.stdout: "handle_client_session": 0, 2024-02-18T11:53:43.946 INFO:teuthology.orchestra.run.smithi158.stdout: "handle_peer_request": 0, 2024-02-18T11:53:43.947 INFO:teuthology.orchestra.run.smithi158.stdout: "req_create_latency": { 2024-02-18T11:53:43.947 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:43.947 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:43.947 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:43.947 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:43.947 INFO:teuthology.orchestra.run.smithi158.stdout: "req_getattr_latency": { 2024-02-18T11:53:43.947 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:43.947 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:43.947 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:43.947 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:43.947 INFO:teuthology.orchestra.run.smithi158.stdout: "req_getfilelock_latency": { 2024-02-18T11:53:43.947 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:43.947 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:43.947 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:43.947 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:43.948 INFO:teuthology.orchestra.run.smithi158.stdout: "req_link_latency": { 2024-02-18T11:53:43.948 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:43.948 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:43.948 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:43.948 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:43.948 INFO:teuthology.orchestra.run.smithi158.stdout: "req_lookup_latency": { 2024-02-18T11:53:43.948 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:43.948 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:43.948 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:43.948 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:43.948 INFO:teuthology.orchestra.run.smithi158.stdout: "req_lookuphash_latency": { 2024-02-18T11:53:43.948 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:43.948 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:43.948 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:43.949 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:43.949 INFO:teuthology.orchestra.run.smithi158.stdout: "req_lookupino_latency": { 2024-02-18T11:53:43.949 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:43.949 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:43.949 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:43.949 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:43.949 INFO:teuthology.orchestra.run.smithi158.stdout: "req_lookupname_latency": { 2024-02-18T11:53:43.949 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:43.949 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:43.949 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:43.949 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:43.949 INFO:teuthology.orchestra.run.smithi158.stdout: "req_lookupparent_latency": { 2024-02-18T11:53:43.949 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:43.949 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:43.949 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:43.950 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:43.950 INFO:teuthology.orchestra.run.smithi158.stdout: "req_lookupsnap_latency": { 2024-02-18T11:53:43.950 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:43.950 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:43.950 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:43.950 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:43.950 INFO:teuthology.orchestra.run.smithi158.stdout: "req_lssnap_latency": { 2024-02-18T11:53:43.950 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:43.950 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:43.950 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:43.950 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:43.950 INFO:teuthology.orchestra.run.smithi158.stdout: "req_mkdir_latency": { 2024-02-18T11:53:43.950 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:43.950 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:43.950 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:43.950 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:43.951 INFO:teuthology.orchestra.run.smithi158.stdout: "req_mknod_latency": { 2024-02-18T11:53:43.951 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:43.951 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:43.951 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:43.951 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:43.952 INFO:teuthology.orchestra.run.smithi158.stdout: "req_mksnap_latency": { 2024-02-18T11:53:43.952 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:43.952 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:43.952 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:43.952 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:43.952 INFO:teuthology.orchestra.run.smithi158.stdout: "req_open_latency": { 2024-02-18T11:53:43.952 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:43.952 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:43.952 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:43.952 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:43.952 INFO:teuthology.orchestra.run.smithi158.stdout: "req_readdir_latency": { 2024-02-18T11:53:43.952 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:43.952 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:43.952 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:43.953 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:43.953 INFO:teuthology.orchestra.run.smithi158.stdout: "req_rename_latency": { 2024-02-18T11:53:43.953 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:43.953 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:43.953 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:43.953 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:43.953 INFO:teuthology.orchestra.run.smithi158.stdout: "req_renamesnap_latency": { 2024-02-18T11:53:43.953 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:43.953 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:43.953 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:43.953 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:43.953 INFO:teuthology.orchestra.run.smithi158.stdout: "req_snapdiff_latency": { 2024-02-18T11:53:43.953 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:43.953 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:43.953 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:43.954 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:43.954 INFO:teuthology.orchestra.run.smithi158.stdout: "req_rmdir_latency": { 2024-02-18T11:53:43.954 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:43.954 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:43.954 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:43.954 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:43.954 INFO:teuthology.orchestra.run.smithi158.stdout: "req_rmsnap_latency": { 2024-02-18T11:53:43.954 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:43.954 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:43.954 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:43.954 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:43.954 INFO:teuthology.orchestra.run.smithi158.stdout: "req_rmxattr_latency": { 2024-02-18T11:53:43.954 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:43.954 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:43.954 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:43.955 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:43.955 INFO:teuthology.orchestra.run.smithi158.stdout: "req_setattr_latency": { 2024-02-18T11:53:43.955 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:43.955 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:43.955 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:43.955 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:43.955 INFO:teuthology.orchestra.run.smithi158.stdout: "req_setdirlayout_latency": { 2024-02-18T11:53:43.955 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:43.955 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:43.955 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:43.955 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:43.955 INFO:teuthology.orchestra.run.smithi158.stdout: "req_setfilelock_latency": { 2024-02-18T11:53:43.955 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:43.955 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:43.955 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:43.956 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:43.956 INFO:teuthology.orchestra.run.smithi158.stdout: "req_setlayout_latency": { 2024-02-18T11:53:43.956 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:43.956 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:43.956 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:43.956 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:43.956 INFO:teuthology.orchestra.run.smithi158.stdout: "req_setxattr_latency": { 2024-02-18T11:53:43.956 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:43.956 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:43.956 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:43.956 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:43.956 INFO:teuthology.orchestra.run.smithi158.stdout: "req_symlink_latency": { 2024-02-18T11:53:43.956 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:43.956 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:43.957 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:43.957 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:43.957 INFO:teuthology.orchestra.run.smithi158.stdout: "req_unlink_latency": { 2024-02-18T11:53:43.957 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:43.957 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:43.957 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:43.957 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:43.957 INFO:teuthology.orchestra.run.smithi158.stdout: "cap_revoke_eviction": 0, 2024-02-18T11:53:43.957 INFO:teuthology.orchestra.run.smithi158.stdout: "cap_acquisition_throttle": 0, 2024-02-18T11:53:43.957 INFO:teuthology.orchestra.run.smithi158.stdout: "req_getvxattr_latency": { 2024-02-18T11:53:43.957 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:43.957 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:43.957 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:43.957 INFO:teuthology.orchestra.run.smithi158.stdout: } 2024-02-18T11:53:43.957 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:43.958 INFO:teuthology.orchestra.run.smithi158.stdout: "mds_sessions": { 2024-02-18T11:53:43.958 INFO:teuthology.orchestra.run.smithi158.stdout: "session_count": 1, 2024-02-18T11:53:43.958 INFO:teuthology.orchestra.run.smithi158.stdout: "session_add": 2, 2024-02-18T11:53:43.958 INFO:teuthology.orchestra.run.smithi158.stdout: "session_remove": 1, 2024-02-18T11:53:43.958 INFO:teuthology.orchestra.run.smithi158.stdout: "sessions_open": 2, 2024-02-18T11:53:43.958 INFO:teuthology.orchestra.run.smithi158.stdout: "sessions_stale": 0, 2024-02-18T11:53:43.958 INFO:teuthology.orchestra.run.smithi158.stdout: "total_load": 0, 2024-02-18T11:53:43.958 INFO:teuthology.orchestra.run.smithi158.stdout: "average_load": 0, 2024-02-18T11:53:43.958 INFO:teuthology.orchestra.run.smithi158.stdout: "avg_session_uptime": 3307, 2024-02-18T11:53:43.958 INFO:teuthology.orchestra.run.smithi158.stdout: "mdthresh_evicted": 0 2024-02-18T11:53:43.958 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:43.958 INFO:teuthology.orchestra.run.smithi158.stdout: "mempool": { 2024-02-18T11:53:43.958 INFO:teuthology.orchestra.run.smithi158.stdout: "bloom_filter_bytes": 0, 2024-02-18T11:53:43.958 INFO:teuthology.orchestra.run.smithi158.stdout: "bloom_filter_items": 0, 2024-02-18T11:53:43.959 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_alloc_bytes": 0, 2024-02-18T11:53:43.959 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_alloc_items": 0, 2024-02-18T11:53:43.959 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_data_bytes": 0, 2024-02-18T11:53:43.959 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_data_items": 0, 2024-02-18T11:53:43.959 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-18T11:53:43.959 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_onode_items": 0, 2024-02-18T11:53:43.959 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-18T11:53:43.959 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_meta_items": 0, 2024-02-18T11:53:43.959 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_other_bytes": 0, 2024-02-18T11:53:43.959 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_other_items": 0, 2024-02-18T11:53:43.959 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-18T11:53:43.959 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_buffer_items": 0, 2024-02-18T11:53:43.959 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_extent_bytes": 0, 2024-02-18T11:53:43.959 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_extent_items": 0, 2024-02-18T11:53:43.959 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_blob_bytes": 0, 2024-02-18T11:53:43.960 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_blob_items": 0, 2024-02-18T11:53:43.960 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-18T11:53:43.960 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_shared_blob_items": 0, 2024-02-18T11:53:43.960 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-18T11:53:43.960 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_inline_bl_items": 0, 2024-02-18T11:53:43.960 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_fsck_bytes": 0, 2024-02-18T11:53:43.960 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_fsck_items": 0, 2024-02-18T11:53:43.960 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_txc_bytes": 0, 2024-02-18T11:53:43.960 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_txc_items": 0, 2024-02-18T11:53:43.960 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-18T11:53:43.960 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_writing_deferred_items": 0, 2024-02-18T11:53:43.960 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_writing_bytes": 0, 2024-02-18T11:53:43.960 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_writing_items": 0, 2024-02-18T11:53:43.960 INFO:teuthology.orchestra.run.smithi158.stdout: "bluefs_bytes": 0, 2024-02-18T11:53:43.960 INFO:teuthology.orchestra.run.smithi158.stdout: "bluefs_items": 0, 2024-02-18T11:53:43.961 INFO:teuthology.orchestra.run.smithi158.stdout: "bluefs_file_reader_bytes": 0, 2024-02-18T11:53:43.961 INFO:teuthology.orchestra.run.smithi158.stdout: "bluefs_file_reader_items": 0, 2024-02-18T11:53:43.961 INFO:teuthology.orchestra.run.smithi158.stdout: "bluefs_file_writer_bytes": 0, 2024-02-18T11:53:43.961 INFO:teuthology.orchestra.run.smithi158.stdout: "bluefs_file_writer_items": 0, 2024-02-18T11:53:43.961 INFO:teuthology.orchestra.run.smithi158.stdout: "buffer_anon_bytes": 189383, 2024-02-18T11:53:43.961 INFO:teuthology.orchestra.run.smithi158.stdout: "buffer_anon_items": 50, 2024-02-18T11:53:43.961 INFO:teuthology.orchestra.run.smithi158.stdout: "buffer_meta_bytes": 0, 2024-02-18T11:53:43.961 INFO:teuthology.orchestra.run.smithi158.stdout: "buffer_meta_items": 0, 2024-02-18T11:53:43.961 INFO:teuthology.orchestra.run.smithi158.stdout: "osd_bytes": 0, 2024-02-18T11:53:43.961 INFO:teuthology.orchestra.run.smithi158.stdout: "osd_items": 0, 2024-02-18T11:53:43.961 INFO:teuthology.orchestra.run.smithi158.stdout: "osd_mapbl_bytes": 0, 2024-02-18T11:53:43.961 INFO:teuthology.orchestra.run.smithi158.stdout: "osd_mapbl_items": 0, 2024-02-18T11:53:43.961 INFO:teuthology.orchestra.run.smithi158.stdout: "osd_pglog_bytes": 0, 2024-02-18T11:53:43.961 INFO:teuthology.orchestra.run.smithi158.stdout: "osd_pglog_items": 0, 2024-02-18T11:53:43.961 INFO:teuthology.orchestra.run.smithi158.stdout: "osdmap_bytes": 7528, 2024-02-18T11:53:43.962 INFO:teuthology.orchestra.run.smithi158.stdout: "osdmap_items": 158, 2024-02-18T11:53:43.962 INFO:teuthology.orchestra.run.smithi158.stdout: "osdmap_mapping_bytes": 0, 2024-02-18T11:53:43.962 INFO:teuthology.orchestra.run.smithi158.stdout: "osdmap_mapping_items": 0, 2024-02-18T11:53:43.962 INFO:teuthology.orchestra.run.smithi158.stdout: "pgmap_bytes": 0, 2024-02-18T11:53:43.962 INFO:teuthology.orchestra.run.smithi158.stdout: "pgmap_items": 0, 2024-02-18T11:53:43.962 INFO:teuthology.orchestra.run.smithi158.stdout: "mds_co_bytes": 8320, 2024-02-18T11:53:43.962 INFO:teuthology.orchestra.run.smithi158.stdout: "mds_co_items": 13, 2024-02-18T11:53:43.962 INFO:teuthology.orchestra.run.smithi158.stdout: "unittest_1_bytes": 0, 2024-02-18T11:53:43.962 INFO:teuthology.orchestra.run.smithi158.stdout: "unittest_1_items": 0, 2024-02-18T11:53:43.962 INFO:teuthology.orchestra.run.smithi158.stdout: "unittest_2_bytes": 0, 2024-02-18T11:53:43.962 INFO:teuthology.orchestra.run.smithi158.stdout: "unittest_2_items": 0 2024-02-18T11:53:43.962 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:43.962 INFO:teuthology.orchestra.run.smithi158.stdout: "objecter": { 2024-02-18T11:53:43.962 INFO:teuthology.orchestra.run.smithi158.stdout: "op_active": 0, 2024-02-18T11:53:43.962 INFO:teuthology.orchestra.run.smithi158.stdout: "op_laggy": 0, 2024-02-18T11:53:43.963 INFO:teuthology.orchestra.run.smithi158.stdout: "op_send": 2459, 2024-02-18T11:53:43.963 INFO:teuthology.orchestra.run.smithi158.stdout: "op_send_bytes": 21, 2024-02-18T11:53:43.963 INFO:teuthology.orchestra.run.smithi158.stdout: "op_resend": 3, 2024-02-18T11:53:43.963 INFO:teuthology.orchestra.run.smithi158.stdout: "op_reply": 2459, 2024-02-18T11:53:43.963 INFO:teuthology.orchestra.run.smithi158.stdout: "op_latency": { 2024-02-18T11:53:43.963 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 2459, 2024-02-18T11:53:43.963 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 40.539565607, 2024-02-18T11:53:43.963 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.016486199 2024-02-18T11:53:43.963 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:43.963 INFO:teuthology.orchestra.run.smithi158.stdout: "op_inflight": 0, 2024-02-18T11:53:43.963 INFO:teuthology.orchestra.run.smithi158.stdout: "oplen_avg": { 2024-02-18T11:53:43.963 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 2459, 2024-02-18T11:53:43.963 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 2460 2024-02-18T11:53:43.963 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:43.963 INFO:teuthology.orchestra.run.smithi158.stdout: "op": 2459, 2024-02-18T11:53:43.963 INFO:teuthology.orchestra.run.smithi158.stdout: "op_r": 2459, 2024-02-18T11:53:43.964 INFO:teuthology.orchestra.run.smithi158.stdout: "op_w": 0, 2024-02-18T11:53:43.964 INFO:teuthology.orchestra.run.smithi158.stdout: "op_rmw": 0, 2024-02-18T11:53:43.964 INFO:teuthology.orchestra.run.smithi158.stdout: "op_pg": 0, 2024-02-18T11:53:43.964 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_stat": 1597, 2024-02-18T11:53:43.964 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_create": 0, 2024-02-18T11:53:43.964 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_read": 860, 2024-02-18T11:53:43.964 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_write": 0, 2024-02-18T11:53:43.964 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_writefull": 0, 2024-02-18T11:53:43.964 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_writesame": 0, 2024-02-18T11:53:43.964 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_append": 0, 2024-02-18T11:53:43.964 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_zero": 0, 2024-02-18T11:53:43.964 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_truncate": 0, 2024-02-18T11:53:43.964 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_delete": 0, 2024-02-18T11:53:43.964 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_mapext": 0, 2024-02-18T11:53:43.965 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_sparse_read": 0, 2024-02-18T11:53:43.965 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_clonerange": 0, 2024-02-18T11:53:43.965 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_getxattr": 1, 2024-02-18T11:53:43.965 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_setxattr": 0, 2024-02-18T11:53:43.965 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_cmpxattr": 0, 2024-02-18T11:53:43.965 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_rmxattr": 0, 2024-02-18T11:53:43.965 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_resetxattrs": 0, 2024-02-18T11:53:43.965 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_call": 0, 2024-02-18T11:53:43.965 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_watch": 0, 2024-02-18T11:53:43.965 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_notify": 0, 2024-02-18T11:53:43.965 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_src_cmpxattr": 0, 2024-02-18T11:53:43.965 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_pgls": 0, 2024-02-18T11:53:43.965 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_pgls_filter": 0, 2024-02-18T11:53:43.965 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_other": 0, 2024-02-18T11:53:43.965 INFO:teuthology.orchestra.run.smithi158.stdout: "linger_active": 0, 2024-02-18T11:53:43.965 INFO:teuthology.orchestra.run.smithi158.stdout: "linger_send": 0, 2024-02-18T11:53:43.966 INFO:teuthology.orchestra.run.smithi158.stdout: "linger_resend": 0, 2024-02-18T11:53:43.966 INFO:teuthology.orchestra.run.smithi158.stdout: "linger_ping": 0, 2024-02-18T11:53:43.966 INFO:teuthology.orchestra.run.smithi158.stdout: "poolop_active": 0, 2024-02-18T11:53:43.966 INFO:teuthology.orchestra.run.smithi158.stdout: "poolop_send": 0, 2024-02-18T11:53:43.966 INFO:teuthology.orchestra.run.smithi158.stdout: "poolop_resend": 0, 2024-02-18T11:53:43.966 INFO:teuthology.orchestra.run.smithi158.stdout: "poolstat_active": 0, 2024-02-18T11:53:43.966 INFO:teuthology.orchestra.run.smithi158.stdout: "poolstat_send": 0, 2024-02-18T11:53:43.966 INFO:teuthology.orchestra.run.smithi158.stdout: "poolstat_resend": 0, 2024-02-18T11:53:43.966 INFO:teuthology.orchestra.run.smithi158.stdout: "statfs_active": 0, 2024-02-18T11:53:43.966 INFO:teuthology.orchestra.run.smithi158.stdout: "statfs_send": 0, 2024-02-18T11:53:43.966 INFO:teuthology.orchestra.run.smithi158.stdout: "statfs_resend": 0, 2024-02-18T11:53:43.966 INFO:teuthology.orchestra.run.smithi158.stdout: "command_active": 0, 2024-02-18T11:53:43.966 INFO:teuthology.orchestra.run.smithi158.stdout: "command_send": 0, 2024-02-18T11:53:43.966 INFO:teuthology.orchestra.run.smithi158.stdout: "command_resend": 0, 2024-02-18T11:53:43.966 INFO:teuthology.orchestra.run.smithi158.stdout: "map_epoch": 89, 2024-02-18T11:53:43.967 INFO:teuthology.orchestra.run.smithi158.stdout: "map_full": 0, 2024-02-18T11:53:43.967 INFO:teuthology.orchestra.run.smithi158.stdout: "map_inc": 2, 2024-02-18T11:53:43.967 INFO:teuthology.orchestra.run.smithi158.stdout: "osd_sessions": 5, 2024-02-18T11:53:43.967 INFO:teuthology.orchestra.run.smithi158.stdout: "osd_session_open": 5, 2024-02-18T11:53:43.967 INFO:teuthology.orchestra.run.smithi158.stdout: "osd_session_close": 0, 2024-02-18T11:53:43.967 INFO:teuthology.orchestra.run.smithi158.stdout: "osd_laggy": 0, 2024-02-18T11:53:43.967 INFO:teuthology.orchestra.run.smithi158.stdout: "omap_wr": 0, 2024-02-18T11:53:43.967 INFO:teuthology.orchestra.run.smithi158.stdout: "omap_rd": 2, 2024-02-18T11:53:43.967 INFO:teuthology.orchestra.run.smithi158.stdout: "omap_del": 0 2024-02-18T11:53:43.967 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:43.967 INFO:teuthology.orchestra.run.smithi158.stdout: "oft": { 2024-02-18T11:53:43.967 INFO:teuthology.orchestra.run.smithi158.stdout: "omap_total_objs": 0, 2024-02-18T11:53:43.967 INFO:teuthology.orchestra.run.smithi158.stdout: "omap_total_kv_pairs": 0, 2024-02-18T11:53:43.967 INFO:teuthology.orchestra.run.smithi158.stdout: "omap_total_updates": 0, 2024-02-18T11:53:43.967 INFO:teuthology.orchestra.run.smithi158.stdout: "omap_total_removes": 0 2024-02-18T11:53:43.968 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:43.968 INFO:teuthology.orchestra.run.smithi158.stdout: "osd-slow-ops": { 2024-02-18T11:53:43.968 INFO:teuthology.orchestra.run.smithi158.stdout: "slow_ops_count": 0 2024-02-18T11:53:43.968 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:43.968 INFO:teuthology.orchestra.run.smithi158.stdout: "purge_queue": { 2024-02-18T11:53:43.968 INFO:teuthology.orchestra.run.smithi158.stdout: "pq_executing_ops": 0, 2024-02-18T11:53:43.968 INFO:teuthology.orchestra.run.smithi158.stdout: "pq_executing_ops_high_water": 0, 2024-02-18T11:53:43.968 INFO:teuthology.orchestra.run.smithi158.stdout: "pq_executing": 0, 2024-02-18T11:53:43.968 INFO:teuthology.orchestra.run.smithi158.stdout: "pq_executing_high_water": 0, 2024-02-18T11:53:43.968 INFO:teuthology.orchestra.run.smithi158.stdout: "pq_executed_ops": 0, 2024-02-18T11:53:43.968 INFO:teuthology.orchestra.run.smithi158.stdout: "pq_executed": 0, 2024-02-18T11:53:43.968 INFO:teuthology.orchestra.run.smithi158.stdout: "pq_item_in_journal": 0 2024-02-18T11:53:43.968 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:43.968 INFO:teuthology.orchestra.run.smithi158.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-18T11:53:43.968 INFO:teuthology.orchestra.run.smithi158.stdout: "val": 0, 2024-02-18T11:53:43.969 INFO:teuthology.orchestra.run.smithi158.stdout: "max": 104857600, 2024-02-18T11:53:43.969 INFO:teuthology.orchestra.run.smithi158.stdout: "get_started": 0, 2024-02-18T11:53:43.969 INFO:teuthology.orchestra.run.smithi158.stdout: "get": 2720, 2024-02-18T11:53:43.969 INFO:teuthology.orchestra.run.smithi158.stdout: "get_sum": 1217717, 2024-02-18T11:53:43.969 INFO:teuthology.orchestra.run.smithi158.stdout: "get_or_fail_fail": 0, 2024-02-18T11:53:43.969 INFO:teuthology.orchestra.run.smithi158.stdout: "get_or_fail_success": 2720, 2024-02-18T11:53:43.969 INFO:teuthology.orchestra.run.smithi158.stdout: "take": 0, 2024-02-18T11:53:43.969 INFO:teuthology.orchestra.run.smithi158.stdout: "take_sum": 0, 2024-02-18T11:53:43.969 INFO:teuthology.orchestra.run.smithi158.stdout: "put": 2720, 2024-02-18T11:53:43.969 INFO:teuthology.orchestra.run.smithi158.stdout: "put_sum": 1217717, 2024-02-18T11:53:43.969 INFO:teuthology.orchestra.run.smithi158.stdout: "wait": { 2024-02-18T11:53:43.969 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:43.969 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:43.969 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:43.969 INFO:teuthology.orchestra.run.smithi158.stdout: } 2024-02-18T11:53:43.970 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:43.970 INFO:teuthology.orchestra.run.smithi158.stdout: "throttle-objecter_bytes": { 2024-02-18T11:53:43.970 INFO:teuthology.orchestra.run.smithi158.stdout: "val": 0, 2024-02-18T11:53:43.970 INFO:teuthology.orchestra.run.smithi158.stdout: "max": 104857600, 2024-02-18T11:53:43.970 INFO:teuthology.orchestra.run.smithi158.stdout: "get_started": 0, 2024-02-18T11:53:43.970 INFO:teuthology.orchestra.run.smithi158.stdout: "get": 0, 2024-02-18T11:53:43.970 INFO:teuthology.orchestra.run.smithi158.stdout: "get_sum": 0, 2024-02-18T11:53:43.970 INFO:teuthology.orchestra.run.smithi158.stdout: "get_or_fail_fail": 0, 2024-02-18T11:53:43.970 INFO:teuthology.orchestra.run.smithi158.stdout: "get_or_fail_success": 0, 2024-02-18T11:53:43.970 INFO:teuthology.orchestra.run.smithi158.stdout: "take": 2459, 2024-02-18T11:53:43.970 INFO:teuthology.orchestra.run.smithi158.stdout: "take_sum": 1433, 2024-02-18T11:53:43.970 INFO:teuthology.orchestra.run.smithi158.stdout: "put": 2459, 2024-02-18T11:53:43.970 INFO:teuthology.orchestra.run.smithi158.stdout: "put_sum": 1433, 2024-02-18T11:53:43.970 INFO:teuthology.orchestra.run.smithi158.stdout: "wait": { 2024-02-18T11:53:43.970 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:43.970 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:43.971 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:43.971 INFO:teuthology.orchestra.run.smithi158.stdout: } 2024-02-18T11:53:43.971 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:43.971 INFO:teuthology.orchestra.run.smithi158.stdout: "throttle-objecter_ops": { 2024-02-18T11:53:43.971 INFO:teuthology.orchestra.run.smithi158.stdout: "val": 0, 2024-02-18T11:53:43.971 INFO:teuthology.orchestra.run.smithi158.stdout: "max": 1024, 2024-02-18T11:53:43.971 INFO:teuthology.orchestra.run.smithi158.stdout: "get_started": 0, 2024-02-18T11:53:43.971 INFO:teuthology.orchestra.run.smithi158.stdout: "get": 0, 2024-02-18T11:53:43.971 INFO:teuthology.orchestra.run.smithi158.stdout: "get_sum": 0, 2024-02-18T11:53:43.971 INFO:teuthology.orchestra.run.smithi158.stdout: "get_or_fail_fail": 0, 2024-02-18T11:53:43.971 INFO:teuthology.orchestra.run.smithi158.stdout: "get_or_fail_success": 0, 2024-02-18T11:53:43.971 INFO:teuthology.orchestra.run.smithi158.stdout: "take": 2459, 2024-02-18T11:53:43.971 INFO:teuthology.orchestra.run.smithi158.stdout: "take_sum": 2459, 2024-02-18T11:53:43.971 INFO:teuthology.orchestra.run.smithi158.stdout: "put": 2459, 2024-02-18T11:53:43.971 INFO:teuthology.orchestra.run.smithi158.stdout: "put_sum": 2459, 2024-02-18T11:53:43.972 INFO:teuthology.orchestra.run.smithi158.stdout: "wait": { 2024-02-18T11:53:43.972 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:43.972 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:43.972 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:43.972 INFO:teuthology.orchestra.run.smithi158.stdout: } 2024-02-18T11:53:43.972 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:43.972 INFO:teuthology.orchestra.run.smithi158.stdout: "throttle-write_buf_throttle": { 2024-02-18T11:53:43.972 INFO:teuthology.orchestra.run.smithi158.stdout: "val": 0, 2024-02-18T11:53:43.972 INFO:teuthology.orchestra.run.smithi158.stdout: "max": 3758096384, 2024-02-18T11:53:43.972 INFO:teuthology.orchestra.run.smithi158.stdout: "get_started": 0, 2024-02-18T11:53:43.972 INFO:teuthology.orchestra.run.smithi158.stdout: "get": 0, 2024-02-18T11:53:43.972 INFO:teuthology.orchestra.run.smithi158.stdout: "get_sum": 0, 2024-02-18T11:53:43.972 INFO:teuthology.orchestra.run.smithi158.stdout: "get_or_fail_fail": 0, 2024-02-18T11:53:43.972 INFO:teuthology.orchestra.run.smithi158.stdout: "get_or_fail_success": 0, 2024-02-18T11:53:43.973 INFO:teuthology.orchestra.run.smithi158.stdout: "take": 0, 2024-02-18T11:53:43.973 INFO:teuthology.orchestra.run.smithi158.stdout: "take_sum": 0, 2024-02-18T11:53:43.973 INFO:teuthology.orchestra.run.smithi158.stdout: "put": 0, 2024-02-18T11:53:43.973 INFO:teuthology.orchestra.run.smithi158.stdout: "put_sum": 0, 2024-02-18T11:53:43.973 INFO:teuthology.orchestra.run.smithi158.stdout: "wait": { 2024-02-18T11:53:43.973 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:43.973 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:43.973 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:43.973 INFO:teuthology.orchestra.run.smithi158.stdout: } 2024-02-18T11:53:43.973 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:43.973 INFO:teuthology.orchestra.run.smithi158.stdout: "throttle-write_buf_throttle-0x559e9b01e960": { 2024-02-18T11:53:43.973 INFO:teuthology.orchestra.run.smithi158.stdout: "val": 0, 2024-02-18T11:53:43.973 INFO:teuthology.orchestra.run.smithi158.stdout: "max": 3758096384, 2024-02-18T11:53:43.973 INFO:teuthology.orchestra.run.smithi158.stdout: "get_started": 0, 2024-02-18T11:53:43.973 INFO:teuthology.orchestra.run.smithi158.stdout: "get": 0, 2024-02-18T11:53:43.973 INFO:teuthology.orchestra.run.smithi158.stdout: "get_sum": 0, 2024-02-18T11:53:43.974 INFO:teuthology.orchestra.run.smithi158.stdout: "get_or_fail_fail": 0, 2024-02-18T11:53:43.974 INFO:teuthology.orchestra.run.smithi158.stdout: "get_or_fail_success": 0, 2024-02-18T11:53:43.974 INFO:teuthology.orchestra.run.smithi158.stdout: "take": 0, 2024-02-18T11:53:43.974 INFO:teuthology.orchestra.run.smithi158.stdout: "take_sum": 0, 2024-02-18T11:53:43.974 INFO:teuthology.orchestra.run.smithi158.stdout: "put": 0, 2024-02-18T11:53:43.974 INFO:teuthology.orchestra.run.smithi158.stdout: "put_sum": 0, 2024-02-18T11:53:43.974 INFO:teuthology.orchestra.run.smithi158.stdout: "wait": { 2024-02-18T11:53:43.974 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:43.974 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:43.974 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:43.974 INFO:teuthology.orchestra.run.smithi158.stdout: } 2024-02-18T11:53:43.974 INFO:teuthology.orchestra.run.smithi158.stdout: } 2024-02-18T11:53:43.974 INFO:teuthology.orchestra.run.smithi158.stdout:} 2024-02-18T11:53:44.763 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-02-18T11:53:44.763 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-02-18T11:53:44.763 DEBUG:tasks.check_counter:Getting stats from h 2024-02-18T11:53:44.763 DEBUG:teuthology.orchestra.run.smithi158:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph daemon mds.h perf dump 2024-02-18T11:53:45.006 INFO:teuthology.orchestra.run.smithi158.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T11:53:45.380 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:53:45 smithi158 ceph-mon[36626]: pgmap v735: 97 pgs: 97 active+clean; 8.2 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 26 MiB/s wr, 113 op/s 2024-02-18T11:53:45.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:53:45 smithi012 ceph-mon[32460]: pgmap v735: 97 pgs: 97 active+clean; 8.2 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 26 MiB/s wr, 113 op/s 2024-02-18T11:53:45.602 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:53:45 smithi188 ceph-mon[36925]: pgmap v735: 97 pgs: 97 active+clean; 8.2 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 26 MiB/s wr, 113 op/s 2024-02-18T11:53:47.047 INFO:teuthology.orchestra.run.smithi158.stdout:{ 2024-02-18T11:53:47.048 INFO:teuthology.orchestra.run.smithi158.stdout: "AsyncMessenger::Worker-0": { 2024-02-18T11:53:47.048 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_recv_messages": 2283, 2024-02-18T11:53:47.048 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_messages": 2283, 2024-02-18T11:53:47.048 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_recv_bytes": 451140177, 2024-02-18T11:53:47.048 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_bytes": 709227, 2024-02-18T11:53:47.048 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_created_connections": 8, 2024-02-18T11:53:47.048 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_active_connections": 4, 2024-02-18T11:53:47.048 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_total_time": 0.524955829, 2024-02-18T11:53:47.048 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_send_time": 0.070213830, 2024-02-18T11:53:47.048 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_recv_time": 0.351761011, 2024-02-18T11:53:47.048 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_fast_dispatch_time": 0.083659546, 2024-02-18T11:53:47.048 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_messages_queue_lat": { 2024-02-18T11:53:47.049 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 2282, 2024-02-18T11:53:47.049 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.109044003, 2024-02-18T11:53:47.049 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000047784 2024-02-18T11:53:47.049 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:47.049 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_handle_ack_lat": { 2024-02-18T11:53:47.049 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:47.049 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:47.049 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:47.049 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:47.049 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-02-18T11:53:47.049 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_encrypted_bytes": 448 2024-02-18T11:53:47.049 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:47.049 INFO:teuthology.orchestra.run.smithi158.stdout: "AsyncMessenger::Worker-1": { 2024-02-18T11:53:47.049 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_recv_messages": 1036, 2024-02-18T11:53:47.049 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_messages": 1222, 2024-02-18T11:53:47.050 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_recv_bytes": 386181908, 2024-02-18T11:53:47.050 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_bytes": 727727, 2024-02-18T11:53:47.050 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_created_connections": 7, 2024-02-18T11:53:47.050 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_active_connections": 6, 2024-02-18T11:53:47.050 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_total_time": 0.433309874, 2024-02-18T11:53:47.050 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_send_time": 0.032578829, 2024-02-18T11:53:47.050 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_recv_time": 0.344795360, 2024-02-18T11:53:47.050 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_fast_dispatch_time": 0.042416894, 2024-02-18T11:53:47.050 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_messages_queue_lat": { 2024-02-18T11:53:47.050 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 1220, 2024-02-18T11:53:47.050 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.060142721, 2024-02-18T11:53:47.050 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000049297 2024-02-18T11:53:47.050 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:47.050 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_handle_ack_lat": { 2024-02-18T11:53:47.050 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:47.051 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:47.051 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:47.051 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:47.051 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_recv_encrypted_bytes": 160, 2024-02-18T11:53:47.051 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_encrypted_bytes": 405776 2024-02-18T11:53:47.051 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:47.051 INFO:teuthology.orchestra.run.smithi158.stdout: "AsyncMessenger::Worker-2": { 2024-02-18T11:53:47.051 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_recv_messages": 1461, 2024-02-18T11:53:47.051 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_messages": 1455, 2024-02-18T11:53:47.051 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_recv_bytes": 391193511, 2024-02-18T11:53:47.051 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_bytes": 522600, 2024-02-18T11:53:47.051 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_created_connections": 5, 2024-02-18T11:53:47.051 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_active_connections": 4, 2024-02-18T11:53:47.051 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_total_time": 0.582049439, 2024-02-18T11:53:47.051 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_send_time": 0.043072717, 2024-02-18T11:53:47.052 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_recv_time": 0.464781716, 2024-02-18T11:53:47.052 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_fast_dispatch_time": 0.057879045, 2024-02-18T11:53:47.052 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_messages_queue_lat": { 2024-02-18T11:53:47.052 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 1454, 2024-02-18T11:53:47.052 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.107841322, 2024-02-18T11:53:47.052 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000074168 2024-02-18T11:53:47.052 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:47.052 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_handle_ack_lat": { 2024-02-18T11:53:47.052 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:47.052 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:47.052 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:47.052 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:47.052 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_recv_encrypted_bytes": 778096, 2024-02-18T11:53:47.052 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_encrypted_bytes": 149088 2024-02-18T11:53:47.053 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:47.053 INFO:teuthology.orchestra.run.smithi158.stdout: "cct": { 2024-02-18T11:53:47.053 INFO:teuthology.orchestra.run.smithi158.stdout: "total_workers": 1, 2024-02-18T11:53:47.053 INFO:teuthology.orchestra.run.smithi158.stdout: "unhealthy_workers": 0 2024-02-18T11:53:47.053 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:47.053 INFO:teuthology.orchestra.run.smithi158.stdout: "finisher-MDSRank": { 2024-02-18T11:53:47.053 INFO:teuthology.orchestra.run.smithi158.stdout: "queue_len": 0, 2024-02-18T11:53:47.053 INFO:teuthology.orchestra.run.smithi158.stdout: "complete_latency": { 2024-02-18T11:53:47.053 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 6631, 2024-02-18T11:53:47.053 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.208005046, 2024-02-18T11:53:47.053 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000031368 2024-02-18T11:53:47.053 INFO:teuthology.orchestra.run.smithi158.stdout: } 2024-02-18T11:53:47.053 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:47.053 INFO:teuthology.orchestra.run.smithi158.stdout: "finisher-PurgeQueue": { 2024-02-18T11:53:47.053 INFO:teuthology.orchestra.run.smithi158.stdout: "queue_len": 0, 2024-02-18T11:53:47.054 INFO:teuthology.orchestra.run.smithi158.stdout: "complete_latency": { 2024-02-18T11:53:47.054 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:47.054 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:47.054 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:47.054 INFO:teuthology.orchestra.run.smithi158.stdout: } 2024-02-18T11:53:47.054 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:47.054 INFO:teuthology.orchestra.run.smithi158.stdout: "mds": { 2024-02-18T11:53:47.054 INFO:teuthology.orchestra.run.smithi158.stdout: "request": 0, 2024-02-18T11:53:47.054 INFO:teuthology.orchestra.run.smithi158.stdout: "reply": 0, 2024-02-18T11:53:47.054 INFO:teuthology.orchestra.run.smithi158.stdout: "reply_latency": { 2024-02-18T11:53:47.055 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:47.055 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:47.055 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:47.055 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:47.055 INFO:teuthology.orchestra.run.smithi158.stdout: "slow_reply": 0, 2024-02-18T11:53:47.055 INFO:teuthology.orchestra.run.smithi158.stdout: "forward": 0, 2024-02-18T11:53:47.055 INFO:teuthology.orchestra.run.smithi158.stdout: "dir_fetch_complete": 0, 2024-02-18T11:53:47.055 INFO:teuthology.orchestra.run.smithi158.stdout: "dir_fetch_keys": 0, 2024-02-18T11:53:47.055 INFO:teuthology.orchestra.run.smithi158.stdout: "dir_commit": 0, 2024-02-18T11:53:47.055 INFO:teuthology.orchestra.run.smithi158.stdout: "dir_split": 0, 2024-02-18T11:53:47.055 INFO:teuthology.orchestra.run.smithi158.stdout: "dir_merge": 0, 2024-02-18T11:53:47.055 INFO:teuthology.orchestra.run.smithi158.stdout: "inodes": 36, 2024-02-18T11:53:47.055 INFO:teuthology.orchestra.run.smithi158.stdout: "inodes_top": 4, 2024-02-18T11:53:47.055 INFO:teuthology.orchestra.run.smithi158.stdout: "inodes_bottom": 32, 2024-02-18T11:53:47.056 INFO:teuthology.orchestra.run.smithi158.stdout: "inodes_pin_tail": 0, 2024-02-18T11:53:47.056 INFO:teuthology.orchestra.run.smithi158.stdout: "inodes_pinned": 29, 2024-02-18T11:53:47.056 INFO:teuthology.orchestra.run.smithi158.stdout: "inodes_expired": 0, 2024-02-18T11:53:47.056 INFO:teuthology.orchestra.run.smithi158.stdout: "inodes_with_caps": 0, 2024-02-18T11:53:47.056 INFO:teuthology.orchestra.run.smithi158.stdout: "caps": 0, 2024-02-18T11:53:47.056 INFO:teuthology.orchestra.run.smithi158.stdout: "subtrees": 2, 2024-02-18T11:53:47.056 INFO:teuthology.orchestra.run.smithi158.stdout: "traverse": 0, 2024-02-18T11:53:47.056 INFO:teuthology.orchestra.run.smithi158.stdout: "traverse_hit": 0, 2024-02-18T11:53:47.056 INFO:teuthology.orchestra.run.smithi158.stdout: "traverse_forward": 0, 2024-02-18T11:53:47.056 INFO:teuthology.orchestra.run.smithi158.stdout: "traverse_discover": 0, 2024-02-18T11:53:47.056 INFO:teuthology.orchestra.run.smithi158.stdout: "traverse_dir_fetch": 0, 2024-02-18T11:53:47.056 INFO:teuthology.orchestra.run.smithi158.stdout: "traverse_remote_ino": 0, 2024-02-18T11:53:47.056 INFO:teuthology.orchestra.run.smithi158.stdout: "traverse_lock": 0, 2024-02-18T11:53:47.056 INFO:teuthology.orchestra.run.smithi158.stdout: "load_cent": 0, 2024-02-18T11:53:47.057 INFO:teuthology.orchestra.run.smithi158.stdout: "q": 0, 2024-02-18T11:53:47.057 INFO:teuthology.orchestra.run.smithi158.stdout: "exported": 0, 2024-02-18T11:53:47.057 INFO:teuthology.orchestra.run.smithi158.stdout: "exported_inodes": 0, 2024-02-18T11:53:47.057 INFO:teuthology.orchestra.run.smithi158.stdout: "imported": 0, 2024-02-18T11:53:47.057 INFO:teuthology.orchestra.run.smithi158.stdout: "imported_inodes": 0, 2024-02-18T11:53:47.057 INFO:teuthology.orchestra.run.smithi158.stdout: "openino_dir_fetch": 0, 2024-02-18T11:53:47.057 INFO:teuthology.orchestra.run.smithi158.stdout: "openino_backtrace_fetch": 0, 2024-02-18T11:53:47.057 INFO:teuthology.orchestra.run.smithi158.stdout: "openino_peer_discover": 0, 2024-02-18T11:53:47.057 INFO:teuthology.orchestra.run.smithi158.stdout: "root_rfiles": 4, 2024-02-18T11:53:47.057 INFO:teuthology.orchestra.run.smithi158.stdout: "root_rbytes": 234, 2024-02-18T11:53:47.057 INFO:teuthology.orchestra.run.smithi158.stdout: "root_rsnaps": 0, 2024-02-18T11:53:47.057 INFO:teuthology.orchestra.run.smithi158.stdout: "scrub_backtrace_fetch": 0, 2024-02-18T11:53:47.057 INFO:teuthology.orchestra.run.smithi158.stdout: "scrub_set_tag": 0, 2024-02-18T11:53:47.057 INFO:teuthology.orchestra.run.smithi158.stdout: "scrub_backtrace_repaired": 0, 2024-02-18T11:53:47.057 INFO:teuthology.orchestra.run.smithi158.stdout: "scrub_inotable_repaired": 0, 2024-02-18T11:53:47.058 INFO:teuthology.orchestra.run.smithi158.stdout: "scrub_dir_inodes": 0, 2024-02-18T11:53:47.058 INFO:teuthology.orchestra.run.smithi158.stdout: "scrub_dir_base_inodes": 0, 2024-02-18T11:53:47.058 INFO:teuthology.orchestra.run.smithi158.stdout: "scrub_dirfrag_rstats": 0, 2024-02-18T11:53:47.058 INFO:teuthology.orchestra.run.smithi158.stdout: "scrub_file_inodes": 0, 2024-02-18T11:53:47.058 INFO:teuthology.orchestra.run.smithi158.stdout: "handle_inode_file_caps": 0, 2024-02-18T11:53:47.058 INFO:teuthology.orchestra.run.smithi158.stdout: "ceph_cap_op_revoke": 0, 2024-02-18T11:53:47.058 INFO:teuthology.orchestra.run.smithi158.stdout: "ceph_cap_op_grant": 0, 2024-02-18T11:53:47.058 INFO:teuthology.orchestra.run.smithi158.stdout: "ceph_cap_op_trunc": 0, 2024-02-18T11:53:47.058 INFO:teuthology.orchestra.run.smithi158.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-02-18T11:53:47.058 INFO:teuthology.orchestra.run.smithi158.stdout: "ceph_cap_op_flush_ack": 0, 2024-02-18T11:53:47.058 INFO:teuthology.orchestra.run.smithi158.stdout: "handle_client_caps": 0, 2024-02-18T11:53:47.058 INFO:teuthology.orchestra.run.smithi158.stdout: "handle_client_caps_dirty": 0, 2024-02-18T11:53:47.058 INFO:teuthology.orchestra.run.smithi158.stdout: "handle_client_cap_release": 0, 2024-02-18T11:53:47.058 INFO:teuthology.orchestra.run.smithi158.stdout: "process_request_cap_release": 0 2024-02-18T11:53:47.059 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:47.059 INFO:teuthology.orchestra.run.smithi158.stdout: "mds_cache": { 2024-02-18T11:53:47.059 INFO:teuthology.orchestra.run.smithi158.stdout: "dir_update": 0, 2024-02-18T11:53:47.059 INFO:teuthology.orchestra.run.smithi158.stdout: "dir_update_receipt": 0, 2024-02-18T11:53:47.059 INFO:teuthology.orchestra.run.smithi158.stdout: "dir_try_discover": 0, 2024-02-18T11:53:47.059 INFO:teuthology.orchestra.run.smithi158.stdout: "dir_send_discover": 0, 2024-02-18T11:53:47.059 INFO:teuthology.orchestra.run.smithi158.stdout: "dir_handle_discover": 0, 2024-02-18T11:53:47.059 INFO:teuthology.orchestra.run.smithi158.stdout: "num_strays": 0, 2024-02-18T11:53:47.059 INFO:teuthology.orchestra.run.smithi158.stdout: "num_strays_delayed": 0, 2024-02-18T11:53:47.059 INFO:teuthology.orchestra.run.smithi158.stdout: "num_strays_enqueuing": 0, 2024-02-18T11:53:47.059 INFO:teuthology.orchestra.run.smithi158.stdout: "strays_created": 0, 2024-02-18T11:53:47.059 INFO:teuthology.orchestra.run.smithi158.stdout: "strays_enqueued": 0, 2024-02-18T11:53:47.059 INFO:teuthology.orchestra.run.smithi158.stdout: "strays_reintegrated": 0, 2024-02-18T11:53:47.059 INFO:teuthology.orchestra.run.smithi158.stdout: "strays_migrated": 0, 2024-02-18T11:53:47.059 INFO:teuthology.orchestra.run.smithi158.stdout: "num_recovering_processing": 0, 2024-02-18T11:53:47.060 INFO:teuthology.orchestra.run.smithi158.stdout: "num_recovering_enqueued": 0, 2024-02-18T11:53:47.060 INFO:teuthology.orchestra.run.smithi158.stdout: "num_recovering_prioritized": 0, 2024-02-18T11:53:47.060 INFO:teuthology.orchestra.run.smithi158.stdout: "recovery_started": 0, 2024-02-18T11:53:47.060 INFO:teuthology.orchestra.run.smithi158.stdout: "recovery_completed": 0, 2024-02-18T11:53:47.060 INFO:teuthology.orchestra.run.smithi158.stdout: "ireq_enqueue_scrub": 0, 2024-02-18T11:53:47.060 INFO:teuthology.orchestra.run.smithi158.stdout: "ireq_exportdir": 0, 2024-02-18T11:53:47.060 INFO:teuthology.orchestra.run.smithi158.stdout: "ireq_flush": 0, 2024-02-18T11:53:47.060 INFO:teuthology.orchestra.run.smithi158.stdout: "ireq_fragmentdir": 0, 2024-02-18T11:53:47.060 INFO:teuthology.orchestra.run.smithi158.stdout: "ireq_fragstats": 0, 2024-02-18T11:53:47.060 INFO:teuthology.orchestra.run.smithi158.stdout: "ireq_inodestats": 0 2024-02-18T11:53:47.060 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:47.060 INFO:teuthology.orchestra.run.smithi158.stdout: "mds_log": { 2024-02-18T11:53:47.060 INFO:teuthology.orchestra.run.smithi158.stdout: "evlrg": 0, 2024-02-18T11:53:47.060 INFO:teuthology.orchestra.run.smithi158.stdout: "evadd": 0, 2024-02-18T11:53:47.060 INFO:teuthology.orchestra.run.smithi158.stdout: "evex": 0, 2024-02-18T11:53:47.061 INFO:teuthology.orchestra.run.smithi158.stdout: "evtrm": 0, 2024-02-18T11:53:47.061 INFO:teuthology.orchestra.run.smithi158.stdout: "ev": 707102, 2024-02-18T11:53:47.061 INFO:teuthology.orchestra.run.smithi158.stdout: "evexg": 0, 2024-02-18T11:53:47.061 INFO:teuthology.orchestra.run.smithi158.stdout: "evexd": 0, 2024-02-18T11:53:47.061 INFO:teuthology.orchestra.run.smithi158.stdout: "segadd": 0, 2024-02-18T11:53:47.061 INFO:teuthology.orchestra.run.smithi158.stdout: "segex": 0, 2024-02-18T11:53:47.061 INFO:teuthology.orchestra.run.smithi158.stdout: "segtrm": 0, 2024-02-18T11:53:47.061 INFO:teuthology.orchestra.run.smithi158.stdout: "seg": 136, 2024-02-18T11:53:47.061 INFO:teuthology.orchestra.run.smithi158.stdout: "segmjr": 59, 2024-02-18T11:53:47.061 INFO:teuthology.orchestra.run.smithi158.stdout: "segexg": 0, 2024-02-18T11:53:47.061 INFO:teuthology.orchestra.run.smithi158.stdout: "segexd": 0, 2024-02-18T11:53:47.061 INFO:teuthology.orchestra.run.smithi158.stdout: "expos": 1015262913, 2024-02-18T11:53:47.061 INFO:teuthology.orchestra.run.smithi158.stdout: "wrpos": 1230577632, 2024-02-18T11:53:47.061 INFO:teuthology.orchestra.run.smithi158.stdout: "rdpos": 1230575980, 2024-02-18T11:53:47.061 INFO:teuthology.orchestra.run.smithi158.stdout: "jlat": { 2024-02-18T11:53:47.062 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:47.062 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:47.062 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:47.062 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:47.062 INFO:teuthology.orchestra.run.smithi158.stdout: "replayed": 707102 2024-02-18T11:53:47.062 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:47.062 INFO:teuthology.orchestra.run.smithi158.stdout: "mds_mem": { 2024-02-18T11:53:47.062 INFO:teuthology.orchestra.run.smithi158.stdout: "ino": 18, 2024-02-18T11:53:47.062 INFO:teuthology.orchestra.run.smithi158.stdout: "ino+": 36, 2024-02-18T11:53:47.062 INFO:teuthology.orchestra.run.smithi158.stdout: "ino-": 18, 2024-02-18T11:53:47.062 INFO:teuthology.orchestra.run.smithi158.stdout: "dir": 9, 2024-02-18T11:53:47.062 INFO:teuthology.orchestra.run.smithi158.stdout: "dir+": 21, 2024-02-18T11:53:47.062 INFO:teuthology.orchestra.run.smithi158.stdout: "dir-": 12, 2024-02-18T11:53:47.062 INFO:teuthology.orchestra.run.smithi158.stdout: "dn": 36, 2024-02-18T11:53:47.062 INFO:teuthology.orchestra.run.smithi158.stdout: "dn+": 57, 2024-02-18T11:53:47.063 INFO:teuthology.orchestra.run.smithi158.stdout: "dn-": 21, 2024-02-18T11:53:47.063 INFO:teuthology.orchestra.run.smithi158.stdout: "cap": 0, 2024-02-18T11:53:47.063 INFO:teuthology.orchestra.run.smithi158.stdout: "cap+": 0, 2024-02-18T11:53:47.063 INFO:teuthology.orchestra.run.smithi158.stdout: "cap-": 0, 2024-02-18T11:53:47.063 INFO:teuthology.orchestra.run.smithi158.stdout: "rss": 54248, 2024-02-18T11:53:47.063 INFO:teuthology.orchestra.run.smithi158.stdout: "heap": 215420 2024-02-18T11:53:47.063 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:47.063 INFO:teuthology.orchestra.run.smithi158.stdout: "mds_server": { 2024-02-18T11:53:47.063 INFO:teuthology.orchestra.run.smithi158.stdout: "dispatch_client_request": 0, 2024-02-18T11:53:47.063 INFO:teuthology.orchestra.run.smithi158.stdout: "dispatch_server_request": 0, 2024-02-18T11:53:47.063 INFO:teuthology.orchestra.run.smithi158.stdout: "handle_client_request": 0, 2024-02-18T11:53:47.063 INFO:teuthology.orchestra.run.smithi158.stdout: "handle_client_session": 0, 2024-02-18T11:53:47.063 INFO:teuthology.orchestra.run.smithi158.stdout: "handle_peer_request": 0, 2024-02-18T11:53:47.063 INFO:teuthology.orchestra.run.smithi158.stdout: "req_create_latency": { 2024-02-18T11:53:47.063 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:47.064 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:47.064 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:47.064 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:47.064 INFO:teuthology.orchestra.run.smithi158.stdout: "req_getattr_latency": { 2024-02-18T11:53:47.064 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:47.064 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:47.064 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:47.064 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:47.064 INFO:teuthology.orchestra.run.smithi158.stdout: "req_getfilelock_latency": { 2024-02-18T11:53:47.064 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:47.064 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:47.064 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:47.064 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:47.064 INFO:teuthology.orchestra.run.smithi158.stdout: "req_link_latency": { 2024-02-18T11:53:47.065 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:47.065 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:47.065 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:47.065 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:47.065 INFO:teuthology.orchestra.run.smithi158.stdout: "req_lookup_latency": { 2024-02-18T11:53:47.065 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:47.065 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:47.065 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:47.065 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:47.065 INFO:teuthology.orchestra.run.smithi158.stdout: "req_lookuphash_latency": { 2024-02-18T11:53:47.065 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:47.065 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:47.065 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:47.065 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:47.065 INFO:teuthology.orchestra.run.smithi158.stdout: "req_lookupino_latency": { 2024-02-18T11:53:47.066 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:47.066 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:47.066 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:47.066 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:47.066 INFO:teuthology.orchestra.run.smithi158.stdout: "req_lookupname_latency": { 2024-02-18T11:53:47.066 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:47.066 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:47.066 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:47.066 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:47.066 INFO:teuthology.orchestra.run.smithi158.stdout: "req_lookupparent_latency": { 2024-02-18T11:53:47.066 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:47.066 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:47.066 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:47.066 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:47.066 INFO:teuthology.orchestra.run.smithi158.stdout: "req_lookupsnap_latency": { 2024-02-18T11:53:47.067 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:47.067 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:47.067 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:47.067 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:47.067 INFO:teuthology.orchestra.run.smithi158.stdout: "req_lssnap_latency": { 2024-02-18T11:53:47.067 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:47.067 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:47.067 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:47.067 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:47.067 INFO:teuthology.orchestra.run.smithi158.stdout: "req_mkdir_latency": { 2024-02-18T11:53:47.067 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:47.067 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:47.067 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:47.067 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:47.068 INFO:teuthology.orchestra.run.smithi158.stdout: "req_mknod_latency": { 2024-02-18T11:53:47.068 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:47.068 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:47.068 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:47.068 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:47.068 INFO:teuthology.orchestra.run.smithi158.stdout: "req_mksnap_latency": { 2024-02-18T11:53:47.068 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:47.068 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:47.068 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:47.068 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:47.068 INFO:teuthology.orchestra.run.smithi158.stdout: "req_open_latency": { 2024-02-18T11:53:47.068 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:47.068 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:47.068 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:47.068 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:47.069 INFO:teuthology.orchestra.run.smithi158.stdout: "req_readdir_latency": { 2024-02-18T11:53:47.069 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:47.069 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:47.069 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:47.069 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:47.069 INFO:teuthology.orchestra.run.smithi158.stdout: "req_rename_latency": { 2024-02-18T11:53:47.069 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:47.069 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:47.069 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:47.069 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:47.069 INFO:teuthology.orchestra.run.smithi158.stdout: "req_renamesnap_latency": { 2024-02-18T11:53:47.069 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:47.069 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:47.069 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:47.069 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:47.070 INFO:teuthology.orchestra.run.smithi158.stdout: "req_snapdiff_latency": { 2024-02-18T11:53:47.070 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:47.070 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:47.070 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:47.070 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:47.070 INFO:teuthology.orchestra.run.smithi158.stdout: "req_rmdir_latency": { 2024-02-18T11:53:47.070 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:47.070 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:47.070 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:47.070 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:47.070 INFO:teuthology.orchestra.run.smithi158.stdout: "req_rmsnap_latency": { 2024-02-18T11:53:47.070 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:47.070 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:47.070 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:47.071 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:47.071 INFO:teuthology.orchestra.run.smithi158.stdout: "req_rmxattr_latency": { 2024-02-18T11:53:47.071 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:47.071 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:47.071 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:47.071 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:47.071 INFO:teuthology.orchestra.run.smithi158.stdout: "req_setattr_latency": { 2024-02-18T11:53:47.071 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:47.071 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:47.071 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:47.071 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:47.071 INFO:teuthology.orchestra.run.smithi158.stdout: "req_setdirlayout_latency": { 2024-02-18T11:53:47.071 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:47.071 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:47.071 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:47.072 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:47.072 INFO:teuthology.orchestra.run.smithi158.stdout: "req_setfilelock_latency": { 2024-02-18T11:53:47.072 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:47.072 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:47.072 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:47.072 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:47.072 INFO:teuthology.orchestra.run.smithi158.stdout: "req_setlayout_latency": { 2024-02-18T11:53:47.072 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:47.072 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:47.072 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:47.072 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:47.072 INFO:teuthology.orchestra.run.smithi158.stdout: "req_setxattr_latency": { 2024-02-18T11:53:47.072 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:47.072 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:47.072 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:47.073 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:47.073 INFO:teuthology.orchestra.run.smithi158.stdout: "req_symlink_latency": { 2024-02-18T11:53:47.073 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:47.073 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:47.073 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:47.073 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:47.073 INFO:teuthology.orchestra.run.smithi158.stdout: "req_unlink_latency": { 2024-02-18T11:53:47.073 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:47.073 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:47.073 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:47.073 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:47.073 INFO:teuthology.orchestra.run.smithi158.stdout: "cap_revoke_eviction": 0, 2024-02-18T11:53:47.073 INFO:teuthology.orchestra.run.smithi158.stdout: "cap_acquisition_throttle": 0, 2024-02-18T11:53:47.073 INFO:teuthology.orchestra.run.smithi158.stdout: "req_getvxattr_latency": { 2024-02-18T11:53:47.074 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:47.074 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:47.074 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:47.074 INFO:teuthology.orchestra.run.smithi158.stdout: } 2024-02-18T11:53:47.074 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:47.074 INFO:teuthology.orchestra.run.smithi158.stdout: "mds_sessions": { 2024-02-18T11:53:47.074 INFO:teuthology.orchestra.run.smithi158.stdout: "session_count": 1, 2024-02-18T11:53:47.074 INFO:teuthology.orchestra.run.smithi158.stdout: "session_add": 3, 2024-02-18T11:53:47.074 INFO:teuthology.orchestra.run.smithi158.stdout: "session_remove": 2, 2024-02-18T11:53:47.074 INFO:teuthology.orchestra.run.smithi158.stdout: "sessions_open": 2, 2024-02-18T11:53:47.074 INFO:teuthology.orchestra.run.smithi158.stdout: "sessions_stale": 0, 2024-02-18T11:53:47.074 INFO:teuthology.orchestra.run.smithi158.stdout: "total_load": 0, 2024-02-18T11:53:47.074 INFO:teuthology.orchestra.run.smithi158.stdout: "average_load": 0, 2024-02-18T11:53:47.074 INFO:teuthology.orchestra.run.smithi158.stdout: "avg_session_uptime": 3311, 2024-02-18T11:53:47.074 INFO:teuthology.orchestra.run.smithi158.stdout: "mdthresh_evicted": 0 2024-02-18T11:53:47.075 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:47.075 INFO:teuthology.orchestra.run.smithi158.stdout: "mempool": { 2024-02-18T11:53:47.075 INFO:teuthology.orchestra.run.smithi158.stdout: "bloom_filter_bytes": 0, 2024-02-18T11:53:47.075 INFO:teuthology.orchestra.run.smithi158.stdout: "bloom_filter_items": 0, 2024-02-18T11:53:47.075 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_alloc_bytes": 0, 2024-02-18T11:53:47.075 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_alloc_items": 0, 2024-02-18T11:53:47.075 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_data_bytes": 0, 2024-02-18T11:53:47.075 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_data_items": 0, 2024-02-18T11:53:47.075 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-18T11:53:47.075 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_onode_items": 0, 2024-02-18T11:53:47.075 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-18T11:53:47.075 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_meta_items": 0, 2024-02-18T11:53:47.075 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_other_bytes": 0, 2024-02-18T11:53:47.075 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_other_items": 0, 2024-02-18T11:53:47.076 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-18T11:53:47.076 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_buffer_items": 0, 2024-02-18T11:53:47.076 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_extent_bytes": 0, 2024-02-18T11:53:47.076 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_extent_items": 0, 2024-02-18T11:53:47.076 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_blob_bytes": 0, 2024-02-18T11:53:47.076 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_blob_items": 0, 2024-02-18T11:53:47.076 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-18T11:53:47.076 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_shared_blob_items": 0, 2024-02-18T11:53:47.076 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-18T11:53:47.076 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_inline_bl_items": 0, 2024-02-18T11:53:47.076 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_fsck_bytes": 0, 2024-02-18T11:53:47.076 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_fsck_items": 0, 2024-02-18T11:53:47.076 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_txc_bytes": 0, 2024-02-18T11:53:47.076 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_txc_items": 0, 2024-02-18T11:53:47.076 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-18T11:53:47.077 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_writing_deferred_items": 0, 2024-02-18T11:53:47.077 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_writing_bytes": 0, 2024-02-18T11:53:47.077 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_writing_items": 0, 2024-02-18T11:53:47.077 INFO:teuthology.orchestra.run.smithi158.stdout: "bluefs_bytes": 0, 2024-02-18T11:53:47.077 INFO:teuthology.orchestra.run.smithi158.stdout: "bluefs_items": 0, 2024-02-18T11:53:47.077 INFO:teuthology.orchestra.run.smithi158.stdout: "bluefs_file_reader_bytes": 0, 2024-02-18T11:53:47.077 INFO:teuthology.orchestra.run.smithi158.stdout: "bluefs_file_reader_items": 0, 2024-02-18T11:53:47.077 INFO:teuthology.orchestra.run.smithi158.stdout: "bluefs_file_writer_bytes": 0, 2024-02-18T11:53:47.077 INFO:teuthology.orchestra.run.smithi158.stdout: "bluefs_file_writer_items": 0, 2024-02-18T11:53:47.077 INFO:teuthology.orchestra.run.smithi158.stdout: "buffer_anon_bytes": 238835, 2024-02-18T11:53:47.077 INFO:teuthology.orchestra.run.smithi158.stdout: "buffer_anon_items": 81, 2024-02-18T11:53:47.077 INFO:teuthology.orchestra.run.smithi158.stdout: "buffer_meta_bytes": 0, 2024-02-18T11:53:47.077 INFO:teuthology.orchestra.run.smithi158.stdout: "buffer_meta_items": 0, 2024-02-18T11:53:47.077 INFO:teuthology.orchestra.run.smithi158.stdout: "osd_bytes": 0, 2024-02-18T11:53:47.077 INFO:teuthology.orchestra.run.smithi158.stdout: "osd_items": 0, 2024-02-18T11:53:47.078 INFO:teuthology.orchestra.run.smithi158.stdout: "osd_mapbl_bytes": 0, 2024-02-18T11:53:47.078 INFO:teuthology.orchestra.run.smithi158.stdout: "osd_mapbl_items": 0, 2024-02-18T11:53:47.078 INFO:teuthology.orchestra.run.smithi158.stdout: "osd_pglog_bytes": 0, 2024-02-18T11:53:47.078 INFO:teuthology.orchestra.run.smithi158.stdout: "osd_pglog_items": 0, 2024-02-18T11:53:47.078 INFO:teuthology.orchestra.run.smithi158.stdout: "osdmap_bytes": 7528, 2024-02-18T11:53:47.078 INFO:teuthology.orchestra.run.smithi158.stdout: "osdmap_items": 158, 2024-02-18T11:53:47.078 INFO:teuthology.orchestra.run.smithi158.stdout: "osdmap_mapping_bytes": 0, 2024-02-18T11:53:47.078 INFO:teuthology.orchestra.run.smithi158.stdout: "osdmap_mapping_items": 0, 2024-02-18T11:53:47.078 INFO:teuthology.orchestra.run.smithi158.stdout: "pgmap_bytes": 0, 2024-02-18T11:53:47.078 INFO:teuthology.orchestra.run.smithi158.stdout: "pgmap_items": 0, 2024-02-18T11:53:47.078 INFO:teuthology.orchestra.run.smithi158.stdout: "mds_co_bytes": 76798, 2024-02-18T11:53:47.078 INFO:teuthology.orchestra.run.smithi158.stdout: "mds_co_items": 805, 2024-02-18T11:53:47.078 INFO:teuthology.orchestra.run.smithi158.stdout: "unittest_1_bytes": 0, 2024-02-18T11:53:47.078 INFO:teuthology.orchestra.run.smithi158.stdout: "unittest_1_items": 0, 2024-02-18T11:53:47.078 INFO:teuthology.orchestra.run.smithi158.stdout: "unittest_2_bytes": 0, 2024-02-18T11:53:47.079 INFO:teuthology.orchestra.run.smithi158.stdout: "unittest_2_items": 0 2024-02-18T11:53:47.079 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:47.079 INFO:teuthology.orchestra.run.smithi158.stdout: "objecter": { 2024-02-18T11:53:47.079 INFO:teuthology.orchestra.run.smithi158.stdout: "op_active": 0, 2024-02-18T11:53:47.079 INFO:teuthology.orchestra.run.smithi158.stdout: "op_laggy": 0, 2024-02-18T11:53:47.079 INFO:teuthology.orchestra.run.smithi158.stdout: "op_send": 4515, 2024-02-18T11:53:47.079 INFO:teuthology.orchestra.run.smithi158.stdout: "op_send_bytes": 26, 2024-02-18T11:53:47.079 INFO:teuthology.orchestra.run.smithi158.stdout: "op_resend": 4, 2024-02-18T11:53:47.079 INFO:teuthology.orchestra.run.smithi158.stdout: "op_reply": 4515, 2024-02-18T11:53:47.079 INFO:teuthology.orchestra.run.smithi158.stdout: "op_latency": { 2024-02-18T11:53:47.079 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 4515, 2024-02-18T11:53:47.079 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 46.450502233, 2024-02-18T11:53:47.079 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.010288040 2024-02-18T11:53:47.080 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:47.080 INFO:teuthology.orchestra.run.smithi158.stdout: "op_inflight": 0, 2024-02-18T11:53:47.080 INFO:teuthology.orchestra.run.smithi158.stdout: "oplen_avg": { 2024-02-18T11:53:47.080 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 4515, 2024-02-18T11:53:47.080 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 4516 2024-02-18T11:53:47.080 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:47.080 INFO:teuthology.orchestra.run.smithi158.stdout: "op": 4515, 2024-02-18T11:53:47.080 INFO:teuthology.orchestra.run.smithi158.stdout: "op_r": 4515, 2024-02-18T11:53:47.080 INFO:teuthology.orchestra.run.smithi158.stdout: "op_w": 0, 2024-02-18T11:53:47.080 INFO:teuthology.orchestra.run.smithi158.stdout: "op_rmw": 0, 2024-02-18T11:53:47.080 INFO:teuthology.orchestra.run.smithi158.stdout: "op_pg": 0, 2024-02-18T11:53:47.080 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_stat": 2858, 2024-02-18T11:53:47.080 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_create": 0, 2024-02-18T11:53:47.080 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_read": 1654, 2024-02-18T11:53:47.081 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_write": 0, 2024-02-18T11:53:47.081 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_writefull": 0, 2024-02-18T11:53:47.081 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_writesame": 0, 2024-02-18T11:53:47.081 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_append": 0, 2024-02-18T11:53:47.081 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_zero": 0, 2024-02-18T11:53:47.081 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_truncate": 0, 2024-02-18T11:53:47.081 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_delete": 0, 2024-02-18T11:53:47.081 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_mapext": 0, 2024-02-18T11:53:47.081 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_sparse_read": 0, 2024-02-18T11:53:47.081 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_clonerange": 0, 2024-02-18T11:53:47.081 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_getxattr": 2, 2024-02-18T11:53:47.081 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_setxattr": 0, 2024-02-18T11:53:47.081 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_cmpxattr": 0, 2024-02-18T11:53:47.081 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_rmxattr": 0, 2024-02-18T11:53:47.081 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_resetxattrs": 0, 2024-02-18T11:53:47.082 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_call": 0, 2024-02-18T11:53:47.082 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_watch": 0, 2024-02-18T11:53:47.082 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_notify": 0, 2024-02-18T11:53:47.082 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_src_cmpxattr": 0, 2024-02-18T11:53:47.082 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_pgls": 0, 2024-02-18T11:53:47.082 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_pgls_filter": 0, 2024-02-18T11:53:47.082 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_other": 0, 2024-02-18T11:53:47.082 INFO:teuthology.orchestra.run.smithi158.stdout: "linger_active": 0, 2024-02-18T11:53:47.082 INFO:teuthology.orchestra.run.smithi158.stdout: "linger_send": 0, 2024-02-18T11:53:47.082 INFO:teuthology.orchestra.run.smithi158.stdout: "linger_resend": 0, 2024-02-18T11:53:47.082 INFO:teuthology.orchestra.run.smithi158.stdout: "linger_ping": 0, 2024-02-18T11:53:47.082 INFO:teuthology.orchestra.run.smithi158.stdout: "poolop_active": 0, 2024-02-18T11:53:47.082 INFO:teuthology.orchestra.run.smithi158.stdout: "poolop_send": 0, 2024-02-18T11:53:47.082 INFO:teuthology.orchestra.run.smithi158.stdout: "poolop_resend": 0, 2024-02-18T11:53:47.082 INFO:teuthology.orchestra.run.smithi158.stdout: "poolstat_active": 0, 2024-02-18T11:53:47.083 INFO:teuthology.orchestra.run.smithi158.stdout: "poolstat_send": 0, 2024-02-18T11:53:47.083 INFO:teuthology.orchestra.run.smithi158.stdout: "poolstat_resend": 0, 2024-02-18T11:53:47.083 INFO:teuthology.orchestra.run.smithi158.stdout: "statfs_active": 0, 2024-02-18T11:53:47.083 INFO:teuthology.orchestra.run.smithi158.stdout: "statfs_send": 0, 2024-02-18T11:53:47.083 INFO:teuthology.orchestra.run.smithi158.stdout: "statfs_resend": 0, 2024-02-18T11:53:47.083 INFO:teuthology.orchestra.run.smithi158.stdout: "command_active": 0, 2024-02-18T11:53:47.083 INFO:teuthology.orchestra.run.smithi158.stdout: "command_send": 0, 2024-02-18T11:53:47.083 INFO:teuthology.orchestra.run.smithi158.stdout: "command_resend": 0, 2024-02-18T11:53:47.083 INFO:teuthology.orchestra.run.smithi158.stdout: "map_epoch": 89, 2024-02-18T11:53:47.083 INFO:teuthology.orchestra.run.smithi158.stdout: "map_full": 0, 2024-02-18T11:53:47.083 INFO:teuthology.orchestra.run.smithi158.stdout: "map_inc": 2, 2024-02-18T11:53:47.083 INFO:teuthology.orchestra.run.smithi158.stdout: "osd_sessions": 11, 2024-02-18T11:53:47.083 INFO:teuthology.orchestra.run.smithi158.stdout: "osd_session_open": 11, 2024-02-18T11:53:47.083 INFO:teuthology.orchestra.run.smithi158.stdout: "osd_session_close": 0, 2024-02-18T11:53:47.083 INFO:teuthology.orchestra.run.smithi158.stdout: "osd_laggy": 0, 2024-02-18T11:53:47.084 INFO:teuthology.orchestra.run.smithi158.stdout: "omap_wr": 0, 2024-02-18T11:53:47.084 INFO:teuthology.orchestra.run.smithi158.stdout: "omap_rd": 2, 2024-02-18T11:53:47.084 INFO:teuthology.orchestra.run.smithi158.stdout: "omap_del": 0 2024-02-18T11:53:47.084 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:47.084 INFO:teuthology.orchestra.run.smithi158.stdout: "oft": { 2024-02-18T11:53:47.084 INFO:teuthology.orchestra.run.smithi158.stdout: "omap_total_objs": 0, 2024-02-18T11:53:47.084 INFO:teuthology.orchestra.run.smithi158.stdout: "omap_total_kv_pairs": 0, 2024-02-18T11:53:47.084 INFO:teuthology.orchestra.run.smithi158.stdout: "omap_total_updates": 0, 2024-02-18T11:53:47.084 INFO:teuthology.orchestra.run.smithi158.stdout: "omap_total_removes": 0 2024-02-18T11:53:47.084 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:47.084 INFO:teuthology.orchestra.run.smithi158.stdout: "osd-slow-ops": { 2024-02-18T11:53:47.084 INFO:teuthology.orchestra.run.smithi158.stdout: "slow_ops_count": 0 2024-02-18T11:53:47.084 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:47.084 INFO:teuthology.orchestra.run.smithi158.stdout: "purge_queue": { 2024-02-18T11:53:47.085 INFO:teuthology.orchestra.run.smithi158.stdout: "pq_executing_ops": 0, 2024-02-18T11:53:47.085 INFO:teuthology.orchestra.run.smithi158.stdout: "pq_executing_ops_high_water": 0, 2024-02-18T11:53:47.085 INFO:teuthology.orchestra.run.smithi158.stdout: "pq_executing": 0, 2024-02-18T11:53:47.085 INFO:teuthology.orchestra.run.smithi158.stdout: "pq_executing_high_water": 0, 2024-02-18T11:53:47.085 INFO:teuthology.orchestra.run.smithi158.stdout: "pq_executed_ops": 0, 2024-02-18T11:53:47.085 INFO:teuthology.orchestra.run.smithi158.stdout: "pq_executed": 0, 2024-02-18T11:53:47.085 INFO:teuthology.orchestra.run.smithi158.stdout: "pq_item_in_journal": 0 2024-02-18T11:53:47.085 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:47.085 INFO:teuthology.orchestra.run.smithi158.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-18T11:53:47.085 INFO:teuthology.orchestra.run.smithi158.stdout: "val": 0, 2024-02-18T11:53:47.085 INFO:teuthology.orchestra.run.smithi158.stdout: "max": 104857600, 2024-02-18T11:53:47.085 INFO:teuthology.orchestra.run.smithi158.stdout: "get_started": 0, 2024-02-18T11:53:47.085 INFO:teuthology.orchestra.run.smithi158.stdout: "get": 4776, 2024-02-18T11:53:47.085 INFO:teuthology.orchestra.run.smithi158.stdout: "get_sum": 1228069418, 2024-02-18T11:53:47.085 INFO:teuthology.orchestra.run.smithi158.stdout: "get_or_fail_fail": 0, 2024-02-18T11:53:47.086 INFO:teuthology.orchestra.run.smithi158.stdout: "get_or_fail_success": 4776, 2024-02-18T11:53:47.086 INFO:teuthology.orchestra.run.smithi158.stdout: "take": 0, 2024-02-18T11:53:47.086 INFO:teuthology.orchestra.run.smithi158.stdout: "take_sum": 0, 2024-02-18T11:53:47.086 INFO:teuthology.orchestra.run.smithi158.stdout: "put": 4776, 2024-02-18T11:53:47.086 INFO:teuthology.orchestra.run.smithi158.stdout: "put_sum": 1228069418, 2024-02-18T11:53:47.086 INFO:teuthology.orchestra.run.smithi158.stdout: "wait": { 2024-02-18T11:53:47.086 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:47.086 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:47.086 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:47.086 INFO:teuthology.orchestra.run.smithi158.stdout: } 2024-02-18T11:53:47.086 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:47.086 INFO:teuthology.orchestra.run.smithi158.stdout: "throttle-objecter_bytes": { 2024-02-18T11:53:47.086 INFO:teuthology.orchestra.run.smithi158.stdout: "val": 0, 2024-02-18T11:53:47.086 INFO:teuthology.orchestra.run.smithi158.stdout: "max": 104857600, 2024-02-18T11:53:47.086 INFO:teuthology.orchestra.run.smithi158.stdout: "get_started": 0, 2024-02-18T11:53:47.087 INFO:teuthology.orchestra.run.smithi158.stdout: "get": 0, 2024-02-18T11:53:47.087 INFO:teuthology.orchestra.run.smithi158.stdout: "get_sum": 0, 2024-02-18T11:53:47.087 INFO:teuthology.orchestra.run.smithi158.stdout: "get_or_fail_fail": 0, 2024-02-18T11:53:47.087 INFO:teuthology.orchestra.run.smithi158.stdout: "get_or_fail_success": 0, 2024-02-18T11:53:47.087 INFO:teuthology.orchestra.run.smithi158.stdout: "take": 4515, 2024-02-18T11:53:47.087 INFO:teuthology.orchestra.run.smithi158.stdout: "take_sum": 1226511406, 2024-02-18T11:53:47.087 INFO:teuthology.orchestra.run.smithi158.stdout: "put": 4515, 2024-02-18T11:53:47.087 INFO:teuthology.orchestra.run.smithi158.stdout: "put_sum": 1226511406, 2024-02-18T11:53:47.087 INFO:teuthology.orchestra.run.smithi158.stdout: "wait": { 2024-02-18T11:53:47.087 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:47.087 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:47.087 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:47.087 INFO:teuthology.orchestra.run.smithi158.stdout: } 2024-02-18T11:53:47.087 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:47.087 INFO:teuthology.orchestra.run.smithi158.stdout: "throttle-objecter_ops": { 2024-02-18T11:53:47.088 INFO:teuthology.orchestra.run.smithi158.stdout: "val": 0, 2024-02-18T11:53:47.088 INFO:teuthology.orchestra.run.smithi158.stdout: "max": 1024, 2024-02-18T11:53:47.088 INFO:teuthology.orchestra.run.smithi158.stdout: "get_started": 0, 2024-02-18T11:53:47.088 INFO:teuthology.orchestra.run.smithi158.stdout: "get": 0, 2024-02-18T11:53:47.088 INFO:teuthology.orchestra.run.smithi158.stdout: "get_sum": 0, 2024-02-18T11:53:47.088 INFO:teuthology.orchestra.run.smithi158.stdout: "get_or_fail_fail": 0, 2024-02-18T11:53:47.088 INFO:teuthology.orchestra.run.smithi158.stdout: "get_or_fail_success": 0, 2024-02-18T11:53:47.088 INFO:teuthology.orchestra.run.smithi158.stdout: "take": 4515, 2024-02-18T11:53:47.088 INFO:teuthology.orchestra.run.smithi158.stdout: "take_sum": 4515, 2024-02-18T11:53:47.088 INFO:teuthology.orchestra.run.smithi158.stdout: "put": 4515, 2024-02-18T11:53:47.088 INFO:teuthology.orchestra.run.smithi158.stdout: "put_sum": 4515, 2024-02-18T11:53:47.088 INFO:teuthology.orchestra.run.smithi158.stdout: "wait": { 2024-02-18T11:53:47.088 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:47.088 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:47.088 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:47.089 INFO:teuthology.orchestra.run.smithi158.stdout: } 2024-02-18T11:53:47.089 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:47.089 INFO:teuthology.orchestra.run.smithi158.stdout: "throttle-write_buf_throttle": { 2024-02-18T11:53:47.089 INFO:teuthology.orchestra.run.smithi158.stdout: "val": 0, 2024-02-18T11:53:47.089 INFO:teuthology.orchestra.run.smithi158.stdout: "max": 3758096384, 2024-02-18T11:53:47.089 INFO:teuthology.orchestra.run.smithi158.stdout: "get_started": 0, 2024-02-18T11:53:47.089 INFO:teuthology.orchestra.run.smithi158.stdout: "get": 0, 2024-02-18T11:53:47.089 INFO:teuthology.orchestra.run.smithi158.stdout: "get_sum": 0, 2024-02-18T11:53:47.089 INFO:teuthology.orchestra.run.smithi158.stdout: "get_or_fail_fail": 0, 2024-02-18T11:53:47.089 INFO:teuthology.orchestra.run.smithi158.stdout: "get_or_fail_success": 0, 2024-02-18T11:53:47.089 INFO:teuthology.orchestra.run.smithi158.stdout: "take": 0, 2024-02-18T11:53:47.089 INFO:teuthology.orchestra.run.smithi158.stdout: "take_sum": 0, 2024-02-18T11:53:47.089 INFO:teuthology.orchestra.run.smithi158.stdout: "put": 0, 2024-02-18T11:53:47.089 INFO:teuthology.orchestra.run.smithi158.stdout: "put_sum": 0, 2024-02-18T11:53:47.089 INFO:teuthology.orchestra.run.smithi158.stdout: "wait": { 2024-02-18T11:53:47.090 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:47.090 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:47.090 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:47.090 INFO:teuthology.orchestra.run.smithi158.stdout: } 2024-02-18T11:53:47.090 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:47.090 INFO:teuthology.orchestra.run.smithi158.stdout: "throttle-write_buf_throttle-0x55dcef3bc960": { 2024-02-18T11:53:47.090 INFO:teuthology.orchestra.run.smithi158.stdout: "val": 0, 2024-02-18T11:53:47.090 INFO:teuthology.orchestra.run.smithi158.stdout: "max": 3758096384, 2024-02-18T11:53:47.090 INFO:teuthology.orchestra.run.smithi158.stdout: "get_started": 0, 2024-02-18T11:53:47.090 INFO:teuthology.orchestra.run.smithi158.stdout: "get": 0, 2024-02-18T11:53:47.090 INFO:teuthology.orchestra.run.smithi158.stdout: "get_sum": 0, 2024-02-18T11:53:47.090 INFO:teuthology.orchestra.run.smithi158.stdout: "get_or_fail_fail": 0, 2024-02-18T11:53:47.090 INFO:teuthology.orchestra.run.smithi158.stdout: "get_or_fail_success": 0, 2024-02-18T11:53:47.090 INFO:teuthology.orchestra.run.smithi158.stdout: "take": 0, 2024-02-18T11:53:47.090 INFO:teuthology.orchestra.run.smithi158.stdout: "take_sum": 0, 2024-02-18T11:53:47.091 INFO:teuthology.orchestra.run.smithi158.stdout: "put": 0, 2024-02-18T11:53:47.091 INFO:teuthology.orchestra.run.smithi158.stdout: "put_sum": 0, 2024-02-18T11:53:47.091 INFO:teuthology.orchestra.run.smithi158.stdout: "wait": { 2024-02-18T11:53:47.091 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:47.091 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:47.091 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:47.091 INFO:teuthology.orchestra.run.smithi158.stdout: } 2024-02-18T11:53:47.091 INFO:teuthology.orchestra.run.smithi158.stdout: } 2024-02-18T11:53:47.091 INFO:teuthology.orchestra.run.smithi158.stdout:} 2024-02-18T11:53:47.242 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:53:47 smithi158 ceph-mon[36626]: pgmap v736: 97 pgs: 97 active+clean; 8.2 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 32 MiB/s wr, 134 op/s 2024-02-18T11:53:47.421 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:53:47 smithi012 ceph-mon[32460]: pgmap v736: 97 pgs: 97 active+clean; 8.2 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 32 MiB/s wr, 134 op/s 2024-02-18T11:53:47.602 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:53:47 smithi188 ceph-mon[36925]: pgmap v736: 97 pgs: 97 active+clean; 8.2 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 32 MiB/s wr, 134 op/s 2024-02-18T11:53:47.943 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-02-18T11:53:47.944 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-02-18T11:53:47.944 DEBUG:tasks.check_counter:Getting stats from k 2024-02-18T11:53:47.944 DEBUG:teuthology.orchestra.run.smithi158:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph daemon mds.k perf dump 2024-02-18T11:53:48.204 INFO:teuthology.orchestra.run.smithi158.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T11:53:49.362 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:53:49 smithi158 ceph-mon[36626]: pgmap v737: 97 pgs: 97 active+clean; 8.2 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 51 MiB/s wr, 213 op/s 2024-02-18T11:53:49.421 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:53:49 smithi012 ceph-mon[32460]: pgmap v737: 97 pgs: 97 active+clean; 8.2 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 51 MiB/s wr, 213 op/s 2024-02-18T11:53:49.606 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:53:49 smithi188 ceph-mon[36925]: pgmap v737: 97 pgs: 97 active+clean; 8.2 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 51 MiB/s wr, 213 op/s 2024-02-18T11:53:49.957 INFO:teuthology.orchestra.run.smithi158.stdout:{ 2024-02-18T11:53:49.957 INFO:teuthology.orchestra.run.smithi158.stdout: "AsyncMessenger::Worker-0": { 2024-02-18T11:53:49.957 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_recv_messages": 0, 2024-02-18T11:53:49.957 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_messages": 2, 2024-02-18T11:53:49.957 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_recv_bytes": 0, 2024-02-18T11:53:49.957 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_bytes": 0, 2024-02-18T11:53:49.957 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_created_connections": 4, 2024-02-18T11:53:49.958 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_active_connections": 0, 2024-02-18T11:53:49.958 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_total_time": 0.001029588, 2024-02-18T11:53:49.958 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_send_time": 0.000000000, 2024-02-18T11:53:49.958 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_recv_time": 0.000100127, 2024-02-18T11:53:49.958 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-18T11:53:49.958 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_messages_queue_lat": { 2024-02-18T11:53:49.958 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:49.958 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:49.958 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:49.958 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:49.958 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_handle_ack_lat": { 2024-02-18T11:53:49.958 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:49.958 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:49.958 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:49.958 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:49.959 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_recv_encrypted_bytes": 0, 2024-02-18T11:53:49.959 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_encrypted_bytes": 0 2024-02-18T11:53:49.959 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:49.959 INFO:teuthology.orchestra.run.smithi158.stdout: "AsyncMessenger::Worker-1": { 2024-02-18T11:53:49.959 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_recv_messages": 2, 2024-02-18T11:53:49.959 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_messages": 192, 2024-02-18T11:53:49.959 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_recv_bytes": 320, 2024-02-18T11:53:49.959 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_bytes": 188864, 2024-02-18T11:53:49.959 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_created_connections": 4, 2024-02-18T11:53:49.959 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_active_connections": 2, 2024-02-18T11:53:49.959 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_total_time": 0.011478536, 2024-02-18T11:53:49.959 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_send_time": 0.008487855, 2024-02-18T11:53:49.959 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_recv_time": 0.000909350, 2024-02-18T11:53:49.959 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-18T11:53:49.959 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_messages_queue_lat": { 2024-02-18T11:53:49.960 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 190, 2024-02-18T11:53:49.960 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.017017429, 2024-02-18T11:53:49.960 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000089565 2024-02-18T11:53:49.960 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:49.960 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_handle_ack_lat": { 2024-02-18T11:53:49.960 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:49.960 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:49.960 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:49.960 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:49.960 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_recv_encrypted_bytes": 320, 2024-02-18T11:53:49.960 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_encrypted_bytes": 188864 2024-02-18T11:53:49.960 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:49.960 INFO:teuthology.orchestra.run.smithi158.stdout: "AsyncMessenger::Worker-2": { 2024-02-18T11:53:49.960 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_recv_messages": 259, 2024-02-18T11:53:49.960 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_messages": 252, 2024-02-18T11:53:49.961 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_recv_bytes": 764144, 2024-02-18T11:53:49.961 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_bytes": 150688, 2024-02-18T11:53:49.961 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_created_connections": 2, 2024-02-18T11:53:49.961 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_active_connections": 1, 2024-02-18T11:53:49.961 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_total_time": 0.046733244, 2024-02-18T11:53:49.961 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_send_time": 0.016136228, 2024-02-18T11:53:49.961 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_recv_time": 0.025258929, 2024-02-18T11:53:49.961 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_fast_dispatch_time": 0.007647791, 2024-02-18T11:53:49.961 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_messages_queue_lat": { 2024-02-18T11:53:49.961 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 252, 2024-02-18T11:53:49.961 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.016690960, 2024-02-18T11:53:49.961 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000066233 2024-02-18T11:53:49.961 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:49.961 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_handle_ack_lat": { 2024-02-18T11:53:49.961 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:49.961 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:49.962 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:49.962 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:49.962 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_recv_encrypted_bytes": 764144, 2024-02-18T11:53:49.962 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_encrypted_bytes": 150688 2024-02-18T11:53:49.962 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:49.962 INFO:teuthology.orchestra.run.smithi158.stdout: "cct": { 2024-02-18T11:53:49.962 INFO:teuthology.orchestra.run.smithi158.stdout: "total_workers": 0, 2024-02-18T11:53:49.962 INFO:teuthology.orchestra.run.smithi158.stdout: "unhealthy_workers": 0 2024-02-18T11:53:49.962 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:49.962 INFO:teuthology.orchestra.run.smithi158.stdout: "mempool": { 2024-02-18T11:53:49.962 INFO:teuthology.orchestra.run.smithi158.stdout: "bloom_filter_bytes": 0, 2024-02-18T11:53:49.962 INFO:teuthology.orchestra.run.smithi158.stdout: "bloom_filter_items": 0, 2024-02-18T11:53:49.962 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_alloc_bytes": 0, 2024-02-18T11:53:49.962 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_alloc_items": 0, 2024-02-18T11:53:49.962 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_data_bytes": 0, 2024-02-18T11:53:49.963 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_data_items": 0, 2024-02-18T11:53:49.963 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-18T11:53:49.963 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_onode_items": 0, 2024-02-18T11:53:49.963 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-18T11:53:49.963 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_meta_items": 0, 2024-02-18T11:53:49.963 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_other_bytes": 0, 2024-02-18T11:53:49.963 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_other_items": 0, 2024-02-18T11:53:49.963 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-18T11:53:49.963 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_buffer_items": 0, 2024-02-18T11:53:49.963 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_extent_bytes": 0, 2024-02-18T11:53:49.963 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_extent_items": 0, 2024-02-18T11:53:49.963 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_blob_bytes": 0, 2024-02-18T11:53:49.963 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_blob_items": 0, 2024-02-18T11:53:49.963 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-18T11:53:49.963 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_shared_blob_items": 0, 2024-02-18T11:53:49.964 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-18T11:53:49.964 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_inline_bl_items": 0, 2024-02-18T11:53:49.964 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_fsck_bytes": 0, 2024-02-18T11:53:49.964 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_fsck_items": 0, 2024-02-18T11:53:49.964 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_txc_bytes": 0, 2024-02-18T11:53:49.964 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_txc_items": 0, 2024-02-18T11:53:49.964 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-18T11:53:49.964 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_writing_deferred_items": 0, 2024-02-18T11:53:49.964 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_writing_bytes": 0, 2024-02-18T11:53:49.964 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_writing_items": 0, 2024-02-18T11:53:49.964 INFO:teuthology.orchestra.run.smithi158.stdout: "bluefs_bytes": 0, 2024-02-18T11:53:49.964 INFO:teuthology.orchestra.run.smithi158.stdout: "bluefs_items": 0, 2024-02-18T11:53:49.964 INFO:teuthology.orchestra.run.smithi158.stdout: "bluefs_file_reader_bytes": 0, 2024-02-18T11:53:49.964 INFO:teuthology.orchestra.run.smithi158.stdout: "bluefs_file_reader_items": 0, 2024-02-18T11:53:49.964 INFO:teuthology.orchestra.run.smithi158.stdout: "bluefs_file_writer_bytes": 0, 2024-02-18T11:53:49.965 INFO:teuthology.orchestra.run.smithi158.stdout: "bluefs_file_writer_items": 0, 2024-02-18T11:53:49.965 INFO:teuthology.orchestra.run.smithi158.stdout: "buffer_anon_bytes": 148682, 2024-02-18T11:53:49.965 INFO:teuthology.orchestra.run.smithi158.stdout: "buffer_anon_items": 29, 2024-02-18T11:53:49.965 INFO:teuthology.orchestra.run.smithi158.stdout: "buffer_meta_bytes": 0, 2024-02-18T11:53:49.965 INFO:teuthology.orchestra.run.smithi158.stdout: "buffer_meta_items": 0, 2024-02-18T11:53:49.965 INFO:teuthology.orchestra.run.smithi158.stdout: "osd_bytes": 0, 2024-02-18T11:53:49.965 INFO:teuthology.orchestra.run.smithi158.stdout: "osd_items": 0, 2024-02-18T11:53:49.965 INFO:teuthology.orchestra.run.smithi158.stdout: "osd_mapbl_bytes": 0, 2024-02-18T11:53:49.965 INFO:teuthology.orchestra.run.smithi158.stdout: "osd_mapbl_items": 0, 2024-02-18T11:53:49.965 INFO:teuthology.orchestra.run.smithi158.stdout: "osd_pglog_bytes": 0, 2024-02-18T11:53:49.965 INFO:teuthology.orchestra.run.smithi158.stdout: "osd_pglog_items": 0, 2024-02-18T11:53:49.965 INFO:teuthology.orchestra.run.smithi158.stdout: "osdmap_bytes": 0, 2024-02-18T11:53:49.965 INFO:teuthology.orchestra.run.smithi158.stdout: "osdmap_items": 0, 2024-02-18T11:53:49.965 INFO:teuthology.orchestra.run.smithi158.stdout: "osdmap_mapping_bytes": 0, 2024-02-18T11:53:49.965 INFO:teuthology.orchestra.run.smithi158.stdout: "osdmap_mapping_items": 0, 2024-02-18T11:53:49.966 INFO:teuthology.orchestra.run.smithi158.stdout: "pgmap_bytes": 0, 2024-02-18T11:53:49.966 INFO:teuthology.orchestra.run.smithi158.stdout: "pgmap_items": 0, 2024-02-18T11:53:49.966 INFO:teuthology.orchestra.run.smithi158.stdout: "mds_co_bytes": 928, 2024-02-18T11:53:49.966 INFO:teuthology.orchestra.run.smithi158.stdout: "mds_co_items": 2, 2024-02-18T11:53:49.966 INFO:teuthology.orchestra.run.smithi158.stdout: "unittest_1_bytes": 0, 2024-02-18T11:53:49.966 INFO:teuthology.orchestra.run.smithi158.stdout: "unittest_1_items": 0, 2024-02-18T11:53:49.966 INFO:teuthology.orchestra.run.smithi158.stdout: "unittest_2_bytes": 0, 2024-02-18T11:53:49.966 INFO:teuthology.orchestra.run.smithi158.stdout: "unittest_2_items": 0 2024-02-18T11:53:49.966 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-02-18T11:53:49.966 INFO:teuthology.orchestra.run.smithi158.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-18T11:53:49.966 INFO:teuthology.orchestra.run.smithi158.stdout: "val": 0, 2024-02-18T11:53:49.966 INFO:teuthology.orchestra.run.smithi158.stdout: "max": 104857600, 2024-02-18T11:53:49.966 INFO:teuthology.orchestra.run.smithi158.stdout: "get_started": 0, 2024-02-18T11:53:49.966 INFO:teuthology.orchestra.run.smithi158.stdout: "get": 257, 2024-02-18T11:53:49.966 INFO:teuthology.orchestra.run.smithi158.stdout: "get_sum": 725142, 2024-02-18T11:53:49.966 INFO:teuthology.orchestra.run.smithi158.stdout: "get_or_fail_fail": 0, 2024-02-18T11:53:49.967 INFO:teuthology.orchestra.run.smithi158.stdout: "get_or_fail_success": 257, 2024-02-18T11:53:49.967 INFO:teuthology.orchestra.run.smithi158.stdout: "take": 0, 2024-02-18T11:53:49.967 INFO:teuthology.orchestra.run.smithi158.stdout: "take_sum": 0, 2024-02-18T11:53:49.967 INFO:teuthology.orchestra.run.smithi158.stdout: "put": 257, 2024-02-18T11:53:49.967 INFO:teuthology.orchestra.run.smithi158.stdout: "put_sum": 725142, 2024-02-18T11:53:49.967 INFO:teuthology.orchestra.run.smithi158.stdout: "wait": { 2024-02-18T11:53:49.967 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-02-18T11:53:49.967 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-02-18T11:53:49.967 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-02-18T11:53:49.967 INFO:teuthology.orchestra.run.smithi158.stdout: } 2024-02-18T11:53:49.967 INFO:teuthology.orchestra.run.smithi158.stdout: } 2024-02-18T11:53:49.967 INFO:teuthology.orchestra.run.smithi158.stdout:} 2024-02-18T11:53:50.693 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-02-18T11:53:50.693 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-02-18T11:53:50.693 DEBUG:tasks.check_counter:Getting stats from c 2024-02-18T11:53:50.693 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph daemon mds.c perf dump 2024-02-18T11:53:50.985 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.c/config 2024-02-18T11:53:51.376 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:53:51 smithi188 ceph-mon[36925]: pgmap v738: 97 pgs: 97 active+clean; 8.2 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 64 MiB/s wr, 262 op/s 2024-02-18T11:53:51.421 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:53:51 smithi012 ceph-mon[32460]: pgmap v738: 97 pgs: 97 active+clean; 8.2 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 64 MiB/s wr, 262 op/s 2024-02-18T11:53:51.492 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:53:51 smithi158 ceph-mon[36626]: pgmap v738: 97 pgs: 97 active+clean; 8.2 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 64 MiB/s wr, 262 op/s 2024-02-18T11:53:53.039 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-02-18T11:53:53.040 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-0": { 2024-02-18T11:53:53.040 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 413, 2024-02-18T11:53:53.040 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 548, 2024-02-18T11:53:53.040 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 81598, 2024-02-18T11:53:53.040 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 229513, 2024-02-18T11:53:53.040 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 8, 2024-02-18T11:53:53.040 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 2, 2024-02-18T11:53:53.040 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 0.060323667, 2024-02-18T11:53:53.040 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 0.021750754, 2024-02-18T11:53:53.040 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 0.030353265, 2024-02-18T11:53:53.040 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 0.006214600, 2024-02-18T11:53:53.040 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-02-18T11:53:53.041 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 548, 2024-02-18T11:53:53.041 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.042427148, 2024-02-18T11:53:53.041 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000077421 2024-02-18T11:53:53.041 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:53.041 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-02-18T11:53:53.041 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 913, 2024-02-18T11:53:53.041 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000355358, 2024-02-18T11:53:53.041 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000389 2024-02-18T11:53:53.041 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:53.041 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 0, 2024-02-18T11:53:53.041 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 0 2024-02-18T11:53:53.041 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:53.041 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-1": { 2024-02-18T11:53:53.042 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 27, 2024-02-18T11:53:53.042 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 209, 2024-02-18T11:53:53.042 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 5475, 2024-02-18T11:53:53.042 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 411980, 2024-02-18T11:53:53.042 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 9, 2024-02-18T11:53:53.042 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 4, 2024-02-18T11:53:53.042 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 0.018461856, 2024-02-18T11:53:53.042 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 0.010400355, 2024-02-18T11:53:53.042 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 0.005598530, 2024-02-18T11:53:53.042 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 0.000328463, 2024-02-18T11:53:53.042 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-02-18T11:53:53.042 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 207, 2024-02-18T11:53:53.042 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.021539023, 2024-02-18T11:53:53.042 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000104053 2024-02-18T11:53:53.043 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:53.043 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-02-18T11:53:53.043 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 12, 2024-02-18T11:53:53.043 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000010088, 2024-02-18T11:53:53.043 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000840 2024-02-18T11:53:53.043 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:53.043 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 160, 2024-02-18T11:53:53.043 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 398672 2024-02-18T11:53:53.043 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:53.043 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-2": { 2024-02-18T11:53:53.043 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 310, 2024-02-18T11:53:53.043 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 308, 2024-02-18T11:53:53.043 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 791810, 2024-02-18T11:53:53.043 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 293686, 2024-02-18T11:53:53.044 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 13, 2024-02-18T11:53:53.044 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 8, 2024-02-18T11:53:53.044 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 0.050215095, 2024-02-18T11:53:53.044 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 0.017122742, 2024-02-18T11:53:53.047 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 0.027042596, 2024-02-18T11:53:53.048 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 0.008094310, 2024-02-18T11:53:53.048 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-02-18T11:53:53.048 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 306, 2024-02-18T11:53:53.048 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.017739858, 2024-02-18T11:53:53.048 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000057973 2024-02-18T11:53:53.048 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:53.049 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-02-18T11:53:53.049 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 11, 2024-02-18T11:53:53.049 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000001195, 2024-02-18T11:53:53.049 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000108 2024-02-18T11:53:53.049 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:53.049 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 789840, 2024-02-18T11:53:53.049 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 252032 2024-02-18T11:53:53.049 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:53.049 INFO:teuthology.orchestra.run.smithi188.stdout: "cct": { 2024-02-18T11:53:53.049 INFO:teuthology.orchestra.run.smithi188.stdout: "total_workers": 1, 2024-02-18T11:53:53.049 INFO:teuthology.orchestra.run.smithi188.stdout: "unhealthy_workers": 0 2024-02-18T11:53:53.049 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:53.049 INFO:teuthology.orchestra.run.smithi188.stdout: "finisher-MDSRank": { 2024-02-18T11:53:53.050 INFO:teuthology.orchestra.run.smithi188.stdout: "queue_len": 0, 2024-02-18T11:53:53.050 INFO:teuthology.orchestra.run.smithi188.stdout: "complete_latency": { 2024-02-18T11:53:53.050 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 49, 2024-02-18T11:53:53.050 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.001501558, 2024-02-18T11:53:53.050 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000030644 2024-02-18T11:53:53.050 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-02-18T11:53:53.050 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:53.050 INFO:teuthology.orchestra.run.smithi188.stdout: "finisher-PurgeQueue": { 2024-02-18T11:53:53.050 INFO:teuthology.orchestra.run.smithi188.stdout: "queue_len": 0, 2024-02-18T11:53:53.050 INFO:teuthology.orchestra.run.smithi188.stdout: "complete_latency": { 2024-02-18T11:53:53.050 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 2, 2024-02-18T11:53:53.050 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000007947, 2024-02-18T11:53:53.050 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000003973 2024-02-18T11:53:53.050 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-02-18T11:53:53.050 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:53.051 INFO:teuthology.orchestra.run.smithi188.stdout: "mds": { 2024-02-18T11:53:53.051 INFO:teuthology.orchestra.run.smithi188.stdout: "request": 1, 2024-02-18T11:53:53.051 INFO:teuthology.orchestra.run.smithi188.stdout: "reply": 0, 2024-02-18T11:53:53.051 INFO:teuthology.orchestra.run.smithi188.stdout: "reply_latency": { 2024-02-18T11:53:53.051 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:53.051 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:53.051 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:53.051 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:53.051 INFO:teuthology.orchestra.run.smithi188.stdout: "slow_reply": 0, 2024-02-18T11:53:53.051 INFO:teuthology.orchestra.run.smithi188.stdout: "forward": 1, 2024-02-18T11:53:53.051 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_fetch_complete": 0, 2024-02-18T11:53:53.051 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_fetch_keys": 0, 2024-02-18T11:53:53.051 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_commit": 11, 2024-02-18T11:53:53.051 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_split": 0, 2024-02-18T11:53:53.052 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_merge": 0, 2024-02-18T11:53:53.052 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes": 10, 2024-02-18T11:53:53.052 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_top": 0, 2024-02-18T11:53:53.052 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_bottom": 10, 2024-02-18T11:53:53.052 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_pin_tail": 0, 2024-02-18T11:53:53.052 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_pinned": 10, 2024-02-18T11:53:53.052 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_expired": 0, 2024-02-18T11:53:53.052 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_with_caps": 0, 2024-02-18T11:53:53.052 INFO:teuthology.orchestra.run.smithi188.stdout: "caps": 0, 2024-02-18T11:53:53.052 INFO:teuthology.orchestra.run.smithi188.stdout: "subtrees": 1, 2024-02-18T11:53:53.052 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse": 0, 2024-02-18T11:53:53.052 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_hit": 0, 2024-02-18T11:53:53.053 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_forward": 0, 2024-02-18T11:53:53.053 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_discover": 0, 2024-02-18T11:53:53.053 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_dir_fetch": 0, 2024-02-18T11:53:53.053 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_remote_ino": 0, 2024-02-18T11:53:53.053 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_lock": 0, 2024-02-18T11:53:53.053 INFO:teuthology.orchestra.run.smithi188.stdout: "load_cent": 0, 2024-02-18T11:53:53.053 INFO:teuthology.orchestra.run.smithi188.stdout: "q": 0, 2024-02-18T11:53:53.053 INFO:teuthology.orchestra.run.smithi188.stdout: "exported": 0, 2024-02-18T11:53:53.053 INFO:teuthology.orchestra.run.smithi188.stdout: "exported_inodes": 0, 2024-02-18T11:53:53.053 INFO:teuthology.orchestra.run.smithi188.stdout: "imported": 0, 2024-02-18T11:53:53.053 INFO:teuthology.orchestra.run.smithi188.stdout: "imported_inodes": 0, 2024-02-18T11:53:53.053 INFO:teuthology.orchestra.run.smithi188.stdout: "openino_dir_fetch": 0, 2024-02-18T11:53:53.054 INFO:teuthology.orchestra.run.smithi188.stdout: "openino_backtrace_fetch": 0, 2024-02-18T11:53:53.054 INFO:teuthology.orchestra.run.smithi188.stdout: "openino_peer_discover": 1, 2024-02-18T11:53:53.054 INFO:teuthology.orchestra.run.smithi188.stdout: "root_rfiles": 4, 2024-02-18T11:53:53.054 INFO:teuthology.orchestra.run.smithi188.stdout: "root_rbytes": 234, 2024-02-18T11:53:53.054 INFO:teuthology.orchestra.run.smithi188.stdout: "root_rsnaps": 0, 2024-02-18T11:53:53.054 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_backtrace_fetch": 0, 2024-02-18T11:53:53.054 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_set_tag": 0, 2024-02-18T11:53:53.054 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_backtrace_repaired": 0, 2024-02-18T11:53:53.055 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_inotable_repaired": 0, 2024-02-18T11:53:53.055 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_dir_inodes": 0, 2024-02-18T11:53:53.055 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_dir_base_inodes": 0, 2024-02-18T11:53:53.055 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_dirfrag_rstats": 0, 2024-02-18T11:53:53.055 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_file_inodes": 0, 2024-02-18T11:53:53.055 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_inode_file_caps": 0, 2024-02-18T11:53:53.055 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_revoke": 0, 2024-02-18T11:53:53.055 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_grant": 0, 2024-02-18T11:53:53.055 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_trunc": 0, 2024-02-18T11:53:53.055 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-02-18T11:53:53.055 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_flush_ack": 0, 2024-02-18T11:53:53.055 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_caps": 0, 2024-02-18T11:53:53.055 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_caps_dirty": 0, 2024-02-18T11:53:53.055 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_cap_release": 0, 2024-02-18T11:53:53.056 INFO:teuthology.orchestra.run.smithi188.stdout: "process_request_cap_release": 0 2024-02-18T11:53:53.056 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:53.056 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_cache": { 2024-02-18T11:53:53.056 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_update": 0, 2024-02-18T11:53:53.056 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_update_receipt": 0, 2024-02-18T11:53:53.056 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_try_discover": 0, 2024-02-18T11:53:53.056 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_send_discover": 1, 2024-02-18T11:53:53.056 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_handle_discover": 0, 2024-02-18T11:53:53.056 INFO:teuthology.orchestra.run.smithi188.stdout: "num_strays": 0, 2024-02-18T11:53:53.056 INFO:teuthology.orchestra.run.smithi188.stdout: "num_strays_delayed": 0, 2024-02-18T11:53:53.056 INFO:teuthology.orchestra.run.smithi188.stdout: "num_strays_enqueuing": 0, 2024-02-18T11:53:53.056 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_created": 0, 2024-02-18T11:53:53.056 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_enqueued": 0, 2024-02-18T11:53:53.056 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_reintegrated": 0, 2024-02-18T11:53:53.057 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_migrated": 0, 2024-02-18T11:53:53.057 INFO:teuthology.orchestra.run.smithi188.stdout: "num_recovering_processing": 0, 2024-02-18T11:53:53.057 INFO:teuthology.orchestra.run.smithi188.stdout: "num_recovering_enqueued": 0, 2024-02-18T11:53:53.057 INFO:teuthology.orchestra.run.smithi188.stdout: "num_recovering_prioritized": 0, 2024-02-18T11:53:53.057 INFO:teuthology.orchestra.run.smithi188.stdout: "recovery_started": 0, 2024-02-18T11:53:53.057 INFO:teuthology.orchestra.run.smithi188.stdout: "recovery_completed": 0, 2024-02-18T11:53:53.057 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_enqueue_scrub": 0, 2024-02-18T11:53:53.057 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_exportdir": 0, 2024-02-18T11:53:53.057 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_flush": 0, 2024-02-18T11:53:53.057 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_fragmentdir": 0, 2024-02-18T11:53:53.057 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_fragstats": 0, 2024-02-18T11:53:53.057 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_inodestats": 0 2024-02-18T11:53:53.057 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:53.057 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_log": { 2024-02-18T11:53:53.057 INFO:teuthology.orchestra.run.smithi188.stdout: "evlrg": 0, 2024-02-18T11:53:53.058 INFO:teuthology.orchestra.run.smithi188.stdout: "evadd": 5, 2024-02-18T11:53:53.058 INFO:teuthology.orchestra.run.smithi188.stdout: "evex": 0, 2024-02-18T11:53:53.058 INFO:teuthology.orchestra.run.smithi188.stdout: "evtrm": 0, 2024-02-18T11:53:53.058 INFO:teuthology.orchestra.run.smithi188.stdout: "ev": 5, 2024-02-18T11:53:53.058 INFO:teuthology.orchestra.run.smithi188.stdout: "evexg": 0, 2024-02-18T11:53:53.058 INFO:teuthology.orchestra.run.smithi188.stdout: "evexd": 0, 2024-02-18T11:53:53.058 INFO:teuthology.orchestra.run.smithi188.stdout: "segadd": 2, 2024-02-18T11:53:53.058 INFO:teuthology.orchestra.run.smithi188.stdout: "segex": 0, 2024-02-18T11:53:53.058 INFO:teuthology.orchestra.run.smithi188.stdout: "segtrm": 0, 2024-02-18T11:53:53.058 INFO:teuthology.orchestra.run.smithi188.stdout: "seg": 2, 2024-02-18T11:53:53.058 INFO:teuthology.orchestra.run.smithi188.stdout: "segmjr": 2, 2024-02-18T11:53:53.058 INFO:teuthology.orchestra.run.smithi188.stdout: "segexg": 0, 2024-02-18T11:53:53.059 INFO:teuthology.orchestra.run.smithi188.stdout: "segexd": 0, 2024-02-18T11:53:53.059 INFO:teuthology.orchestra.run.smithi188.stdout: "expos": 4194304, 2024-02-18T11:53:53.059 INFO:teuthology.orchestra.run.smithi188.stdout: "wrpos": 4195732, 2024-02-18T11:53:53.059 INFO:teuthology.orchestra.run.smithi188.stdout: "rdpos": 0, 2024-02-18T11:53:53.059 INFO:teuthology.orchestra.run.smithi188.stdout: "jlat": { 2024-02-18T11:53:53.059 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 4, 2024-02-18T11:53:53.059 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.007796351, 2024-02-18T11:53:53.059 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.001949087 2024-02-18T11:53:53.059 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:53.059 INFO:teuthology.orchestra.run.smithi188.stdout: "replayed": 0 2024-02-18T11:53:53.059 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:53.059 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_mem": { 2024-02-18T11:53:53.059 INFO:teuthology.orchestra.run.smithi188.stdout: "ino": 13, 2024-02-18T11:53:53.059 INFO:teuthology.orchestra.run.smithi188.stdout: "ino+": 13, 2024-02-18T11:53:53.060 INFO:teuthology.orchestra.run.smithi188.stdout: "ino-": 0, 2024-02-18T11:53:53.060 INFO:teuthology.orchestra.run.smithi188.stdout: "dir": 11, 2024-02-18T11:53:53.060 INFO:teuthology.orchestra.run.smithi188.stdout: "dir+": 12, 2024-02-18T11:53:53.060 INFO:teuthology.orchestra.run.smithi188.stdout: "dir-": 1, 2024-02-18T11:53:53.060 INFO:teuthology.orchestra.run.smithi188.stdout: "dn": 10, 2024-02-18T11:53:53.060 INFO:teuthology.orchestra.run.smithi188.stdout: "dn+": 10, 2024-02-18T11:53:53.060 INFO:teuthology.orchestra.run.smithi188.stdout: "dn-": 0, 2024-02-18T11:53:53.060 INFO:teuthology.orchestra.run.smithi188.stdout: "cap": 0, 2024-02-18T11:53:53.060 INFO:teuthology.orchestra.run.smithi188.stdout: "cap+": 0, 2024-02-18T11:53:53.060 INFO:teuthology.orchestra.run.smithi188.stdout: "cap-": 0, 2024-02-18T11:53:53.060 INFO:teuthology.orchestra.run.smithi188.stdout: "rss": 51336, 2024-02-18T11:53:53.060 INFO:teuthology.orchestra.run.smithi188.stdout: "heap": 207228 2024-02-18T11:53:53.060 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:53.060 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_server": { 2024-02-18T11:53:53.061 INFO:teuthology.orchestra.run.smithi188.stdout: "dispatch_client_request": 1, 2024-02-18T11:53:53.061 INFO:teuthology.orchestra.run.smithi188.stdout: "dispatch_server_request": 0, 2024-02-18T11:53:53.061 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_request": 1, 2024-02-18T11:53:53.061 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_session": 15, 2024-02-18T11:53:53.061 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_peer_request": 0, 2024-02-18T11:53:53.061 INFO:teuthology.orchestra.run.smithi188.stdout: "req_create_latency": { 2024-02-18T11:53:53.061 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:53.061 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:53.061 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:53.061 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:53.061 INFO:teuthology.orchestra.run.smithi188.stdout: "req_getattr_latency": { 2024-02-18T11:53:53.061 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:53.061 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:53.062 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:53.062 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:53.062 INFO:teuthology.orchestra.run.smithi188.stdout: "req_getfilelock_latency": { 2024-02-18T11:53:53.062 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:53.062 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:53.062 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:53.062 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:53.062 INFO:teuthology.orchestra.run.smithi188.stdout: "req_link_latency": { 2024-02-18T11:53:53.062 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:53.062 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:53.062 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:53.062 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:53.063 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookup_latency": { 2024-02-18T11:53:53.063 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:53.063 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:53.063 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:53.063 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:53.063 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookuphash_latency": { 2024-02-18T11:53:53.063 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:53.063 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:53.063 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:53.063 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:53.063 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupino_latency": { 2024-02-18T11:53:53.063 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:53.063 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:53.064 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:53.064 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:53.064 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupname_latency": { 2024-02-18T11:53:53.064 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:53.064 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:53.064 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:53.064 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:53.064 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupparent_latency": { 2024-02-18T11:53:53.064 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:53.064 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:53.064 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:53.064 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:53.064 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupsnap_latency": { 2024-02-18T11:53:53.065 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:53.065 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:53.065 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:53.065 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:53.065 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lssnap_latency": { 2024-02-18T11:53:53.065 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:53.065 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:53.065 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:53.065 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:53.065 INFO:teuthology.orchestra.run.smithi188.stdout: "req_mkdir_latency": { 2024-02-18T11:53:53.065 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:53.065 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:53.065 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:53.065 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:53.066 INFO:teuthology.orchestra.run.smithi188.stdout: "req_mknod_latency": { 2024-02-18T11:53:53.066 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:53.066 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:53.066 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:53.067 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:53.067 INFO:teuthology.orchestra.run.smithi188.stdout: "req_mksnap_latency": { 2024-02-18T11:53:53.067 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:53.067 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:53.067 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:53.067 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:53.067 INFO:teuthology.orchestra.run.smithi188.stdout: "req_open_latency": { 2024-02-18T11:53:53.067 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:53.067 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:53.067 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:53.067 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:53.067 INFO:teuthology.orchestra.run.smithi188.stdout: "req_readdir_latency": { 2024-02-18T11:53:53.067 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:53.067 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:53.067 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:53.068 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:53.068 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rename_latency": { 2024-02-18T11:53:53.068 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:53.068 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:53.068 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:53.068 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:53.068 INFO:teuthology.orchestra.run.smithi188.stdout: "req_renamesnap_latency": { 2024-02-18T11:53:53.068 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:53.068 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:53.068 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:53.068 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:53.068 INFO:teuthology.orchestra.run.smithi188.stdout: "req_snapdiff_latency": { 2024-02-18T11:53:53.068 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:53.068 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:53.069 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:53.069 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:53.069 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rmdir_latency": { 2024-02-18T11:53:53.069 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:53.069 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:53.069 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:53.069 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:53.069 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rmsnap_latency": { 2024-02-18T11:53:53.069 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:53.069 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:53.069 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:53.069 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:53.069 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rmxattr_latency": { 2024-02-18T11:53:53.069 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:53.070 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:53.070 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:53.070 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:53.070 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setattr_latency": { 2024-02-18T11:53:53.070 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:53.070 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:53.070 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:53.070 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:53.070 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setdirlayout_latency": { 2024-02-18T11:53:53.070 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:53.070 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:53.070 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:53.070 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:53.070 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setfilelock_latency": { 2024-02-18T11:53:53.071 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:53.071 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:53.071 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:53.071 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:53.071 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setlayout_latency": { 2024-02-18T11:53:53.071 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:53.071 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:53.071 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:53.071 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:53.071 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setxattr_latency": { 2024-02-18T11:53:53.071 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:53.071 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:53.071 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:53.071 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:53.071 INFO:teuthology.orchestra.run.smithi188.stdout: "req_symlink_latency": { 2024-02-18T11:53:53.072 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:53.072 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:53.072 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:53.072 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:53.072 INFO:teuthology.orchestra.run.smithi188.stdout: "req_unlink_latency": { 2024-02-18T11:53:53.072 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:53.072 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:53.072 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:53.072 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:53.072 INFO:teuthology.orchestra.run.smithi188.stdout: "cap_revoke_eviction": 0, 2024-02-18T11:53:53.072 INFO:teuthology.orchestra.run.smithi188.stdout: "cap_acquisition_throttle": 0, 2024-02-18T11:53:53.072 INFO:teuthology.orchestra.run.smithi188.stdout: "req_getvxattr_latency": { 2024-02-18T11:53:53.072 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:53.072 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:53.073 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:53.073 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-02-18T11:53:53.073 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:53.073 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_sessions": { 2024-02-18T11:53:53.073 INFO:teuthology.orchestra.run.smithi188.stdout: "session_count": 1, 2024-02-18T11:53:53.073 INFO:teuthology.orchestra.run.smithi188.stdout: "session_add": 2, 2024-02-18T11:53:53.073 INFO:teuthology.orchestra.run.smithi188.stdout: "session_remove": 1, 2024-02-18T11:53:53.073 INFO:teuthology.orchestra.run.smithi188.stdout: "sessions_open": 1, 2024-02-18T11:53:53.073 INFO:teuthology.orchestra.run.smithi188.stdout: "sessions_stale": 0, 2024-02-18T11:53:53.073 INFO:teuthology.orchestra.run.smithi188.stdout: "total_load": 0, 2024-02-18T11:53:53.073 INFO:teuthology.orchestra.run.smithi188.stdout: "average_load": 0, 2024-02-18T11:53:53.073 INFO:teuthology.orchestra.run.smithi188.stdout: "avg_session_uptime": 792, 2024-02-18T11:53:53.073 INFO:teuthology.orchestra.run.smithi188.stdout: "mdthresh_evicted": 0 2024-02-18T11:53:53.073 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:53.073 INFO:teuthology.orchestra.run.smithi188.stdout: "mempool": { 2024-02-18T11:53:53.074 INFO:teuthology.orchestra.run.smithi188.stdout: "bloom_filter_bytes": 0, 2024-02-18T11:53:53.074 INFO:teuthology.orchestra.run.smithi188.stdout: "bloom_filter_items": 0, 2024-02-18T11:53:53.074 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_alloc_bytes": 0, 2024-02-18T11:53:53.074 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_alloc_items": 0, 2024-02-18T11:53:53.074 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_data_bytes": 0, 2024-02-18T11:53:53.074 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_data_items": 0, 2024-02-18T11:53:53.074 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-18T11:53:53.074 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_onode_items": 0, 2024-02-18T11:53:53.074 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-18T11:53:53.074 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_meta_items": 0, 2024-02-18T11:53:53.074 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_other_bytes": 0, 2024-02-18T11:53:53.074 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_other_items": 0, 2024-02-18T11:53:53.074 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-18T11:53:53.074 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_buffer_items": 0, 2024-02-18T11:53:53.075 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_extent_bytes": 0, 2024-02-18T11:53:53.075 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_extent_items": 0, 2024-02-18T11:53:53.075 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_blob_bytes": 0, 2024-02-18T11:53:53.075 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_blob_items": 0, 2024-02-18T11:53:53.075 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-18T11:53:53.075 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_shared_blob_items": 0, 2024-02-18T11:53:53.075 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-18T11:53:53.075 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_inline_bl_items": 0, 2024-02-18T11:53:53.075 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_fsck_bytes": 0, 2024-02-18T11:53:53.075 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_fsck_items": 0, 2024-02-18T11:53:53.075 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_txc_bytes": 0, 2024-02-18T11:53:53.075 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_txc_items": 0, 2024-02-18T11:53:53.075 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-18T11:53:53.075 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_deferred_items": 0, 2024-02-18T11:53:53.075 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_bytes": 0, 2024-02-18T11:53:53.076 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_items": 0, 2024-02-18T11:53:53.076 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_bytes": 0, 2024-02-18T11:53:53.076 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_items": 0, 2024-02-18T11:53:53.076 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_reader_bytes": 0, 2024-02-18T11:53:53.076 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_reader_items": 0, 2024-02-18T11:53:53.076 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_writer_bytes": 0, 2024-02-18T11:53:53.076 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_writer_items": 0, 2024-02-18T11:53:53.076 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_anon_bytes": 222533, 2024-02-18T11:53:53.076 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_anon_items": 72, 2024-02-18T11:53:53.076 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_meta_bytes": 0, 2024-02-18T11:53:53.076 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_meta_items": 0, 2024-02-18T11:53:53.076 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_bytes": 0, 2024-02-18T11:53:53.076 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_items": 0, 2024-02-18T11:53:53.076 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_mapbl_bytes": 0, 2024-02-18T11:53:53.077 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_mapbl_items": 0, 2024-02-18T11:53:53.077 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_pglog_bytes": 0, 2024-02-18T11:53:53.077 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_pglog_items": 0, 2024-02-18T11:53:53.077 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_bytes": 7528, 2024-02-18T11:53:53.077 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_items": 158, 2024-02-18T11:53:53.077 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_mapping_bytes": 0, 2024-02-18T11:53:53.077 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_mapping_items": 0, 2024-02-18T11:53:53.077 INFO:teuthology.orchestra.run.smithi188.stdout: "pgmap_bytes": 0, 2024-02-18T11:53:53.077 INFO:teuthology.orchestra.run.smithi188.stdout: "pgmap_items": 0, 2024-02-18T11:53:53.077 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_co_bytes": 50040, 2024-02-18T11:53:53.077 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_co_items": 198, 2024-02-18T11:53:53.077 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_1_bytes": 0, 2024-02-18T11:53:53.077 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_1_items": 0, 2024-02-18T11:53:53.077 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_2_bytes": 0, 2024-02-18T11:53:53.078 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_2_items": 0 2024-02-18T11:53:53.078 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:53.078 INFO:teuthology.orchestra.run.smithi188.stdout: "objecter": { 2024-02-18T11:53:53.078 INFO:teuthology.orchestra.run.smithi188.stdout: "op_active": 0, 2024-02-18T11:53:53.078 INFO:teuthology.orchestra.run.smithi188.stdout: "op_laggy": 0, 2024-02-18T11:53:53.078 INFO:teuthology.orchestra.run.smithi188.stdout: "op_send": 40, 2024-02-18T11:53:53.078 INFO:teuthology.orchestra.run.smithi188.stdout: "op_send_bytes": 11730, 2024-02-18T11:53:53.078 INFO:teuthology.orchestra.run.smithi188.stdout: "op_resend": 33, 2024-02-18T11:53:53.078 INFO:teuthology.orchestra.run.smithi188.stdout: "op_reply": 40, 2024-02-18T11:53:53.078 INFO:teuthology.orchestra.run.smithi188.stdout: "op_latency": { 2024-02-18T11:53:53.078 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 40, 2024-02-18T11:53:53.078 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.232995961, 2024-02-18T11:53:53.078 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.005824899 2024-02-18T11:53:53.078 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:53.078 INFO:teuthology.orchestra.run.smithi188.stdout: "op_inflight": 0, 2024-02-18T11:53:53.079 INFO:teuthology.orchestra.run.smithi188.stdout: "oplen_avg": { 2024-02-18T11:53:53.079 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 40, 2024-02-18T11:53:53.079 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 62 2024-02-18T11:53:53.079 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:53.079 INFO:teuthology.orchestra.run.smithi188.stdout: "op": 40, 2024-02-18T11:53:53.079 INFO:teuthology.orchestra.run.smithi188.stdout: "op_r": 0, 2024-02-18T11:53:53.079 INFO:teuthology.orchestra.run.smithi188.stdout: "op_w": 40, 2024-02-18T11:53:53.079 INFO:teuthology.orchestra.run.smithi188.stdout: "op_rmw": 0, 2024-02-18T11:53:53.079 INFO:teuthology.orchestra.run.smithi188.stdout: "op_pg": 0, 2024-02-18T11:53:53.079 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_stat": 0, 2024-02-18T11:53:53.079 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_create": 10, 2024-02-18T11:53:53.079 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_read": 0, 2024-02-18T11:53:53.079 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_write": 4, 2024-02-18T11:53:53.079 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_writefull": 7, 2024-02-18T11:53:53.079 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_writesame": 0, 2024-02-18T11:53:53.080 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_append": 0, 2024-02-18T11:53:53.080 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_zero": 0, 2024-02-18T11:53:53.080 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_truncate": 0, 2024-02-18T11:53:53.080 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_delete": 6, 2024-02-18T11:53:53.080 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_mapext": 0, 2024-02-18T11:53:53.080 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_sparse_read": 0, 2024-02-18T11:53:53.080 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_clonerange": 0, 2024-02-18T11:53:53.080 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_getxattr": 0, 2024-02-18T11:53:53.080 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_setxattr": 20, 2024-02-18T11:53:53.080 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_cmpxattr": 0, 2024-02-18T11:53:53.080 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_rmxattr": 0, 2024-02-18T11:53:53.080 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_resetxattrs": 0, 2024-02-18T11:53:53.080 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_call": 0, 2024-02-18T11:53:53.080 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_watch": 0, 2024-02-18T11:53:53.080 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_notify": 0, 2024-02-18T11:53:53.081 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_src_cmpxattr": 0, 2024-02-18T11:53:53.081 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_pgls": 0, 2024-02-18T11:53:53.081 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_pgls_filter": 0, 2024-02-18T11:53:53.081 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_other": 0, 2024-02-18T11:53:53.081 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_active": 0, 2024-02-18T11:53:53.081 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_send": 0, 2024-02-18T11:53:53.081 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_resend": 0, 2024-02-18T11:53:53.081 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_ping": 0, 2024-02-18T11:53:53.081 INFO:teuthology.orchestra.run.smithi188.stdout: "poolop_active": 0, 2024-02-18T11:53:53.081 INFO:teuthology.orchestra.run.smithi188.stdout: "poolop_send": 0, 2024-02-18T11:53:53.081 INFO:teuthology.orchestra.run.smithi188.stdout: "poolop_resend": 0, 2024-02-18T11:53:53.081 INFO:teuthology.orchestra.run.smithi188.stdout: "poolstat_active": 0, 2024-02-18T11:53:53.081 INFO:teuthology.orchestra.run.smithi188.stdout: "poolstat_send": 0, 2024-02-18T11:53:53.081 INFO:teuthology.orchestra.run.smithi188.stdout: "poolstat_resend": 0, 2024-02-18T11:53:53.082 INFO:teuthology.orchestra.run.smithi188.stdout: "statfs_active": 0, 2024-02-18T11:53:53.082 INFO:teuthology.orchestra.run.smithi188.stdout: "statfs_send": 0, 2024-02-18T11:53:53.082 INFO:teuthology.orchestra.run.smithi188.stdout: "statfs_resend": 0, 2024-02-18T11:53:53.082 INFO:teuthology.orchestra.run.smithi188.stdout: "command_active": 0, 2024-02-18T11:53:53.082 INFO:teuthology.orchestra.run.smithi188.stdout: "command_send": 0, 2024-02-18T11:53:53.082 INFO:teuthology.orchestra.run.smithi188.stdout: "command_resend": 0, 2024-02-18T11:53:53.082 INFO:teuthology.orchestra.run.smithi188.stdout: "map_epoch": 89, 2024-02-18T11:53:53.082 INFO:teuthology.orchestra.run.smithi188.stdout: "map_full": 0, 2024-02-18T11:53:53.082 INFO:teuthology.orchestra.run.smithi188.stdout: "map_inc": 2, 2024-02-18T11:53:53.082 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_sessions": 8, 2024-02-18T11:53:53.082 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_session_open": 12, 2024-02-18T11:53:53.082 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_session_close": 4, 2024-02-18T11:53:53.082 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_laggy": 0, 2024-02-18T11:53:53.082 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_wr": 14, 2024-02-18T11:53:53.082 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_rd": 0, 2024-02-18T11:53:53.082 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_del": 1 2024-02-18T11:53:53.083 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:53.083 INFO:teuthology.orchestra.run.smithi188.stdout: "oft": { 2024-02-18T11:53:53.083 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_objs": 1, 2024-02-18T11:53:53.083 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_kv_pairs": 0, 2024-02-18T11:53:53.083 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_updates": 0, 2024-02-18T11:53:53.083 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_removes": 0 2024-02-18T11:53:53.083 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:53.083 INFO:teuthology.orchestra.run.smithi188.stdout: "osd-slow-ops": { 2024-02-18T11:53:53.083 INFO:teuthology.orchestra.run.smithi188.stdout: "slow_ops_count": 0 2024-02-18T11:53:53.083 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:53.083 INFO:teuthology.orchestra.run.smithi188.stdout: "purge_queue": { 2024-02-18T11:53:53.083 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing_ops": 0, 2024-02-18T11:53:53.083 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing_ops_high_water": 0, 2024-02-18T11:53:53.083 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing": 0, 2024-02-18T11:53:53.083 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing_high_water": 0, 2024-02-18T11:53:53.084 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executed_ops": 0, 2024-02-18T11:53:53.084 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executed": 0, 2024-02-18T11:53:53.084 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_item_in_journal": 0 2024-02-18T11:53:53.084 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:53.084 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-18T11:53:53.084 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-02-18T11:53:53.084 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 104857600, 2024-02-18T11:53:53.084 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-02-18T11:53:53.084 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 746, 2024-02-18T11:53:53.084 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 793896, 2024-02-18T11:53:53.084 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-02-18T11:53:53.084 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 746, 2024-02-18T11:53:53.084 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 0, 2024-02-18T11:53:53.084 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 0, 2024-02-18T11:53:53.084 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 746, 2024-02-18T11:53:53.085 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 793896, 2024-02-18T11:53:53.085 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-02-18T11:53:53.085 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:53.085 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:53.085 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:53.085 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-02-18T11:53:53.085 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:53.085 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-objecter_bytes": { 2024-02-18T11:53:53.085 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-02-18T11:53:53.085 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 104857600, 2024-02-18T11:53:53.085 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-02-18T11:53:53.085 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 0, 2024-02-18T11:53:53.085 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 0, 2024-02-18T11:53:53.085 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-02-18T11:53:53.086 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 0, 2024-02-18T11:53:53.086 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 40, 2024-02-18T11:53:53.086 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 11730, 2024-02-18T11:53:53.086 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 40, 2024-02-18T11:53:53.086 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 11730, 2024-02-18T11:53:53.086 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-02-18T11:53:53.086 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:53.086 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:53.086 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:53.086 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-02-18T11:53:53.086 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:53.086 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-objecter_ops": { 2024-02-18T11:53:53.086 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-02-18T11:53:53.086 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 1024, 2024-02-18T11:53:53.086 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-02-18T11:53:53.087 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 0, 2024-02-18T11:53:53.087 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 0, 2024-02-18T11:53:53.087 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-02-18T11:53:53.087 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 0, 2024-02-18T11:53:53.087 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 40, 2024-02-18T11:53:53.087 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 40, 2024-02-18T11:53:53.087 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 40, 2024-02-18T11:53:53.087 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 40, 2024-02-18T11:53:53.087 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-02-18T11:53:53.087 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:53.087 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:53.087 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:53.087 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-02-18T11:53:53.087 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:53.088 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-write_buf_throttle": { 2024-02-18T11:53:53.088 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-02-18T11:53:53.088 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 3758096384, 2024-02-18T11:53:53.088 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-02-18T11:53:53.088 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 0, 2024-02-18T11:53:53.088 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 0, 2024-02-18T11:53:53.088 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-02-18T11:53:53.088 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 0, 2024-02-18T11:53:53.088 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 0, 2024-02-18T11:53:53.088 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 0, 2024-02-18T11:53:53.088 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 0, 2024-02-18T11:53:53.088 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 0, 2024-02-18T11:53:53.088 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-02-18T11:53:53.088 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:53.088 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:53.089 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:53.089 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-02-18T11:53:53.089 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:53.089 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-write_buf_throttle-0x561b058ba6e0": { 2024-02-18T11:53:53.089 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-02-18T11:53:53.089 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 3758096384, 2024-02-18T11:53:53.089 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-02-18T11:53:53.089 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 5, 2024-02-18T11:53:53.089 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 1428, 2024-02-18T11:53:53.089 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-02-18T11:53:53.089 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 5, 2024-02-18T11:53:53.089 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 0, 2024-02-18T11:53:53.089 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 0, 2024-02-18T11:53:53.089 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 4, 2024-02-18T11:53:53.089 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 1428, 2024-02-18T11:53:53.089 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-02-18T11:53:53.090 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:53.090 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:53.090 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:53.090 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-02-18T11:53:53.090 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-02-18T11:53:53.090 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-02-18T11:53:53.301 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:53:53 smithi188 ceph-mon[36925]: pgmap v739: 97 pgs: 97 active+clean; 8.2 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 73 MiB/s wr, 354 op/s 2024-02-18T11:53:53.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:53:53 smithi012 ceph-mon[32460]: pgmap v739: 97 pgs: 97 active+clean; 8.2 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 73 MiB/s wr, 354 op/s 2024-02-18T11:53:53.499 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:53:53 smithi158 ceph-mon[36626]: pgmap v739: 97 pgs: 97 active+clean; 8.2 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 73 MiB/s wr, 354 op/s 2024-02-18T11:53:53.922 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-02-18T11:53:53.923 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-02-18T11:53:53.923 DEBUG:tasks.check_counter:Getting stats from f 2024-02-18T11:53:53.923 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph daemon mds.f perf dump 2024-02-18T11:53:54.170 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.c/config 2024-02-18T11:53:55.352 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:53:55 smithi188 ceph-mon[36925]: pgmap v740: 97 pgs: 97 active+clean; 7.3 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 74 MiB/s wr, 465 op/s 2024-02-18T11:53:55.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:53:55 smithi012 ceph-mon[32460]: pgmap v740: 97 pgs: 97 active+clean; 7.3 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 74 MiB/s wr, 465 op/s 2024-02-18T11:53:55.494 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:53:55 smithi158 ceph-mon[36626]: pgmap v740: 97 pgs: 97 active+clean; 7.3 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 74 MiB/s wr, 465 op/s 2024-02-18T11:53:55.823 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-02-18T11:53:55.823 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-0": { 2024-02-18T11:53:55.823 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 2613, 2024-02-18T11:53:55.824 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 2613, 2024-02-18T11:53:55.824 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 742503, 2024-02-18T11:53:55.824 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 811854, 2024-02-18T11:53:55.824 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 6, 2024-02-18T11:53:55.824 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 2, 2024-02-18T11:53:55.824 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 0.238035036, 2024-02-18T11:53:55.824 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 0.068612750, 2024-02-18T11:53:55.824 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 0.060991831, 2024-02-18T11:53:55.824 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 0.091520242, 2024-02-18T11:53:55.825 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-02-18T11:53:55.825 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 2612, 2024-02-18T11:53:55.825 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.099454810, 2024-02-18T11:53:55.825 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000038076 2024-02-18T11:53:55.825 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:55.825 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-02-18T11:53:55.825 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:55.825 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:55.825 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:55.825 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:55.825 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-02-18T11:53:55.826 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 448 2024-02-18T11:53:55.826 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:55.826 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-1": { 2024-02-18T11:53:55.826 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 264, 2024-02-18T11:53:55.826 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 258, 2024-02-18T11:53:55.826 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 779308, 2024-02-18T11:53:55.826 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 151484, 2024-02-18T11:53:55.826 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 3, 2024-02-18T11:53:55.827 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 2, 2024-02-18T11:53:55.827 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 0.046514416, 2024-02-18T11:53:55.827 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 0.016131126, 2024-02-18T11:53:55.827 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 0.022953170, 2024-02-18T11:53:55.827 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 0.008693110, 2024-02-18T11:53:55.827 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-02-18T11:53:55.827 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 257, 2024-02-18T11:53:55.827 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.012482542, 2024-02-18T11:53:55.827 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000048570 2024-02-18T11:53:55.828 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:55.828 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-02-18T11:53:55.828 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:55.828 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:55.828 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:55.828 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:55.828 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 778752, 2024-02-18T11:53:55.828 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 150800 2024-02-18T11:53:55.828 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:55.828 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-2": { 2024-02-18T11:53:55.828 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 3, 2024-02-18T11:53:55.829 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 192, 2024-02-18T11:53:55.829 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 588, 2024-02-18T11:53:55.829 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 496807, 2024-02-18T11:53:55.829 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 6, 2024-02-18T11:53:55.829 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 3, 2024-02-18T11:53:55.829 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 0.012728910, 2024-02-18T11:53:55.829 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 0.009115521, 2024-02-18T11:53:55.829 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 0.001560000, 2024-02-18T11:53:55.829 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 0.000041314, 2024-02-18T11:53:55.829 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-02-18T11:53:55.830 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 190, 2024-02-18T11:53:55.830 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.013222973, 2024-02-18T11:53:55.830 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000069594 2024-02-18T11:53:55.830 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:55.830 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-02-18T11:53:55.830 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:55.830 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:55.830 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:55.830 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:55.830 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 320, 2024-02-18T11:53:55.831 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 496496 2024-02-18T11:53:55.831 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:55.831 INFO:teuthology.orchestra.run.smithi188.stdout: "cct": { 2024-02-18T11:53:55.831 INFO:teuthology.orchestra.run.smithi188.stdout: "total_workers": 1, 2024-02-18T11:53:55.831 INFO:teuthology.orchestra.run.smithi188.stdout: "unhealthy_workers": 0 2024-02-18T11:53:55.831 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:55.831 INFO:teuthology.orchestra.run.smithi188.stdout: "finisher-MDSRank": { 2024-02-18T11:53:55.831 INFO:teuthology.orchestra.run.smithi188.stdout: "queue_len": 0, 2024-02-18T11:53:55.831 INFO:teuthology.orchestra.run.smithi188.stdout: "complete_latency": { 2024-02-18T11:53:55.832 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 4381, 2024-02-18T11:53:55.832 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.135213665, 2024-02-18T11:53:55.832 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000030863 2024-02-18T11:53:55.832 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-02-18T11:53:55.832 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:55.832 INFO:teuthology.orchestra.run.smithi188.stdout: "finisher-PurgeQueue": { 2024-02-18T11:53:55.832 INFO:teuthology.orchestra.run.smithi188.stdout: "queue_len": 0, 2024-02-18T11:53:55.832 INFO:teuthology.orchestra.run.smithi188.stdout: "complete_latency": { 2024-02-18T11:53:55.832 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:55.832 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:55.832 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:55.833 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-02-18T11:53:55.833 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:55.833 INFO:teuthology.orchestra.run.smithi188.stdout: "mds": { 2024-02-18T11:53:55.833 INFO:teuthology.orchestra.run.smithi188.stdout: "request": 0, 2024-02-18T11:53:55.833 INFO:teuthology.orchestra.run.smithi188.stdout: "reply": 0, 2024-02-18T11:53:55.833 INFO:teuthology.orchestra.run.smithi188.stdout: "reply_latency": { 2024-02-18T11:53:55.833 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:55.833 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:55.833 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:55.834 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:55.834 INFO:teuthology.orchestra.run.smithi188.stdout: "slow_reply": 0, 2024-02-18T11:53:55.834 INFO:teuthology.orchestra.run.smithi188.stdout: "forward": 0, 2024-02-18T11:53:55.834 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_fetch_complete": 0, 2024-02-18T11:53:55.834 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_fetch_keys": 0, 2024-02-18T11:53:55.834 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_commit": 0, 2024-02-18T11:53:55.834 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_split": 0, 2024-02-18T11:53:55.834 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_merge": 0, 2024-02-18T11:53:55.834 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes": 0, 2024-02-18T11:53:55.834 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_top": 0, 2024-02-18T11:53:55.835 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_bottom": 0, 2024-02-18T11:53:55.835 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_pin_tail": 0, 2024-02-18T11:53:55.835 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_pinned": 0, 2024-02-18T11:53:55.835 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_expired": 0, 2024-02-18T11:53:55.835 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_with_caps": 0, 2024-02-18T11:53:55.835 INFO:teuthology.orchestra.run.smithi188.stdout: "caps": 0, 2024-02-18T11:53:55.835 INFO:teuthology.orchestra.run.smithi188.stdout: "subtrees": 1, 2024-02-18T11:53:55.835 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse": 0, 2024-02-18T11:53:55.835 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_hit": 0, 2024-02-18T11:53:55.836 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_forward": 0, 2024-02-18T11:53:55.836 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_discover": 0, 2024-02-18T11:53:55.836 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_dir_fetch": 0, 2024-02-18T11:53:55.836 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_remote_ino": 0, 2024-02-18T11:53:55.837 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_lock": 0, 2024-02-18T11:53:55.837 INFO:teuthology.orchestra.run.smithi188.stdout: "load_cent": 0, 2024-02-18T11:53:55.837 INFO:teuthology.orchestra.run.smithi188.stdout: "q": 0, 2024-02-18T11:53:55.837 INFO:teuthology.orchestra.run.smithi188.stdout: "exported": 0, 2024-02-18T11:53:55.837 INFO:teuthology.orchestra.run.smithi188.stdout: "exported_inodes": 0, 2024-02-18T11:53:55.837 INFO:teuthology.orchestra.run.smithi188.stdout: "imported": 0, 2024-02-18T11:53:55.837 INFO:teuthology.orchestra.run.smithi188.stdout: "imported_inodes": 0, 2024-02-18T11:53:55.837 INFO:teuthology.orchestra.run.smithi188.stdout: "openino_dir_fetch": 0, 2024-02-18T11:53:55.837 INFO:teuthology.orchestra.run.smithi188.stdout: "openino_backtrace_fetch": 0, 2024-02-18T11:53:55.837 INFO:teuthology.orchestra.run.smithi188.stdout: "openino_peer_discover": 0, 2024-02-18T11:53:55.837 INFO:teuthology.orchestra.run.smithi188.stdout: "root_rfiles": 0, 2024-02-18T11:53:55.837 INFO:teuthology.orchestra.run.smithi188.stdout: "root_rbytes": 0, 2024-02-18T11:53:55.838 INFO:teuthology.orchestra.run.smithi188.stdout: "root_rsnaps": 0, 2024-02-18T11:53:55.838 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_backtrace_fetch": 0, 2024-02-18T11:53:55.838 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_set_tag": 0, 2024-02-18T11:53:55.838 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_backtrace_repaired": 0, 2024-02-18T11:53:55.838 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_inotable_repaired": 0, 2024-02-18T11:53:55.838 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_dir_inodes": 0, 2024-02-18T11:53:55.838 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_dir_base_inodes": 0, 2024-02-18T11:53:55.838 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_dirfrag_rstats": 0, 2024-02-18T11:53:55.838 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_file_inodes": 0, 2024-02-18T11:53:55.838 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_inode_file_caps": 0, 2024-02-18T11:53:55.838 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_revoke": 0, 2024-02-18T11:53:55.838 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_grant": 0, 2024-02-18T11:53:55.839 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_trunc": 0, 2024-02-18T11:53:55.839 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-02-18T11:53:55.839 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_flush_ack": 0, 2024-02-18T11:53:55.839 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_caps": 0, 2024-02-18T11:53:55.839 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_caps_dirty": 0, 2024-02-18T11:53:55.839 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_cap_release": 0, 2024-02-18T11:53:55.839 INFO:teuthology.orchestra.run.smithi188.stdout: "process_request_cap_release": 0 2024-02-18T11:53:55.839 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:55.839 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_cache": { 2024-02-18T11:53:55.840 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_update": 0, 2024-02-18T11:53:55.840 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_update_receipt": 0, 2024-02-18T11:53:55.840 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_try_discover": 0, 2024-02-18T11:53:55.840 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_send_discover": 0, 2024-02-18T11:53:55.840 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_handle_discover": 0, 2024-02-18T11:53:55.840 INFO:teuthology.orchestra.run.smithi188.stdout: "num_strays": 0, 2024-02-18T11:53:55.840 INFO:teuthology.orchestra.run.smithi188.stdout: "num_strays_delayed": 0, 2024-02-18T11:53:55.840 INFO:teuthology.orchestra.run.smithi188.stdout: "num_strays_enqueuing": 0, 2024-02-18T11:53:55.840 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_created": 0, 2024-02-18T11:53:55.840 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_enqueued": 0, 2024-02-18T11:53:55.840 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_reintegrated": 0, 2024-02-18T11:53:55.841 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_migrated": 0, 2024-02-18T11:53:55.841 INFO:teuthology.orchestra.run.smithi188.stdout: "num_recovering_processing": 0, 2024-02-18T11:53:55.841 INFO:teuthology.orchestra.run.smithi188.stdout: "num_recovering_enqueued": 0, 2024-02-18T11:53:55.841 INFO:teuthology.orchestra.run.smithi188.stdout: "num_recovering_prioritized": 0, 2024-02-18T11:53:55.841 INFO:teuthology.orchestra.run.smithi188.stdout: "recovery_started": 0, 2024-02-18T11:53:55.841 INFO:teuthology.orchestra.run.smithi188.stdout: "recovery_completed": 0, 2024-02-18T11:53:55.841 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_enqueue_scrub": 0, 2024-02-18T11:53:55.841 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_exportdir": 0, 2024-02-18T11:53:55.841 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_flush": 0, 2024-02-18T11:53:55.842 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_fragmentdir": 0, 2024-02-18T11:53:55.842 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_fragstats": 0, 2024-02-18T11:53:55.842 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_inodestats": 0 2024-02-18T11:53:55.842 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:55.842 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_log": { 2024-02-18T11:53:55.842 INFO:teuthology.orchestra.run.smithi188.stdout: "evlrg": 0, 2024-02-18T11:53:55.842 INFO:teuthology.orchestra.run.smithi188.stdout: "evadd": 0, 2024-02-18T11:53:55.842 INFO:teuthology.orchestra.run.smithi188.stdout: "evex": 0, 2024-02-18T11:53:55.842 INFO:teuthology.orchestra.run.smithi188.stdout: "evtrm": 0, 2024-02-18T11:53:55.842 INFO:teuthology.orchestra.run.smithi188.stdout: "ev": 7, 2024-02-18T11:53:55.842 INFO:teuthology.orchestra.run.smithi188.stdout: "evexg": 0, 2024-02-18T11:53:55.843 INFO:teuthology.orchestra.run.smithi188.stdout: "evexd": 0, 2024-02-18T11:53:55.843 INFO:teuthology.orchestra.run.smithi188.stdout: "segadd": 0, 2024-02-18T11:53:55.843 INFO:teuthology.orchestra.run.smithi188.stdout: "segex": 0, 2024-02-18T11:53:55.843 INFO:teuthology.orchestra.run.smithi188.stdout: "segtrm": 0, 2024-02-18T11:53:55.843 INFO:teuthology.orchestra.run.smithi188.stdout: "seg": 2, 2024-02-18T11:53:55.843 INFO:teuthology.orchestra.run.smithi188.stdout: "segmjr": 2, 2024-02-18T11:53:55.843 INFO:teuthology.orchestra.run.smithi188.stdout: "segexg": 0, 2024-02-18T11:53:55.843 INFO:teuthology.orchestra.run.smithi188.stdout: "segexd": 0, 2024-02-18T11:53:55.843 INFO:teuthology.orchestra.run.smithi188.stdout: "expos": 4194304, 2024-02-18T11:53:55.843 INFO:teuthology.orchestra.run.smithi188.stdout: "wrpos": 4196264, 2024-02-18T11:53:55.843 INFO:teuthology.orchestra.run.smithi188.stdout: "rdpos": 4196123, 2024-02-18T11:53:55.843 INFO:teuthology.orchestra.run.smithi188.stdout: "jlat": { 2024-02-18T11:53:55.844 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:55.844 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:55.844 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:55.844 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:55.844 INFO:teuthology.orchestra.run.smithi188.stdout: "replayed": 7 2024-02-18T11:53:55.844 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:55.844 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_mem": { 2024-02-18T11:53:55.844 INFO:teuthology.orchestra.run.smithi188.stdout: "ino": 3, 2024-02-18T11:53:55.844 INFO:teuthology.orchestra.run.smithi188.stdout: "ino+": 3, 2024-02-18T11:53:55.844 INFO:teuthology.orchestra.run.smithi188.stdout: "ino-": 0, 2024-02-18T11:53:55.844 INFO:teuthology.orchestra.run.smithi188.stdout: "dir": 1, 2024-02-18T11:53:55.845 INFO:teuthology.orchestra.run.smithi188.stdout: "dir+": 1, 2024-02-18T11:53:55.845 INFO:teuthology.orchestra.run.smithi188.stdout: "dir-": 0, 2024-02-18T11:53:55.845 INFO:teuthology.orchestra.run.smithi188.stdout: "dn": 0, 2024-02-18T11:53:55.845 INFO:teuthology.orchestra.run.smithi188.stdout: "dn+": 0, 2024-02-18T11:53:55.845 INFO:teuthology.orchestra.run.smithi188.stdout: "dn-": 0, 2024-02-18T11:53:55.845 INFO:teuthology.orchestra.run.smithi188.stdout: "cap": 0, 2024-02-18T11:53:55.845 INFO:teuthology.orchestra.run.smithi188.stdout: "cap+": 0, 2024-02-18T11:53:55.845 INFO:teuthology.orchestra.run.smithi188.stdout: "cap-": 0, 2024-02-18T11:53:55.845 INFO:teuthology.orchestra.run.smithi188.stdout: "rss": 46668, 2024-02-18T11:53:55.845 INFO:teuthology.orchestra.run.smithi188.stdout: "heap": 215420 2024-02-18T11:53:55.845 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:55.845 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_server": { 2024-02-18T11:53:55.846 INFO:teuthology.orchestra.run.smithi188.stdout: "dispatch_client_request": 0, 2024-02-18T11:53:55.846 INFO:teuthology.orchestra.run.smithi188.stdout: "dispatch_server_request": 0, 2024-02-18T11:53:55.846 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_request": 0, 2024-02-18T11:53:55.846 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_session": 0, 2024-02-18T11:53:55.846 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_peer_request": 0, 2024-02-18T11:53:55.846 INFO:teuthology.orchestra.run.smithi188.stdout: "req_create_latency": { 2024-02-18T11:53:55.846 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:55.846 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:55.847 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:55.847 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:55.847 INFO:teuthology.orchestra.run.smithi188.stdout: "req_getattr_latency": { 2024-02-18T11:53:55.847 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:55.847 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:55.847 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:55.847 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:55.847 INFO:teuthology.orchestra.run.smithi188.stdout: "req_getfilelock_latency": { 2024-02-18T11:53:55.847 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:55.847 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:55.848 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:55.848 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:55.848 INFO:teuthology.orchestra.run.smithi188.stdout: "req_link_latency": { 2024-02-18T11:53:55.848 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:55.848 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:55.848 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:55.848 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:55.848 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookup_latency": { 2024-02-18T11:53:55.848 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:55.848 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:55.849 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:55.849 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:55.849 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookuphash_latency": { 2024-02-18T11:53:55.849 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:55.849 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:55.849 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:55.849 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:55.849 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupino_latency": { 2024-02-18T11:53:55.849 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:55.849 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:55.849 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:55.850 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:55.850 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupname_latency": { 2024-02-18T11:53:55.850 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:55.850 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:55.850 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:55.850 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:55.850 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupparent_latency": { 2024-02-18T11:53:55.850 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:55.851 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:55.851 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:55.851 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:55.851 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupsnap_latency": { 2024-02-18T11:53:55.851 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:55.851 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:55.851 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:55.851 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:55.851 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lssnap_latency": { 2024-02-18T11:53:55.852 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:55.852 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:55.852 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:55.852 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:55.852 INFO:teuthology.orchestra.run.smithi188.stdout: "req_mkdir_latency": { 2024-02-18T11:53:55.852 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:55.852 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:55.852 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:55.852 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:55.853 INFO:teuthology.orchestra.run.smithi188.stdout: "req_mknod_latency": { 2024-02-18T11:53:55.854 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:55.854 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:55.854 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:55.854 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:55.854 INFO:teuthology.orchestra.run.smithi188.stdout: "req_mksnap_latency": { 2024-02-18T11:53:55.854 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:55.854 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:55.854 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:55.854 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:55.854 INFO:teuthology.orchestra.run.smithi188.stdout: "req_open_latency": { 2024-02-18T11:53:55.854 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:55.855 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:55.855 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:55.855 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:55.855 INFO:teuthology.orchestra.run.smithi188.stdout: "req_readdir_latency": { 2024-02-18T11:53:55.855 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:55.855 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:55.855 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:55.856 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:55.856 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rename_latency": { 2024-02-18T11:53:55.856 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:55.856 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:55.856 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:55.856 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:55.856 INFO:teuthology.orchestra.run.smithi188.stdout: "req_renamesnap_latency": { 2024-02-18T11:53:55.856 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:55.856 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:55.857 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:55.857 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:55.857 INFO:teuthology.orchestra.run.smithi188.stdout: "req_snapdiff_latency": { 2024-02-18T11:53:55.857 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:55.857 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:55.857 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:55.857 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:55.857 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rmdir_latency": { 2024-02-18T11:53:55.858 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:55.858 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:55.858 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:55.858 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:55.858 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rmsnap_latency": { 2024-02-18T11:53:55.858 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:55.858 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:55.858 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:55.859 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:55.859 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rmxattr_latency": { 2024-02-18T11:53:55.859 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:55.859 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:55.859 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:55.859 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:55.859 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setattr_latency": { 2024-02-18T11:53:55.859 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:55.859 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:55.859 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:55.860 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:55.860 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setdirlayout_latency": { 2024-02-18T11:53:55.860 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:55.860 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:55.860 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:55.860 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:55.860 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setfilelock_latency": { 2024-02-18T11:53:55.860 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:55.860 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:55.861 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:55.861 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:55.861 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setlayout_latency": { 2024-02-18T11:53:55.861 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:55.861 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:55.861 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:55.861 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:55.861 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setxattr_latency": { 2024-02-18T11:53:55.861 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:55.861 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:55.862 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:55.862 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:55.862 INFO:teuthology.orchestra.run.smithi188.stdout: "req_symlink_latency": { 2024-02-18T11:53:55.862 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:55.862 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:55.862 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:55.862 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:55.862 INFO:teuthology.orchestra.run.smithi188.stdout: "req_unlink_latency": { 2024-02-18T11:53:55.862 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:55.863 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:55.863 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:55.863 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:55.863 INFO:teuthology.orchestra.run.smithi188.stdout: "cap_revoke_eviction": 0, 2024-02-18T11:53:55.863 INFO:teuthology.orchestra.run.smithi188.stdout: "cap_acquisition_throttle": 0, 2024-02-18T11:53:55.863 INFO:teuthology.orchestra.run.smithi188.stdout: "req_getvxattr_latency": { 2024-02-18T11:53:55.863 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:55.863 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:55.863 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:55.864 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-02-18T11:53:55.864 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:55.864 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_sessions": { 2024-02-18T11:53:55.864 INFO:teuthology.orchestra.run.smithi188.stdout: "session_count": 1, 2024-02-18T11:53:55.864 INFO:teuthology.orchestra.run.smithi188.stdout: "session_add": 3, 2024-02-18T11:53:55.864 INFO:teuthology.orchestra.run.smithi188.stdout: "session_remove": 2, 2024-02-18T11:53:55.864 INFO:teuthology.orchestra.run.smithi188.stdout: "sessions_open": 2, 2024-02-18T11:53:55.864 INFO:teuthology.orchestra.run.smithi188.stdout: "sessions_stale": 0, 2024-02-18T11:53:55.865 INFO:teuthology.orchestra.run.smithi188.stdout: "total_load": 0, 2024-02-18T11:53:55.865 INFO:teuthology.orchestra.run.smithi188.stdout: "average_load": 0, 2024-02-18T11:53:55.865 INFO:teuthology.orchestra.run.smithi188.stdout: "avg_session_uptime": 3398, 2024-02-18T11:53:55.865 INFO:teuthology.orchestra.run.smithi188.stdout: "mdthresh_evicted": 0 2024-02-18T11:53:55.865 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:55.865 INFO:teuthology.orchestra.run.smithi188.stdout: "mempool": { 2024-02-18T11:53:55.865 INFO:teuthology.orchestra.run.smithi188.stdout: "bloom_filter_bytes": 0, 2024-02-18T11:53:55.865 INFO:teuthology.orchestra.run.smithi188.stdout: "bloom_filter_items": 0, 2024-02-18T11:53:55.865 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_alloc_bytes": 0, 2024-02-18T11:53:55.865 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_alloc_items": 0, 2024-02-18T11:53:55.865 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_data_bytes": 0, 2024-02-18T11:53:55.866 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_data_items": 0, 2024-02-18T11:53:55.866 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-18T11:53:55.866 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_onode_items": 0, 2024-02-18T11:53:55.866 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-18T11:53:55.866 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_meta_items": 0, 2024-02-18T11:53:55.866 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_other_bytes": 0, 2024-02-18T11:53:55.866 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_other_items": 0, 2024-02-18T11:53:55.866 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-18T11:53:55.866 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_buffer_items": 0, 2024-02-18T11:53:55.866 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_extent_bytes": 0, 2024-02-18T11:53:55.866 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_extent_items": 0, 2024-02-18T11:53:55.867 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_blob_bytes": 0, 2024-02-18T11:53:55.867 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_blob_items": 0, 2024-02-18T11:53:55.867 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-18T11:53:55.867 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_shared_blob_items": 0, 2024-02-18T11:53:55.867 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-18T11:53:55.867 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_inline_bl_items": 0, 2024-02-18T11:53:55.867 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_fsck_bytes": 0, 2024-02-18T11:53:55.867 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_fsck_items": 0, 2024-02-18T11:53:55.867 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_txc_bytes": 0, 2024-02-18T11:53:55.867 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_txc_items": 0, 2024-02-18T11:53:55.868 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-18T11:53:55.868 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_deferred_items": 0, 2024-02-18T11:53:55.868 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_bytes": 0, 2024-02-18T11:53:55.868 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_items": 0, 2024-02-18T11:53:55.868 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_bytes": 0, 2024-02-18T11:53:55.868 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_items": 0, 2024-02-18T11:53:55.868 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_reader_bytes": 0, 2024-02-18T11:53:55.868 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_reader_items": 0, 2024-02-18T11:53:55.868 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_writer_bytes": 0, 2024-02-18T11:53:55.868 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_writer_items": 0, 2024-02-18T11:53:55.869 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_anon_bytes": 181243, 2024-02-18T11:53:55.869 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_anon_items": 45, 2024-02-18T11:53:55.869 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_meta_bytes": 0, 2024-02-18T11:53:55.869 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_meta_items": 0, 2024-02-18T11:53:55.869 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_bytes": 0, 2024-02-18T11:53:55.869 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_items": 0, 2024-02-18T11:53:55.869 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_mapbl_bytes": 0, 2024-02-18T11:53:55.869 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_mapbl_items": 0, 2024-02-18T11:53:55.869 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_pglog_bytes": 0, 2024-02-18T11:53:55.869 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_pglog_items": 0, 2024-02-18T11:53:55.869 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_bytes": 7528, 2024-02-18T11:53:55.870 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_items": 158, 2024-02-18T11:53:55.870 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_mapping_bytes": 0, 2024-02-18T11:53:55.870 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_mapping_items": 0, 2024-02-18T11:53:55.870 INFO:teuthology.orchestra.run.smithi188.stdout: "pgmap_bytes": 0, 2024-02-18T11:53:55.870 INFO:teuthology.orchestra.run.smithi188.stdout: "pgmap_items": 0, 2024-02-18T11:53:55.870 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_co_bytes": 8320, 2024-02-18T11:53:55.870 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_co_items": 13, 2024-02-18T11:53:55.870 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_1_bytes": 0, 2024-02-18T11:53:55.870 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_1_items": 0, 2024-02-18T11:53:55.870 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_2_bytes": 0, 2024-02-18T11:53:55.871 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_2_items": 0 2024-02-18T11:53:55.871 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:55.871 INFO:teuthology.orchestra.run.smithi188.stdout: "objecter": { 2024-02-18T11:53:55.871 INFO:teuthology.orchestra.run.smithi188.stdout: "op_active": 0, 2024-02-18T11:53:55.871 INFO:teuthology.orchestra.run.smithi188.stdout: "op_laggy": 0, 2024-02-18T11:53:55.871 INFO:teuthology.orchestra.run.smithi188.stdout: "op_send": 2612, 2024-02-18T11:53:55.871 INFO:teuthology.orchestra.run.smithi188.stdout: "op_send_bytes": 21, 2024-02-18T11:53:55.871 INFO:teuthology.orchestra.run.smithi188.stdout: "op_resend": 3, 2024-02-18T11:53:55.871 INFO:teuthology.orchestra.run.smithi188.stdout: "op_reply": 2612, 2024-02-18T11:53:55.871 INFO:teuthology.orchestra.run.smithi188.stdout: "op_latency": { 2024-02-18T11:53:55.872 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 2612, 2024-02-18T11:53:55.872 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 8.287855649, 2024-02-18T11:53:55.872 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.003172992 2024-02-18T11:53:55.872 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:55.872 INFO:teuthology.orchestra.run.smithi188.stdout: "op_inflight": 0, 2024-02-18T11:53:55.872 INFO:teuthology.orchestra.run.smithi188.stdout: "oplen_avg": { 2024-02-18T11:53:55.872 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 2612, 2024-02-18T11:53:55.872 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 2613 2024-02-18T11:53:55.872 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:55.872 INFO:teuthology.orchestra.run.smithi188.stdout: "op": 2612, 2024-02-18T11:53:55.873 INFO:teuthology.orchestra.run.smithi188.stdout: "op_r": 2612, 2024-02-18T11:53:55.873 INFO:teuthology.orchestra.run.smithi188.stdout: "op_w": 0, 2024-02-18T11:53:55.873 INFO:teuthology.orchestra.run.smithi188.stdout: "op_rmw": 0, 2024-02-18T11:53:55.873 INFO:teuthology.orchestra.run.smithi188.stdout: "op_pg": 0, 2024-02-18T11:53:55.873 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_stat": 1704, 2024-02-18T11:53:55.873 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_create": 0, 2024-02-18T11:53:55.873 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_read": 906, 2024-02-18T11:53:55.873 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_write": 0, 2024-02-18T11:53:55.873 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_writefull": 0, 2024-02-18T11:53:55.873 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_writesame": 0, 2024-02-18T11:53:55.874 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_append": 0, 2024-02-18T11:53:55.874 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_zero": 0, 2024-02-18T11:53:55.874 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_truncate": 0, 2024-02-18T11:53:55.874 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_delete": 0, 2024-02-18T11:53:55.874 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_mapext": 0, 2024-02-18T11:53:55.874 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_sparse_read": 0, 2024-02-18T11:53:55.874 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_clonerange": 0, 2024-02-18T11:53:55.874 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_getxattr": 1, 2024-02-18T11:53:55.875 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_setxattr": 0, 2024-02-18T11:53:55.875 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_cmpxattr": 0, 2024-02-18T11:53:55.875 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_rmxattr": 0, 2024-02-18T11:53:55.875 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_resetxattrs": 0, 2024-02-18T11:53:55.875 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_call": 0, 2024-02-18T11:53:55.875 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_watch": 0, 2024-02-18T11:53:55.875 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_notify": 0, 2024-02-18T11:53:55.876 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_src_cmpxattr": 0, 2024-02-18T11:53:55.876 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_pgls": 0, 2024-02-18T11:53:55.876 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_pgls_filter": 0, 2024-02-18T11:53:55.876 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_other": 0, 2024-02-18T11:53:55.876 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_active": 0, 2024-02-18T11:53:55.877 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_send": 0, 2024-02-18T11:53:55.877 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_resend": 0, 2024-02-18T11:53:55.877 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_ping": 0, 2024-02-18T11:53:55.877 INFO:teuthology.orchestra.run.smithi188.stdout: "poolop_active": 0, 2024-02-18T11:53:55.877 INFO:teuthology.orchestra.run.smithi188.stdout: "poolop_send": 0, 2024-02-18T11:53:55.877 INFO:teuthology.orchestra.run.smithi188.stdout: "poolop_resend": 0, 2024-02-18T11:53:55.877 INFO:teuthology.orchestra.run.smithi188.stdout: "poolstat_active": 0, 2024-02-18T11:53:55.877 INFO:teuthology.orchestra.run.smithi188.stdout: "poolstat_send": 0, 2024-02-18T11:53:55.877 INFO:teuthology.orchestra.run.smithi188.stdout: "poolstat_resend": 0, 2024-02-18T11:53:55.877 INFO:teuthology.orchestra.run.smithi188.stdout: "statfs_active": 0, 2024-02-18T11:53:55.877 INFO:teuthology.orchestra.run.smithi188.stdout: "statfs_send": 0, 2024-02-18T11:53:55.877 INFO:teuthology.orchestra.run.smithi188.stdout: "statfs_resend": 0, 2024-02-18T11:53:55.877 INFO:teuthology.orchestra.run.smithi188.stdout: "command_active": 0, 2024-02-18T11:53:55.877 INFO:teuthology.orchestra.run.smithi188.stdout: "command_send": 0, 2024-02-18T11:53:55.877 INFO:teuthology.orchestra.run.smithi188.stdout: "command_resend": 0, 2024-02-18T11:53:55.878 INFO:teuthology.orchestra.run.smithi188.stdout: "map_epoch": 89, 2024-02-18T11:53:55.878 INFO:teuthology.orchestra.run.smithi188.stdout: "map_full": 0, 2024-02-18T11:53:55.878 INFO:teuthology.orchestra.run.smithi188.stdout: "map_inc": 2, 2024-02-18T11:53:55.878 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_sessions": 4, 2024-02-18T11:53:55.878 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_session_open": 5, 2024-02-18T11:53:55.878 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_session_close": 1, 2024-02-18T11:53:55.878 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_laggy": 0, 2024-02-18T11:53:55.878 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_wr": 0, 2024-02-18T11:53:55.878 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_rd": 2, 2024-02-18T11:53:55.878 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_del": 0 2024-02-18T11:53:55.878 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:55.878 INFO:teuthology.orchestra.run.smithi188.stdout: "oft": { 2024-02-18T11:53:55.878 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_objs": 0, 2024-02-18T11:53:55.878 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_kv_pairs": 0, 2024-02-18T11:53:55.879 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_updates": 0, 2024-02-18T11:53:55.879 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_removes": 0 2024-02-18T11:53:55.879 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:55.879 INFO:teuthology.orchestra.run.smithi188.stdout: "osd-slow-ops": { 2024-02-18T11:53:55.879 INFO:teuthology.orchestra.run.smithi188.stdout: "slow_ops_count": 0 2024-02-18T11:53:55.879 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:55.879 INFO:teuthology.orchestra.run.smithi188.stdout: "purge_queue": { 2024-02-18T11:53:55.879 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing_ops": 0, 2024-02-18T11:53:55.879 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing_ops_high_water": 0, 2024-02-18T11:53:55.879 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing": 0, 2024-02-18T11:53:55.879 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing_high_water": 0, 2024-02-18T11:53:55.879 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executed_ops": 0, 2024-02-18T11:53:55.879 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executed": 0, 2024-02-18T11:53:55.879 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_item_in_journal": 0 2024-02-18T11:53:55.879 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:55.880 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-18T11:53:55.880 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-02-18T11:53:55.880 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 104857600, 2024-02-18T11:53:55.880 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-02-18T11:53:55.880 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 2876, 2024-02-18T11:53:55.880 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 1247068, 2024-02-18T11:53:55.880 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-02-18T11:53:55.880 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 2876, 2024-02-18T11:53:55.880 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 0, 2024-02-18T11:53:55.880 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 0, 2024-02-18T11:53:55.880 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 2876, 2024-02-18T11:53:55.880 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 1247068, 2024-02-18T11:53:55.880 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-02-18T11:53:55.880 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:55.881 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:55.881 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:55.881 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-02-18T11:53:55.881 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:55.881 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-objecter_bytes": { 2024-02-18T11:53:55.881 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-02-18T11:53:55.881 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 104857600, 2024-02-18T11:53:55.881 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-02-18T11:53:55.881 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 0, 2024-02-18T11:53:55.881 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 0, 2024-02-18T11:53:55.881 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-02-18T11:53:55.881 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 0, 2024-02-18T11:53:55.881 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 2612, 2024-02-18T11:53:55.881 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 1965, 2024-02-18T11:53:55.881 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 2612, 2024-02-18T11:53:55.882 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 1965, 2024-02-18T11:53:55.882 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-02-18T11:53:55.882 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:55.882 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:55.882 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:55.882 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-02-18T11:53:55.882 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:55.882 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-objecter_ops": { 2024-02-18T11:53:55.882 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-02-18T11:53:55.882 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 1024, 2024-02-18T11:53:55.882 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-02-18T11:53:55.882 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 0, 2024-02-18T11:53:55.882 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 0, 2024-02-18T11:53:55.882 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-02-18T11:53:55.883 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 0, 2024-02-18T11:53:55.883 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 2612, 2024-02-18T11:53:55.883 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 2612, 2024-02-18T11:53:55.883 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 2612, 2024-02-18T11:53:55.883 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 2612, 2024-02-18T11:53:55.883 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-02-18T11:53:55.883 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:55.883 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:55.883 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:55.883 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-02-18T11:53:55.883 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:55.883 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-write_buf_throttle": { 2024-02-18T11:53:55.883 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-02-18T11:53:55.883 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 3758096384, 2024-02-18T11:53:55.883 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-02-18T11:53:55.884 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 0, 2024-02-18T11:53:55.884 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 0, 2024-02-18T11:53:55.884 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-02-18T11:53:55.884 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 0, 2024-02-18T11:53:55.884 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 0, 2024-02-18T11:53:55.884 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 0, 2024-02-18T11:53:55.884 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 0, 2024-02-18T11:53:55.884 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 0, 2024-02-18T11:53:55.884 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-02-18T11:53:55.884 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:55.884 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:55.884 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:55.884 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-02-18T11:53:55.884 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:55.885 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-write_buf_throttle-0x55d64f1b8960": { 2024-02-18T11:53:55.885 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-02-18T11:53:55.885 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 3758096384, 2024-02-18T11:53:55.885 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-02-18T11:53:55.885 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 0, 2024-02-18T11:53:55.885 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 0, 2024-02-18T11:53:55.885 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-02-18T11:53:55.885 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 0, 2024-02-18T11:53:55.885 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 0, 2024-02-18T11:53:55.885 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 0, 2024-02-18T11:53:55.885 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 0, 2024-02-18T11:53:55.885 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 0, 2024-02-18T11:53:55.885 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-02-18T11:53:55.885 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:55.885 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:55.885 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:55.886 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-02-18T11:53:55.886 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-02-18T11:53:55.886 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-02-18T11:53:56.780 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-02-18T11:53:56.780 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-02-18T11:53:56.781 DEBUG:tasks.check_counter:Getting stats from i 2024-02-18T11:53:56.781 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph daemon mds.i perf dump 2024-02-18T11:53:57.024 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.c/config 2024-02-18T11:53:57.375 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:53:57 smithi188 ceph-mon[36925]: pgmap v741: 97 pgs: 97 active+clean; 7.0 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 65 MiB/s wr, 466 op/s 2024-02-18T11:53:57.421 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:53:57 smithi012 ceph-mon[32460]: pgmap v741: 97 pgs: 97 active+clean; 7.0 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 65 MiB/s wr, 466 op/s 2024-02-18T11:53:57.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:53:57 smithi158 ceph-mon[36626]: pgmap v741: 97 pgs: 97 active+clean; 7.0 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 65 MiB/s wr, 466 op/s 2024-02-18T11:53:58.679 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-02-18T11:53:58.679 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-0": { 2024-02-18T11:53:58.679 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 756, 2024-02-18T11:53:58.679 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 671, 2024-02-18T11:53:58.679 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 169071, 2024-02-18T11:53:58.679 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 279060, 2024-02-18T11:53:58.679 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 9, 2024-02-18T11:53:58.680 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 2, 2024-02-18T11:53:58.680 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 0.087064526, 2024-02-18T11:53:58.680 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 0.028108246, 2024-02-18T11:53:58.680 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 0.059939235, 2024-02-18T11:53:58.680 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 0.006577632, 2024-02-18T11:53:58.680 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-02-18T11:53:58.680 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 672, 2024-02-18T11:53:58.680 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.044771861, 2024-02-18T11:53:58.680 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000066624 2024-02-18T11:53:58.680 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:58.680 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-02-18T11:53:58.680 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 1348, 2024-02-18T11:53:58.680 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000458098, 2024-02-18T11:53:58.681 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000339 2024-02-18T11:53:58.681 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:58.681 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-02-18T11:53:58.681 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 448 2024-02-18T11:53:58.681 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:58.681 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-1": { 2024-02-18T11:53:58.681 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 29, 2024-02-18T11:53:58.681 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 218, 2024-02-18T11:53:58.681 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 7498, 2024-02-18T11:53:58.681 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 525595, 2024-02-18T11:53:58.681 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 14, 2024-02-18T11:53:58.681 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 6, 2024-02-18T11:53:58.681 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 0.017026853, 2024-02-18T11:53:58.681 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 0.009421763, 2024-02-18T11:53:58.682 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 0.005443941, 2024-02-18T11:53:58.682 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 0.000245639, 2024-02-18T11:53:58.682 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-02-18T11:53:58.682 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 214, 2024-02-18T11:53:58.682 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.020694304, 2024-02-18T11:53:58.682 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000096702 2024-02-18T11:53:58.682 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:58.682 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-02-18T11:53:58.682 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 30, 2024-02-18T11:53:58.682 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000004365, 2024-02-18T11:53:58.682 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000145 2024-02-18T11:53:58.682 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:58.682 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 4416, 2024-02-18T11:53:58.683 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 501056 2024-02-18T11:53:58.683 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:58.683 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-2": { 2024-02-18T11:53:58.683 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 313, 2024-02-18T11:53:58.683 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 304, 2024-02-18T11:53:58.683 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 793406, 2024-02-18T11:53:58.683 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 185877, 2024-02-18T11:53:58.683 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 8, 2024-02-18T11:53:58.683 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 4, 2024-02-18T11:53:58.683 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 0.056333192, 2024-02-18T11:53:58.683 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 0.018788192, 2024-02-18T11:53:58.683 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 0.029360076, 2024-02-18T11:53:58.683 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 0.008746554, 2024-02-18T11:53:58.683 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-02-18T11:53:58.683 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 303, 2024-02-18T11:53:58.684 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.022811815, 2024-02-18T11:53:58.684 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000075286 2024-02-18T11:53:58.684 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:58.684 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-02-18T11:53:58.684 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 14, 2024-02-18T11:53:58.684 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000010952, 2024-02-18T11:53:58.684 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000782 2024-02-18T11:53:58.684 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:58.684 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 788784, 2024-02-18T11:53:58.684 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 154640 2024-02-18T11:53:58.684 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:58.684 INFO:teuthology.orchestra.run.smithi188.stdout: "cct": { 2024-02-18T11:53:58.684 INFO:teuthology.orchestra.run.smithi188.stdout: "total_workers": 1, 2024-02-18T11:53:58.685 INFO:teuthology.orchestra.run.smithi188.stdout: "unhealthy_workers": 0 2024-02-18T11:53:58.685 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:58.685 INFO:teuthology.orchestra.run.smithi188.stdout: "finisher-MDSRank": { 2024-02-18T11:53:58.685 INFO:teuthology.orchestra.run.smithi188.stdout: "queue_len": 0, 2024-02-18T11:53:58.685 INFO:teuthology.orchestra.run.smithi188.stdout: "complete_latency": { 2024-02-18T11:53:58.685 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 51, 2024-02-18T11:53:58.685 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.002042031, 2024-02-18T11:53:58.685 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000040039 2024-02-18T11:53:58.685 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-02-18T11:53:58.685 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:58.685 INFO:teuthology.orchestra.run.smithi188.stdout: "finisher-PurgeQueue": { 2024-02-18T11:53:58.685 INFO:teuthology.orchestra.run.smithi188.stdout: "queue_len": 0, 2024-02-18T11:53:58.685 INFO:teuthology.orchestra.run.smithi188.stdout: "complete_latency": { 2024-02-18T11:53:58.685 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 2, 2024-02-18T11:53:58.685 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000012802, 2024-02-18T11:53:58.686 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000006401 2024-02-18T11:53:58.686 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-02-18T11:53:58.686 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:58.686 INFO:teuthology.orchestra.run.smithi188.stdout: "mds": { 2024-02-18T11:53:58.686 INFO:teuthology.orchestra.run.smithi188.stdout: "request": 8, 2024-02-18T11:53:58.686 INFO:teuthology.orchestra.run.smithi188.stdout: "reply": 5, 2024-02-18T11:53:58.686 INFO:teuthology.orchestra.run.smithi188.stdout: "reply_latency": { 2024-02-18T11:53:58.686 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 5, 2024-02-18T11:53:58.686 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.002435414, 2024-02-18T11:53:58.686 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000487082 2024-02-18T11:53:58.686 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:58.686 INFO:teuthology.orchestra.run.smithi188.stdout: "slow_reply": 0, 2024-02-18T11:53:58.686 INFO:teuthology.orchestra.run.smithi188.stdout: "forward": 3, 2024-02-18T11:53:58.687 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_fetch_complete": 0, 2024-02-18T11:53:58.687 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_fetch_keys": 0, 2024-02-18T11:53:58.687 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_commit": 11, 2024-02-18T11:53:58.687 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_split": 0, 2024-02-18T11:53:58.687 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_merge": 0, 2024-02-18T11:53:58.687 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes": 12, 2024-02-18T11:53:58.687 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_top": 0, 2024-02-18T11:53:58.687 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_bottom": 12, 2024-02-18T11:53:58.687 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_pin_tail": 0, 2024-02-18T11:53:58.687 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_pinned": 11, 2024-02-18T11:53:58.687 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_expired": 0, 2024-02-18T11:53:58.687 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_with_caps": 1, 2024-02-18T11:53:58.687 INFO:teuthology.orchestra.run.smithi188.stdout: "caps": 1, 2024-02-18T11:53:58.688 INFO:teuthology.orchestra.run.smithi188.stdout: "subtrees": 2, 2024-02-18T11:53:58.688 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse": 11, 2024-02-18T11:53:58.688 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_hit": 8, 2024-02-18T11:53:58.688 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_forward": 0, 2024-02-18T11:53:58.688 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_discover": 2, 2024-02-18T11:53:58.688 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_dir_fetch": 0, 2024-02-18T11:53:58.688 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_remote_ino": 0, 2024-02-18T11:53:58.688 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_lock": 0, 2024-02-18T11:53:58.688 INFO:teuthology.orchestra.run.smithi188.stdout: "load_cent": 0, 2024-02-18T11:53:58.688 INFO:teuthology.orchestra.run.smithi188.stdout: "q": 0, 2024-02-18T11:53:58.688 INFO:teuthology.orchestra.run.smithi188.stdout: "exported": 0, 2024-02-18T11:53:58.688 INFO:teuthology.orchestra.run.smithi188.stdout: "exported_inodes": 0, 2024-02-18T11:53:58.688 INFO:teuthology.orchestra.run.smithi188.stdout: "imported": 0, 2024-02-18T11:53:58.689 INFO:teuthology.orchestra.run.smithi188.stdout: "imported_inodes": 0, 2024-02-18T11:53:58.689 INFO:teuthology.orchestra.run.smithi188.stdout: "openino_dir_fetch": 0, 2024-02-18T11:53:58.689 INFO:teuthology.orchestra.run.smithi188.stdout: "openino_backtrace_fetch": 0, 2024-02-18T11:53:58.689 INFO:teuthology.orchestra.run.smithi188.stdout: "openino_peer_discover": 0, 2024-02-18T11:53:58.689 INFO:teuthology.orchestra.run.smithi188.stdout: "root_rfiles": 4, 2024-02-18T11:53:58.689 INFO:teuthology.orchestra.run.smithi188.stdout: "root_rbytes": 234, 2024-02-18T11:53:58.689 INFO:teuthology.orchestra.run.smithi188.stdout: "root_rsnaps": 0, 2024-02-18T11:53:58.689 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_backtrace_fetch": 0, 2024-02-18T11:53:58.689 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_set_tag": 0, 2024-02-18T11:53:58.689 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_backtrace_repaired": 0, 2024-02-18T11:53:58.689 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_inotable_repaired": 0, 2024-02-18T11:53:58.689 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_dir_inodes": 0, 2024-02-18T11:53:58.689 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_dir_base_inodes": 0, 2024-02-18T11:53:58.689 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_dirfrag_rstats": 0, 2024-02-18T11:53:58.689 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_file_inodes": 0, 2024-02-18T11:53:58.690 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_inode_file_caps": 0, 2024-02-18T11:53:58.690 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_revoke": 0, 2024-02-18T11:53:58.690 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_grant": 0, 2024-02-18T11:53:58.690 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_trunc": 0, 2024-02-18T11:53:58.690 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-02-18T11:53:58.690 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_flush_ack": 0, 2024-02-18T11:53:58.690 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_caps": 0, 2024-02-18T11:53:58.690 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_caps_dirty": 0, 2024-02-18T11:53:58.690 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_cap_release": 0, 2024-02-18T11:53:58.690 INFO:teuthology.orchestra.run.smithi188.stdout: "process_request_cap_release": 1 2024-02-18T11:53:58.690 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:58.690 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_cache": { 2024-02-18T11:53:58.690 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_update": 0, 2024-02-18T11:53:58.690 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_update_receipt": 0, 2024-02-18T11:53:58.690 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_try_discover": 0, 2024-02-18T11:53:58.691 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_send_discover": 3, 2024-02-18T11:53:58.691 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_handle_discover": 0, 2024-02-18T11:53:58.691 INFO:teuthology.orchestra.run.smithi188.stdout: "num_strays": 0, 2024-02-18T11:53:58.691 INFO:teuthology.orchestra.run.smithi188.stdout: "num_strays_delayed": 0, 2024-02-18T11:53:58.691 INFO:teuthology.orchestra.run.smithi188.stdout: "num_strays_enqueuing": 0, 2024-02-18T11:53:58.691 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_created": 0, 2024-02-18T11:53:58.691 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_enqueued": 0, 2024-02-18T11:53:58.691 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_reintegrated": 0, 2024-02-18T11:53:58.691 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_migrated": 0, 2024-02-18T11:53:58.691 INFO:teuthology.orchestra.run.smithi188.stdout: "num_recovering_processing": 0, 2024-02-18T11:53:58.691 INFO:teuthology.orchestra.run.smithi188.stdout: "num_recovering_enqueued": 0, 2024-02-18T11:53:58.691 INFO:teuthology.orchestra.run.smithi188.stdout: "num_recovering_prioritized": 0, 2024-02-18T11:53:58.691 INFO:teuthology.orchestra.run.smithi188.stdout: "recovery_started": 0, 2024-02-18T11:53:58.691 INFO:teuthology.orchestra.run.smithi188.stdout: "recovery_completed": 0, 2024-02-18T11:53:58.692 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_enqueue_scrub": 0, 2024-02-18T11:53:58.692 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_exportdir": 0, 2024-02-18T11:53:58.692 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_flush": 0, 2024-02-18T11:53:58.692 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_fragmentdir": 0, 2024-02-18T11:53:58.692 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_fragstats": 0, 2024-02-18T11:53:58.692 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_inodestats": 0 2024-02-18T11:53:58.692 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:58.692 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_log": { 2024-02-18T11:53:58.692 INFO:teuthology.orchestra.run.smithi188.stdout: "evlrg": 0, 2024-02-18T11:53:58.692 INFO:teuthology.orchestra.run.smithi188.stdout: "evadd": 7, 2024-02-18T11:53:58.692 INFO:teuthology.orchestra.run.smithi188.stdout: "evex": 0, 2024-02-18T11:53:58.692 INFO:teuthology.orchestra.run.smithi188.stdout: "evtrm": 0, 2024-02-18T11:53:58.692 INFO:teuthology.orchestra.run.smithi188.stdout: "ev": 7, 2024-02-18T11:53:58.693 INFO:teuthology.orchestra.run.smithi188.stdout: "evexg": 0, 2024-02-18T11:53:58.693 INFO:teuthology.orchestra.run.smithi188.stdout: "evexd": 0, 2024-02-18T11:53:58.693 INFO:teuthology.orchestra.run.smithi188.stdout: "segadd": 2, 2024-02-18T11:53:58.693 INFO:teuthology.orchestra.run.smithi188.stdout: "segex": 0, 2024-02-18T11:53:58.693 INFO:teuthology.orchestra.run.smithi188.stdout: "segtrm": 0, 2024-02-18T11:53:58.693 INFO:teuthology.orchestra.run.smithi188.stdout: "seg": 2, 2024-02-18T11:53:58.693 INFO:teuthology.orchestra.run.smithi188.stdout: "segmjr": 2, 2024-02-18T11:53:58.693 INFO:teuthology.orchestra.run.smithi188.stdout: "segexg": 0, 2024-02-18T11:53:58.693 INFO:teuthology.orchestra.run.smithi188.stdout: "segexd": 0, 2024-02-18T11:53:58.693 INFO:teuthology.orchestra.run.smithi188.stdout: "expos": 4194304, 2024-02-18T11:53:58.693 INFO:teuthology.orchestra.run.smithi188.stdout: "wrpos": 4196264, 2024-02-18T11:53:58.693 INFO:teuthology.orchestra.run.smithi188.stdout: "rdpos": 0, 2024-02-18T11:53:58.693 INFO:teuthology.orchestra.run.smithi188.stdout: "jlat": { 2024-02-18T11:53:58.693 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 6, 2024-02-18T11:53:58.693 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.010483357, 2024-02-18T11:53:58.693 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.001747226 2024-02-18T11:53:58.694 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:58.694 INFO:teuthology.orchestra.run.smithi188.stdout: "replayed": 0 2024-02-18T11:53:58.694 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:58.694 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_mem": { 2024-02-18T11:53:58.694 INFO:teuthology.orchestra.run.smithi188.stdout: "ino": 15, 2024-02-18T11:53:58.694 INFO:teuthology.orchestra.run.smithi188.stdout: "ino+": 15, 2024-02-18T11:53:58.694 INFO:teuthology.orchestra.run.smithi188.stdout: "ino-": 0, 2024-02-18T11:53:58.694 INFO:teuthology.orchestra.run.smithi188.stdout: "dir": 13, 2024-02-18T11:53:58.694 INFO:teuthology.orchestra.run.smithi188.stdout: "dir+": 14, 2024-02-18T11:53:58.694 INFO:teuthology.orchestra.run.smithi188.stdout: "dir-": 1, 2024-02-18T11:53:58.694 INFO:teuthology.orchestra.run.smithi188.stdout: "dn": 12, 2024-02-18T11:53:58.694 INFO:teuthology.orchestra.run.smithi188.stdout: "dn+": 12, 2024-02-18T11:53:58.694 INFO:teuthology.orchestra.run.smithi188.stdout: "dn-": 0, 2024-02-18T11:53:58.694 INFO:teuthology.orchestra.run.smithi188.stdout: "cap": 1, 2024-02-18T11:53:58.695 INFO:teuthology.orchestra.run.smithi188.stdout: "cap+": 5, 2024-02-18T11:53:58.695 INFO:teuthology.orchestra.run.smithi188.stdout: "cap-": 4, 2024-02-18T11:53:58.695 INFO:teuthology.orchestra.run.smithi188.stdout: "rss": 49780, 2024-02-18T11:53:58.695 INFO:teuthology.orchestra.run.smithi188.stdout: "heap": 207228 2024-02-18T11:53:58.695 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:58.695 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_server": { 2024-02-18T11:53:58.695 INFO:teuthology.orchestra.run.smithi188.stdout: "dispatch_client_request": 13, 2024-02-18T11:53:58.695 INFO:teuthology.orchestra.run.smithi188.stdout: "dispatch_server_request": 0, 2024-02-18T11:53:58.695 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_request": 8, 2024-02-18T11:53:58.695 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_session": 19, 2024-02-18T11:53:58.695 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_peer_request": 1, 2024-02-18T11:53:58.695 INFO:teuthology.orchestra.run.smithi188.stdout: "req_create_latency": { 2024-02-18T11:53:58.695 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:58.695 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:58.695 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:58.695 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:58.696 INFO:teuthology.orchestra.run.smithi188.stdout: "req_getattr_latency": { 2024-02-18T11:53:58.696 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 2, 2024-02-18T11:53:58.696 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.001674158, 2024-02-18T11:53:58.696 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000837079 2024-02-18T11:53:58.696 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:58.696 INFO:teuthology.orchestra.run.smithi188.stdout: "req_getfilelock_latency": { 2024-02-18T11:53:58.696 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:58.696 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:58.696 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:58.696 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:58.696 INFO:teuthology.orchestra.run.smithi188.stdout: "req_link_latency": { 2024-02-18T11:53:58.696 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:58.696 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:58.697 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:58.697 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:58.697 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookup_latency": { 2024-02-18T11:53:58.697 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 2, 2024-02-18T11:53:58.697 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000656888, 2024-02-18T11:53:58.697 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000328444 2024-02-18T11:53:58.697 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:58.697 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookuphash_latency": { 2024-02-18T11:53:58.697 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:58.697 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:58.697 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:58.698 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:58.698 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupino_latency": { 2024-02-18T11:53:58.698 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 1, 2024-02-18T11:53:58.698 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000104368, 2024-02-18T11:53:58.698 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000104368 2024-02-18T11:53:58.698 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:58.698 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupname_latency": { 2024-02-18T11:53:58.698 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:58.698 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:58.698 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:58.698 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:58.698 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupparent_latency": { 2024-02-18T11:53:58.698 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:58.698 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:58.699 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:58.699 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:58.699 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupsnap_latency": { 2024-02-18T11:53:58.699 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:58.699 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:58.699 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:58.699 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:58.699 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lssnap_latency": { 2024-02-18T11:53:58.699 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:58.699 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:58.699 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:58.699 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:58.699 INFO:teuthology.orchestra.run.smithi188.stdout: "req_mkdir_latency": { 2024-02-18T11:53:58.699 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:58.699 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:58.700 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:58.700 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:58.700 INFO:teuthology.orchestra.run.smithi188.stdout: "req_mknod_latency": { 2024-02-18T11:53:58.700 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:58.700 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:58.701 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:58.701 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:58.701 INFO:teuthology.orchestra.run.smithi188.stdout: "req_mksnap_latency": { 2024-02-18T11:53:58.701 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:58.701 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:58.701 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:58.701 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:58.701 INFO:teuthology.orchestra.run.smithi188.stdout: "req_open_latency": { 2024-02-18T11:53:58.701 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:58.701 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:58.701 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:58.701 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:58.701 INFO:teuthology.orchestra.run.smithi188.stdout: "req_readdir_latency": { 2024-02-18T11:53:58.701 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:58.702 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:58.702 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:58.702 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:58.702 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rename_latency": { 2024-02-18T11:53:58.702 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:58.702 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:58.702 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:58.702 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:58.702 INFO:teuthology.orchestra.run.smithi188.stdout: "req_renamesnap_latency": { 2024-02-18T11:53:58.702 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:58.702 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:58.702 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:58.702 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:58.702 INFO:teuthology.orchestra.run.smithi188.stdout: "req_snapdiff_latency": { 2024-02-18T11:53:58.702 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:58.702 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:58.703 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:58.703 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:58.703 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rmdir_latency": { 2024-02-18T11:53:58.703 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:58.703 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:58.703 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:58.703 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:58.703 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rmsnap_latency": { 2024-02-18T11:53:58.703 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:58.703 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:58.703 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:58.703 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:58.703 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rmxattr_latency": { 2024-02-18T11:53:58.703 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:58.703 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:58.704 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:58.704 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:58.704 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setattr_latency": { 2024-02-18T11:53:58.704 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:58.704 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:58.704 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:58.704 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:58.704 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setdirlayout_latency": { 2024-02-18T11:53:58.704 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:58.704 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:58.704 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:58.704 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:58.704 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setfilelock_latency": { 2024-02-18T11:53:58.704 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:58.705 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:58.705 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:58.705 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:58.705 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setlayout_latency": { 2024-02-18T11:53:58.705 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:58.705 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:58.705 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:58.705 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:58.705 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setxattr_latency": { 2024-02-18T11:53:58.705 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:58.705 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:58.705 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:58.705 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:58.705 INFO:teuthology.orchestra.run.smithi188.stdout: "req_symlink_latency": { 2024-02-18T11:53:58.705 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:58.706 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:58.706 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:58.706 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:58.706 INFO:teuthology.orchestra.run.smithi188.stdout: "req_unlink_latency": { 2024-02-18T11:53:58.706 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:58.706 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:58.706 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:58.706 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:58.706 INFO:teuthology.orchestra.run.smithi188.stdout: "cap_revoke_eviction": 0, 2024-02-18T11:53:58.706 INFO:teuthology.orchestra.run.smithi188.stdout: "cap_acquisition_throttle": 0, 2024-02-18T11:53:58.706 INFO:teuthology.orchestra.run.smithi188.stdout: "req_getvxattr_latency": { 2024-02-18T11:53:58.706 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:58.707 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:58.707 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:58.707 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-02-18T11:53:58.707 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:58.707 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_sessions": { 2024-02-18T11:53:58.707 INFO:teuthology.orchestra.run.smithi188.stdout: "session_count": 1, 2024-02-18T11:53:58.707 INFO:teuthology.orchestra.run.smithi188.stdout: "session_add": 3, 2024-02-18T11:53:58.707 INFO:teuthology.orchestra.run.smithi188.stdout: "session_remove": 2, 2024-02-18T11:53:58.707 INFO:teuthology.orchestra.run.smithi188.stdout: "sessions_open": 1, 2024-02-18T11:53:58.707 INFO:teuthology.orchestra.run.smithi188.stdout: "sessions_stale": 0, 2024-02-18T11:53:58.707 INFO:teuthology.orchestra.run.smithi188.stdout: "total_load": 4, 2024-02-18T11:53:58.707 INFO:teuthology.orchestra.run.smithi188.stdout: "average_load": 2, 2024-02-18T11:53:58.708 INFO:teuthology.orchestra.run.smithi188.stdout: "avg_session_uptime": 797, 2024-02-18T11:53:58.708 INFO:teuthology.orchestra.run.smithi188.stdout: "mdthresh_evicted": 0 2024-02-18T11:53:58.708 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:58.708 INFO:teuthology.orchestra.run.smithi188.stdout: "mempool": { 2024-02-18T11:53:58.708 INFO:teuthology.orchestra.run.smithi188.stdout: "bloom_filter_bytes": 0, 2024-02-18T11:53:58.708 INFO:teuthology.orchestra.run.smithi188.stdout: "bloom_filter_items": 0, 2024-02-18T11:53:58.708 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_alloc_bytes": 0, 2024-02-18T11:53:58.708 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_alloc_items": 0, 2024-02-18T11:53:58.708 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_data_bytes": 0, 2024-02-18T11:53:58.708 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_data_items": 0, 2024-02-18T11:53:58.708 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-18T11:53:58.708 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_onode_items": 0, 2024-02-18T11:53:58.708 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-18T11:53:58.709 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_meta_items": 0, 2024-02-18T11:53:58.709 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_other_bytes": 0, 2024-02-18T11:53:58.709 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_other_items": 0, 2024-02-18T11:53:58.709 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-18T11:53:58.709 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_buffer_items": 0, 2024-02-18T11:53:58.709 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_extent_bytes": 0, 2024-02-18T11:53:58.709 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_extent_items": 0, 2024-02-18T11:53:58.709 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_blob_bytes": 0, 2024-02-18T11:53:58.709 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_blob_items": 0, 2024-02-18T11:53:58.709 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-18T11:53:58.709 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_shared_blob_items": 0, 2024-02-18T11:53:58.709 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-18T11:53:58.709 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_inline_bl_items": 0, 2024-02-18T11:53:58.709 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_fsck_bytes": 0, 2024-02-18T11:53:58.710 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_fsck_items": 0, 2024-02-18T11:53:58.710 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_txc_bytes": 0, 2024-02-18T11:53:58.710 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_txc_items": 0, 2024-02-18T11:53:58.710 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-18T11:53:58.710 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_deferred_items": 0, 2024-02-18T11:53:58.710 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_bytes": 0, 2024-02-18T11:53:58.710 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_items": 0, 2024-02-18T11:53:58.710 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_bytes": 0, 2024-02-18T11:53:58.710 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_items": 0, 2024-02-18T11:53:58.710 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_reader_bytes": 0, 2024-02-18T11:53:58.710 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_reader_items": 0, 2024-02-18T11:53:58.710 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_writer_bytes": 0, 2024-02-18T11:53:58.710 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_writer_items": 0, 2024-02-18T11:53:58.710 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_anon_bytes": 239893, 2024-02-18T11:53:58.710 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_anon_items": 85, 2024-02-18T11:53:58.711 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_meta_bytes": 176, 2024-02-18T11:53:58.711 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_meta_items": 2, 2024-02-18T11:53:58.711 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_bytes": 0, 2024-02-18T11:53:58.711 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_items": 0, 2024-02-18T11:53:58.711 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_mapbl_bytes": 0, 2024-02-18T11:53:58.711 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_mapbl_items": 0, 2024-02-18T11:53:58.711 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_pglog_bytes": 0, 2024-02-18T11:53:58.711 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_pglog_items": 0, 2024-02-18T11:53:58.711 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_bytes": 7528, 2024-02-18T11:53:58.711 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_items": 158, 2024-02-18T11:53:58.711 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_mapping_bytes": 0, 2024-02-18T11:53:58.711 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_mapping_items": 0, 2024-02-18T11:53:58.711 INFO:teuthology.orchestra.run.smithi188.stdout: "pgmap_bytes": 0, 2024-02-18T11:53:58.711 INFO:teuthology.orchestra.run.smithi188.stdout: "pgmap_items": 0, 2024-02-18T11:53:58.712 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_co_bytes": 58760, 2024-02-18T11:53:58.712 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_co_items": 238, 2024-02-18T11:53:58.712 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_1_bytes": 0, 2024-02-18T11:53:58.712 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_1_items": 0, 2024-02-18T11:53:58.712 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_2_bytes": 0, 2024-02-18T11:53:58.712 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_2_items": 0 2024-02-18T11:53:58.712 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:58.712 INFO:teuthology.orchestra.run.smithi188.stdout: "objecter": { 2024-02-18T11:53:58.712 INFO:teuthology.orchestra.run.smithi188.stdout: "op_active": 0, 2024-02-18T11:53:58.712 INFO:teuthology.orchestra.run.smithi188.stdout: "op_laggy": 0, 2024-02-18T11:53:58.712 INFO:teuthology.orchestra.run.smithi188.stdout: "op_send": 45, 2024-02-18T11:53:58.712 INFO:teuthology.orchestra.run.smithi188.stdout: "op_send_bytes": 12577, 2024-02-18T11:53:58.712 INFO:teuthology.orchestra.run.smithi188.stdout: "op_resend": 27, 2024-02-18T11:53:58.713 INFO:teuthology.orchestra.run.smithi188.stdout: "op_reply": 45, 2024-02-18T11:53:58.713 INFO:teuthology.orchestra.run.smithi188.stdout: "op_latency": { 2024-02-18T11:53:58.713 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 45, 2024-02-18T11:53:58.713 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.271995272, 2024-02-18T11:53:58.713 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.006044339 2024-02-18T11:53:58.713 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:58.713 INFO:teuthology.orchestra.run.smithi188.stdout: "op_inflight": 0, 2024-02-18T11:53:58.713 INFO:teuthology.orchestra.run.smithi188.stdout: "oplen_avg": { 2024-02-18T11:53:58.713 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 45, 2024-02-18T11:53:58.713 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 69 2024-02-18T11:53:58.713 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:58.713 INFO:teuthology.orchestra.run.smithi188.stdout: "op": 45, 2024-02-18T11:53:58.713 INFO:teuthology.orchestra.run.smithi188.stdout: "op_r": 0, 2024-02-18T11:53:58.713 INFO:teuthology.orchestra.run.smithi188.stdout: "op_w": 45, 2024-02-18T11:53:58.714 INFO:teuthology.orchestra.run.smithi188.stdout: "op_rmw": 0, 2024-02-18T11:53:58.714 INFO:teuthology.orchestra.run.smithi188.stdout: "op_pg": 0, 2024-02-18T11:53:58.714 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_stat": 0, 2024-02-18T11:53:58.714 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_create": 10, 2024-02-18T11:53:58.714 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_read": 0, 2024-02-18T11:53:58.714 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_write": 6, 2024-02-18T11:53:58.714 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_writefull": 8, 2024-02-18T11:53:58.714 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_writesame": 0, 2024-02-18T11:53:58.714 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_append": 0, 2024-02-18T11:53:58.714 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_zero": 0, 2024-02-18T11:53:58.714 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_truncate": 0, 2024-02-18T11:53:58.714 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_delete": 6, 2024-02-18T11:53:58.714 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_mapext": 0, 2024-02-18T11:53:58.714 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_sparse_read": 0, 2024-02-18T11:53:58.714 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_clonerange": 0, 2024-02-18T11:53:58.715 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_getxattr": 0, 2024-02-18T11:53:58.715 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_setxattr": 20, 2024-02-18T11:53:58.715 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_cmpxattr": 0, 2024-02-18T11:53:58.715 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_rmxattr": 0, 2024-02-18T11:53:58.715 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_resetxattrs": 0, 2024-02-18T11:53:58.715 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_call": 0, 2024-02-18T11:53:58.715 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_watch": 0, 2024-02-18T11:53:58.715 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_notify": 0, 2024-02-18T11:53:58.715 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_src_cmpxattr": 0, 2024-02-18T11:53:58.715 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_pgls": 0, 2024-02-18T11:53:58.715 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_pgls_filter": 0, 2024-02-18T11:53:58.715 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_other": 0, 2024-02-18T11:53:58.715 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_active": 0, 2024-02-18T11:53:58.715 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_send": 0, 2024-02-18T11:53:58.716 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_resend": 0, 2024-02-18T11:53:58.716 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_ping": 0, 2024-02-18T11:53:58.716 INFO:teuthology.orchestra.run.smithi188.stdout: "poolop_active": 0, 2024-02-18T11:53:58.716 INFO:teuthology.orchestra.run.smithi188.stdout: "poolop_send": 0, 2024-02-18T11:53:58.716 INFO:teuthology.orchestra.run.smithi188.stdout: "poolop_resend": 0, 2024-02-18T11:53:58.716 INFO:teuthology.orchestra.run.smithi188.stdout: "poolstat_active": 0, 2024-02-18T11:53:58.716 INFO:teuthology.orchestra.run.smithi188.stdout: "poolstat_send": 0, 2024-02-18T11:53:58.716 INFO:teuthology.orchestra.run.smithi188.stdout: "poolstat_resend": 0, 2024-02-18T11:53:58.716 INFO:teuthology.orchestra.run.smithi188.stdout: "statfs_active": 0, 2024-02-18T11:53:58.716 INFO:teuthology.orchestra.run.smithi188.stdout: "statfs_send": 0, 2024-02-18T11:53:58.716 INFO:teuthology.orchestra.run.smithi188.stdout: "statfs_resend": 0, 2024-02-18T11:53:58.716 INFO:teuthology.orchestra.run.smithi188.stdout: "command_active": 0, 2024-02-18T11:53:58.716 INFO:teuthology.orchestra.run.smithi188.stdout: "command_send": 0, 2024-02-18T11:53:58.716 INFO:teuthology.orchestra.run.smithi188.stdout: "command_resend": 0, 2024-02-18T11:53:58.717 INFO:teuthology.orchestra.run.smithi188.stdout: "map_epoch": 89, 2024-02-18T11:53:58.717 INFO:teuthology.orchestra.run.smithi188.stdout: "map_full": 0, 2024-02-18T11:53:58.717 INFO:teuthology.orchestra.run.smithi188.stdout: "map_inc": 2, 2024-02-18T11:53:58.717 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_sessions": 9, 2024-02-18T11:53:58.717 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_session_open": 12, 2024-02-18T11:53:58.717 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_session_close": 3, 2024-02-18T11:53:58.717 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_laggy": 0, 2024-02-18T11:53:58.717 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_wr": 17, 2024-02-18T11:53:58.717 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_rd": 0, 2024-02-18T11:53:58.717 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_del": 2 2024-02-18T11:53:58.717 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:58.717 INFO:teuthology.orchestra.run.smithi188.stdout: "oft": { 2024-02-18T11:53:58.717 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_objs": 1, 2024-02-18T11:53:58.717 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_kv_pairs": 0, 2024-02-18T11:53:58.717 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_updates": 0, 2024-02-18T11:53:58.718 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_removes": 0 2024-02-18T11:53:58.718 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:58.718 INFO:teuthology.orchestra.run.smithi188.stdout: "osd-slow-ops": { 2024-02-18T11:53:58.718 INFO:teuthology.orchestra.run.smithi188.stdout: "slow_ops_count": 0 2024-02-18T11:53:58.718 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:58.718 INFO:teuthology.orchestra.run.smithi188.stdout: "purge_queue": { 2024-02-18T11:53:58.718 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing_ops": 0, 2024-02-18T11:53:58.718 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing_ops_high_water": 0, 2024-02-18T11:53:58.718 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing": 0, 2024-02-18T11:53:58.718 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing_high_water": 0, 2024-02-18T11:53:58.718 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executed_ops": 0, 2024-02-18T11:53:58.718 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executed": 0, 2024-02-18T11:53:58.718 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_item_in_journal": 0 2024-02-18T11:53:58.718 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:58.718 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-18T11:53:58.719 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-02-18T11:53:58.719 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 104857600, 2024-02-18T11:53:58.719 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-02-18T11:53:58.719 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 1094, 2024-02-18T11:53:58.719 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 853296, 2024-02-18T11:53:58.719 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-02-18T11:53:58.719 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 1094, 2024-02-18T11:53:58.719 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 0, 2024-02-18T11:53:58.719 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 0, 2024-02-18T11:53:58.719 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 1094, 2024-02-18T11:53:58.719 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 853296, 2024-02-18T11:53:58.719 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-02-18T11:53:58.719 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:58.719 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:58.719 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:58.720 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-02-18T11:53:58.720 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:58.720 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-objecter_bytes": { 2024-02-18T11:53:58.720 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-02-18T11:53:58.720 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 104857600, 2024-02-18T11:53:58.720 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-02-18T11:53:58.720 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 0, 2024-02-18T11:53:58.720 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 0, 2024-02-18T11:53:58.720 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-02-18T11:53:58.720 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 0, 2024-02-18T11:53:58.720 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 45, 2024-02-18T11:53:58.720 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 12577, 2024-02-18T11:53:58.720 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 45, 2024-02-18T11:53:58.720 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 12577, 2024-02-18T11:53:58.720 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-02-18T11:53:58.721 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:58.721 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:58.721 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:58.721 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-02-18T11:53:58.721 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:58.721 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-objecter_ops": { 2024-02-18T11:53:58.721 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-02-18T11:53:58.721 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 1024, 2024-02-18T11:53:58.721 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-02-18T11:53:58.721 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 0, 2024-02-18T11:53:58.721 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 0, 2024-02-18T11:53:58.721 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-02-18T11:53:58.721 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 0, 2024-02-18T11:53:58.721 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 45, 2024-02-18T11:53:58.722 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 45, 2024-02-18T11:53:58.722 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 45, 2024-02-18T11:53:58.722 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 45, 2024-02-18T11:53:58.722 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-02-18T11:53:58.722 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:58.722 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:58.722 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:58.722 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-02-18T11:53:58.722 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:58.722 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-write_buf_throttle": { 2024-02-18T11:53:58.722 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-02-18T11:53:58.722 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 3758096384, 2024-02-18T11:53:58.722 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-02-18T11:53:58.722 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 0, 2024-02-18T11:53:58.723 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 0, 2024-02-18T11:53:58.723 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-02-18T11:53:58.723 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 0, 2024-02-18T11:53:58.723 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 0, 2024-02-18T11:53:58.723 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 0, 2024-02-18T11:53:58.723 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 0, 2024-02-18T11:53:58.723 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 0, 2024-02-18T11:53:58.723 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-02-18T11:53:58.723 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:58.723 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:58.723 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:58.723 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-02-18T11:53:58.723 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:53:58.723 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-write_buf_throttle-0x55d50b8d06e0": { 2024-02-18T11:53:58.724 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-02-18T11:53:58.724 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 3758096384, 2024-02-18T11:53:58.724 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-02-18T11:53:58.724 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 7, 2024-02-18T11:53:58.724 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 1960, 2024-02-18T11:53:58.724 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-02-18T11:53:58.724 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 7, 2024-02-18T11:53:58.724 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 0, 2024-02-18T11:53:58.724 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 0, 2024-02-18T11:53:58.724 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 6, 2024-02-18T11:53:58.724 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 1960, 2024-02-18T11:53:58.724 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-02-18T11:53:58.724 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:53:58.724 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:53:58.724 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:53:58.725 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-02-18T11:53:58.725 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-02-18T11:53:58.725 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-02-18T11:53:59.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:53:59 smithi012 ceph-mon[32460]: pgmap v742: 97 pgs: 97 active+clean; 5.7 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 59 MiB/s wr, 471 op/s 2024-02-18T11:53:59.468 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:53:59 smithi188 ceph-mon[36925]: pgmap v742: 97 pgs: 97 active+clean; 5.7 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 59 MiB/s wr, 471 op/s 2024-02-18T11:53:59.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:53:59 smithi158 ceph-mon[36626]: pgmap v742: 97 pgs: 97 active+clean; 5.7 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 59 MiB/s wr, 471 op/s 2024-02-18T11:53:59.494 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-02-18T11:53:59.494 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-02-18T11:53:59.494 DEBUG:tasks.check_counter:Getting stats from l 2024-02-18T11:53:59.495 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22 shell --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 -- ceph daemon mds.l perf dump 2024-02-18T11:53:59.731 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/mon.c/config 2024-02-18T11:54:01.241 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:54:01 smithi188 ceph-mon[36925]: pgmap v743: 97 pgs: 97 active+clean; 4.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 40 MiB/s wr, 410 op/s 2024-02-18T11:54:01.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:54:01 smithi012 ceph-mon[32460]: pgmap v743: 97 pgs: 97 active+clean; 4.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 40 MiB/s wr, 410 op/s 2024-02-18T11:54:01.492 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:54:01 smithi158 ceph-mon[36626]: pgmap v743: 97 pgs: 97 active+clean; 4.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 40 MiB/s wr, 410 op/s 2024-02-18T11:54:01.510 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-02-18T11:54:01.510 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-0": { 2024-02-18T11:54:01.510 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 0, 2024-02-18T11:54:01.510 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 2, 2024-02-18T11:54:01.510 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 0, 2024-02-18T11:54:01.511 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 0, 2024-02-18T11:54:01.511 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 4, 2024-02-18T11:54:01.511 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 0, 2024-02-18T11:54:01.511 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 0.001075069, 2024-02-18T11:54:01.511 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 0.000000000, 2024-02-18T11:54:01.511 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 0.000287805, 2024-02-18T11:54:01.511 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-18T11:54:01.511 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-02-18T11:54:01.511 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:54:01.511 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:54:01.511 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:54:01.511 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:54:01.511 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-02-18T11:54:01.511 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:54:01.511 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:54:01.512 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:54:01.512 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:54:01.512 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 0, 2024-02-18T11:54:01.512 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 0 2024-02-18T11:54:01.512 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:54:01.512 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-1": { 2024-02-18T11:54:01.512 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 6, 2024-02-18T11:54:01.512 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 197, 2024-02-18T11:54:01.512 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 3344, 2024-02-18T11:54:01.512 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 189840, 2024-02-18T11:54:01.512 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 4, 2024-02-18T11:54:01.512 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 2, 2024-02-18T11:54:01.512 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 0.012103410, 2024-02-18T11:54:01.512 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 0.008565422, 2024-02-18T11:54:01.513 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 0.001500797, 2024-02-18T11:54:01.513 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-18T11:54:01.513 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-02-18T11:54:01.513 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 196, 2024-02-18T11:54:01.513 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.011690836, 2024-02-18T11:54:01.513 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000059647 2024-02-18T11:54:01.513 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:54:01.513 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-02-18T11:54:01.513 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:54:01.513 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:54:01.513 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:54:01.513 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:54:01.513 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-02-18T11:54:01.513 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 189840 2024-02-18T11:54:01.513 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:54:01.513 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-2": { 2024-02-18T11:54:01.514 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 259, 2024-02-18T11:54:01.514 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 254, 2024-02-18T11:54:01.514 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 762208, 2024-02-18T11:54:01.514 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 152736, 2024-02-18T11:54:01.514 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 2, 2024-02-18T11:54:01.514 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 1, 2024-02-18T11:54:01.514 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 0.045148061, 2024-02-18T11:54:01.514 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 0.015841481, 2024-02-18T11:54:01.514 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 0.022560086, 2024-02-18T11:54:01.514 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 0.008185730, 2024-02-18T11:54:01.514 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-02-18T11:54:01.514 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 253, 2024-02-18T11:54:01.514 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.014426909, 2024-02-18T11:54:01.514 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000057023 2024-02-18T11:54:01.514 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:54:01.515 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-02-18T11:54:01.515 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:54:01.515 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:54:01.515 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:54:01.515 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:54:01.515 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 762208, 2024-02-18T11:54:01.515 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 152736 2024-02-18T11:54:01.515 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:54:01.515 INFO:teuthology.orchestra.run.smithi188.stdout: "cct": { 2024-02-18T11:54:01.515 INFO:teuthology.orchestra.run.smithi188.stdout: "total_workers": 0, 2024-02-18T11:54:01.515 INFO:teuthology.orchestra.run.smithi188.stdout: "unhealthy_workers": 0 2024-02-18T11:54:01.515 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:54:01.515 INFO:teuthology.orchestra.run.smithi188.stdout: "mempool": { 2024-02-18T11:54:01.515 INFO:teuthology.orchestra.run.smithi188.stdout: "bloom_filter_bytes": 0, 2024-02-18T11:54:01.515 INFO:teuthology.orchestra.run.smithi188.stdout: "bloom_filter_items": 0, 2024-02-18T11:54:01.515 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_alloc_bytes": 0, 2024-02-18T11:54:01.516 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_alloc_items": 0, 2024-02-18T11:54:01.516 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_data_bytes": 0, 2024-02-18T11:54:01.516 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_data_items": 0, 2024-02-18T11:54:01.516 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-18T11:54:01.516 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_onode_items": 0, 2024-02-18T11:54:01.516 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-18T11:54:01.516 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_meta_items": 0, 2024-02-18T11:54:01.516 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_other_bytes": 0, 2024-02-18T11:54:01.516 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_other_items": 0, 2024-02-18T11:54:01.516 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-18T11:54:01.516 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_buffer_items": 0, 2024-02-18T11:54:01.516 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_extent_bytes": 0, 2024-02-18T11:54:01.516 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_extent_items": 0, 2024-02-18T11:54:01.516 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_blob_bytes": 0, 2024-02-18T11:54:01.516 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_blob_items": 0, 2024-02-18T11:54:01.517 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-18T11:54:01.517 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_shared_blob_items": 0, 2024-02-18T11:54:01.517 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-18T11:54:01.517 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_inline_bl_items": 0, 2024-02-18T11:54:01.517 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_fsck_bytes": 0, 2024-02-18T11:54:01.517 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_fsck_items": 0, 2024-02-18T11:54:01.517 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_txc_bytes": 0, 2024-02-18T11:54:01.517 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_txc_items": 0, 2024-02-18T11:54:01.517 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-18T11:54:01.517 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_deferred_items": 0, 2024-02-18T11:54:01.517 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_bytes": 0, 2024-02-18T11:54:01.517 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_items": 0, 2024-02-18T11:54:01.517 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_bytes": 0, 2024-02-18T11:54:01.517 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_items": 0, 2024-02-18T11:54:01.517 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_reader_bytes": 0, 2024-02-18T11:54:01.517 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_reader_items": 0, 2024-02-18T11:54:01.518 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_writer_bytes": 0, 2024-02-18T11:54:01.518 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_writer_items": 0, 2024-02-18T11:54:01.518 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_anon_bytes": 148704, 2024-02-18T11:54:01.518 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_anon_items": 29, 2024-02-18T11:54:01.518 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_meta_bytes": 0, 2024-02-18T11:54:01.518 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_meta_items": 0, 2024-02-18T11:54:01.518 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_bytes": 0, 2024-02-18T11:54:01.518 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_items": 0, 2024-02-18T11:54:01.518 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_mapbl_bytes": 0, 2024-02-18T11:54:01.518 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_mapbl_items": 0, 2024-02-18T11:54:01.518 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_pglog_bytes": 0, 2024-02-18T11:54:01.518 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_pglog_items": 0, 2024-02-18T11:54:01.518 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_bytes": 0, 2024-02-18T11:54:01.518 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_items": 0, 2024-02-18T11:54:01.518 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_mapping_bytes": 0, 2024-02-18T11:54:01.518 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_mapping_items": 0, 2024-02-18T11:54:01.519 INFO:teuthology.orchestra.run.smithi188.stdout: "pgmap_bytes": 0, 2024-02-18T11:54:01.519 INFO:teuthology.orchestra.run.smithi188.stdout: "pgmap_items": 0, 2024-02-18T11:54:01.519 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_co_bytes": 928, 2024-02-18T11:54:01.519 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_co_items": 2, 2024-02-18T11:54:01.519 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_1_bytes": 0, 2024-02-18T11:54:01.519 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_1_items": 0, 2024-02-18T11:54:01.519 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_2_bytes": 0, 2024-02-18T11:54:01.519 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_2_items": 0 2024-02-18T11:54:01.519 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-02-18T11:54:01.519 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-18T11:54:01.519 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-02-18T11:54:01.519 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 104857600, 2024-02-18T11:54:01.519 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-02-18T11:54:01.519 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 261, 2024-02-18T11:54:01.519 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 725662, 2024-02-18T11:54:01.519 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-02-18T11:54:01.520 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 261, 2024-02-18T11:54:01.520 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 0, 2024-02-18T11:54:01.520 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 0, 2024-02-18T11:54:01.520 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 261, 2024-02-18T11:54:01.520 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 725662, 2024-02-18T11:54:01.520 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-02-18T11:54:01.520 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-02-18T11:54:01.520 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-02-18T11:54:01.520 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-02-18T11:54:01.520 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-02-18T11:54:01.520 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-02-18T11:54:01.520 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-02-18T11:54:02.194 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-02-18T11:54:02.194 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-02-18T11:54:02.194 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-02-18T11:54:02.204 INFO:tasks.kclient:Unmounting kernel clients... 2024-02-18T11:54:02.204 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-02-18T11:54:02.205 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-18T11:54:02.225 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-02-18T11:54:02.225 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-18T11:54:02.285 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-18T11:54:02.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.448+0000 7fe7d9d3e640 1 -- 172.21.15.12:0/1147762601 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7d4101c80 msgr2=0x7fe7d4104070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:54:02.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.448+0000 7fe7d9d3e640 1 --2- 172.21.15.12:0/1147762601 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7d4101c80 0x7fe7d4104070 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fe7c4009900 tx=0x7fe7c402bc30 comp rx=0 tx=0).stop 2024-02-18T11:54:02.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.448+0000 7fe7d9d3e640 1 -- 172.21.15.12:0/1147762601 shutdown_connections 2024-02-18T11:54:02.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.448+0000 7fe7d9d3e640 1 --2- 172.21.15.12:0/1147762601 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7d4106ee0 0x7fe7d4109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:02.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.448+0000 7fe7d9d3e640 1 --2- 172.21.15.12:0/1147762601 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe7d41045b0 0x7fe7d41069a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:02.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.448+0000 7fe7d9d3e640 1 --2- 172.21.15.12:0/1147762601 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7d4101c80 0x7fe7d4104070 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:02.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.448+0000 7fe7d9d3e640 1 -- 172.21.15.12:0/1147762601 >> 172.21.15.12:0/1147762601 conn(0x7fe7d40fb670 msgr2=0x7fe7d40fdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:54:02.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.448+0000 7fe7d9d3e640 1 -- 172.21.15.12:0/1147762601 shutdown_connections 2024-02-18T11:54:02.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.448+0000 7fe7d9d3e640 1 -- 172.21.15.12:0/1147762601 wait complete. 2024-02-18T11:54:02.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.448+0000 7fe7d9d3e640 1 Processor -- start 2024-02-18T11:54:02.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.449+0000 7fe7d9d3e640 1 -- start start 2024-02-18T11:54:02.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.449+0000 7fe7d9d3e640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe7d4101c80 0x7fe7d4197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:54:02.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.449+0000 7fe7d9d3e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7d41045b0 0x7fe7d4197770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:54:02.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.449+0000 7fe7d9d3e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7d4106ee0 0x7fe7d419e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:54:02.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.449+0000 7fe7d9d3e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d419ed20 con 0x7fe7d4106ee0 2024-02-18T11:54:02.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.449+0000 7fe7d9d3e640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d419ee90 con 0x7fe7d4101c80 2024-02-18T11:54:02.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.449+0000 7fe7d9d3e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d419f050 con 0x7fe7d41045b0 2024-02-18T11:54:02.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.449+0000 7fe7d3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7d4106ee0 0x7fe7d419e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:54:02.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.449+0000 7fe7d3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7d4106ee0 0x7fe7d419e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41926/0 (socket says 172.21.15.12:41926) 2024-02-18T11:54:02.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.449+0000 7fe7d3fff640 1 -- 172.21.15.12:0/3609994178 learned_addr learned my addr 172.21.15.12:0/3609994178 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:54:02.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.449+0000 7fe7d37fe640 1 --2- 172.21.15.12:0/3609994178 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe7d4101c80 0x7fe7d4197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:54:02.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.449+0000 7fe7d2ffd640 1 --2- 172.21.15.12:0/3609994178 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7d41045b0 0x7fe7d4197770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:54:02.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.449+0000 7fe7d3fff640 1 -- 172.21.15.12:0/3609994178 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7d41045b0 msgr2=0x7fe7d4197770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:54:02.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.449+0000 7fe7d3fff640 1 --2- 172.21.15.12:0/3609994178 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7d41045b0 0x7fe7d4197770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:02.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.449+0000 7fe7d3fff640 1 -- 172.21.15.12:0/3609994178 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe7d4101c80 msgr2=0x7fe7d4197230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:54:02.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.449+0000 7fe7d3fff640 1 --2- 172.21.15.12:0/3609994178 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe7d4101c80 0x7fe7d4197230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:02.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.449+0000 7fe7d3fff640 1 -- 172.21.15.12:0/3609994178 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7c4009560 con 0x7fe7d4106ee0 2024-02-18T11:54:02.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.449+0000 7fe7d3fff640 1 --2- 172.21.15.12:0/3609994178 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7d4106ee0 0x7fe7d419e7e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc009fd0 tx=0x7fe7cc00def0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:54:02.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.450+0000 7fe7d0ff9640 1 -- 172.21.15.12:0/3609994178 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7cc00bdd0 con 0x7fe7d4106ee0 2024-02-18T11:54:02.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.450+0000 7fe7d9d3e640 1 -- 172.21.15.12:0/3609994178 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7d419f330 con 0x7fe7d4106ee0 2024-02-18T11:54:02.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.450+0000 7fe7d0ff9640 1 -- 172.21.15.12:0/3609994178 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe7cc009940 con 0x7fe7d4106ee0 2024-02-18T11:54:02.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.450+0000 7fe7d0ff9640 1 -- 172.21.15.12:0/3609994178 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7cc0106c0 con 0x7fe7d4106ee0 2024-02-18T11:54:02.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.450+0000 7fe7d9d3e640 1 -- 172.21.15.12:0/3609994178 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7d419f7f0 con 0x7fe7d4106ee0 2024-02-18T11:54:02.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.451+0000 7fe7d9d3e640 1 -- 172.21.15.12:0/3609994178 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7d40fd160 con 0x7fe7d4106ee0 2024-02-18T11:54:02.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.452+0000 7fe7d0ff9640 1 -- 172.21.15.12:0/3609994178 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 20) v1 ==== 137245+0+0 (secure 0 0 0) 0x7fe7cc004190 con 0x7fe7d4106ee0 2024-02-18T11:54:02.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.452+0000 7fe7d0ff9640 1 --2- 172.21.15.12:0/3609994178 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fe7bc0a2750 0x7fe7bc0a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:54:02.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.452+0000 7fe7d37fe640 1 --2- 172.21.15.12:0/3609994178 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fe7bc0a2750 0x7fe7bc0a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:54:02.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.452+0000 7fe7d0ff9640 1 -- 172.21.15.12:0/3609994178 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe7cc0ce1b0 con 0x7fe7d4106ee0 2024-02-18T11:54:02.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.453+0000 7fe7d37fe640 1 --2- 172.21.15.12:0/3609994178 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fe7bc0a2750 0x7fe7bc0a4c10 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe7c4009230 tx=0x7fe7c4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:54:02.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.457+0000 7fe7d0ff9640 1 -- 172.21.15.12:0/3609994178 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe7cc096f50 con 0x7fe7d4106ee0 2024-02-18T11:54:02.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.640+0000 7fe7d9d3e640 1 -- 172.21.15.12:0/3609994178 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe7d402ce80 con 0x7fe7d4106ee0 2024-02-18T11:54:02.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.641+0000 7fe7d0ff9640 1 -- 172.21.15.12:0/3609994178 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+22662 (secure 0 0 0) 0x7fe7cc09a610 con 0x7fe7d4106ee0 2024-02-18T11:54:02.634 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:54:02.635 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":89,"fsid":"0c8bb0de-ce51-11ee-95ba-87774f69a715","created":"2024-02-18T11:30:49.110059+0000","modified":"2024-02-18T11:39:52.478480+0000","last_up_change":"2024-02-18T11:37:57.512337+0000","last_in_change":"2024-02-18T11:37:36.567969+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-18T11:34:15.281138+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-18T11:38:44.802134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-18T11:38:46.081188+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"35f15ea6-deb8-48a1-a7dd-ab649ec13533","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":536522426},{"type":"v1","addr":"172.21.15.12:6803","nonce":536522426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":536522426},{"type":"v1","addr":"172.21.15.12:6805","nonce":536522426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":536522426},{"type":"v1","addr":"172.21.15.12:6809","nonce":536522426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":536522426},{"type":"v1","addr":"172.21.15.12:6807","nonce":536522426}]},"public_addr":"172.21.15.12:6803/536522426","cluster_addr":"172.21.15.12:6805/536522426","heartbeat_back_addr":"172.21.15.12:6809/536522426","heartbeat_front_addr":"172.21.15.12:6807/536522426","state":["exists","up"]},{"osd":1,"uuid":"2af937ef-d827-44e6-b805-70cffbfe1126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":4249111488},{"type":"v1","addr":"172.21.15.158:6801","nonce":4249111488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":4249111488},{"type":"v1","addr":"172.21.15.158:6803","nonce":4249111488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":4249111488},{"type":"v1","addr":"172.21.15.158:6807","nonce":4249111488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":4249111488},{"type":"v1","addr":"172.21.15.158:6805","nonce":4249111488}]},"public_addr":"172.21.15.158:6801/4249111488","cluster_addr":"172.21.15.158:6803/4249111488","heartbeat_back_addr":"172.21.15.158:6807/4249111488","heartbeat_front_addr":"172.21.15.158:6805/4249111488","state":["exists","up"]},{"osd":2,"uuid":"17b804dd-31df-45ae-a637-4a5f65bbaf26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":627976150},{"type":"v1","addr":"172.21.15.188:6801","nonce":627976150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":627976150},{"type":"v1","addr":"172.21.15.188:6803","nonce":627976150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":627976150},{"type":"v1","addr":"172.21.15.188:6807","nonce":627976150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":627976150},{"type":"v1","addr":"172.21.15.188:6805","nonce":627976150}]},"public_addr":"172.21.15.188:6801/627976150","cluster_addr":"172.21.15.188:6803/627976150","heartbeat_back_addr":"172.21.15.188:6807/627976150","heartbeat_front_addr":"172.21.15.188:6805/627976150","state":["exists","up"]},{"osd":3,"uuid":"3f9cc423-cdfa-4acf-aaa2-b58b32ccb9cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":3758512503},{"type":"v1","addr":"172.21.15.12:6811","nonce":3758512503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":3758512503},{"type":"v1","addr":"172.21.15.12:6813","nonce":3758512503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3758512503},{"type":"v1","addr":"172.21.15.12:6817","nonce":3758512503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":3758512503},{"type":"v1","addr":"172.21.15.12:6815","nonce":3758512503}]},"public_addr":"172.21.15.12:6811/3758512503","cluster_addr":"172.21.15.12:6813/3758512503","heartbeat_back_addr":"172.21.15.12:6817/3758512503","heartbeat_front_addr":"172.21.15.12:6815/3758512503","state":["exists","up"]},{"osd":4,"uuid":"a92e20ec-af1c-4ef4-a0cb-caba5fcc71aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":3892820997},{"type":"v1","addr":"172.21.15.158:6809","nonce":3892820997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":3892820997},{"type":"v1","addr":"172.21.15.158:6811","nonce":3892820997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":3892820997},{"type":"v1","addr":"172.21.15.158:6815","nonce":3892820997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":3892820997},{"type":"v1","addr":"172.21.15.158:6813","nonce":3892820997}]},"public_addr":"172.21.15.158:6809/3892820997","cluster_addr":"172.21.15.158:6811/3892820997","heartbeat_back_addr":"172.21.15.158:6815/3892820997","heartbeat_front_addr":"172.21.15.158:6813/3892820997","state":["exists","up"]},{"osd":5,"uuid":"d9c2e0c8-85f5-4ae2-bf3e-125c1be3a2db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3939495896},{"type":"v1","addr":"172.21.15.188:6809","nonce":3939495896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3939495896},{"type":"v1","addr":"172.21.15.188:6811","nonce":3939495896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3939495896},{"type":"v1","addr":"172.21.15.188:6815","nonce":3939495896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3939495896},{"type":"v1","addr":"172.21.15.188:6813","nonce":3939495896}]},"public_addr":"172.21.15.188:6809/3939495896","cluster_addr":"172.21.15.188:6811/3939495896","heartbeat_back_addr":"172.21.15.188:6815/3939495896","heartbeat_front_addr":"172.21.15.188:6813/3939495896","state":["exists","up"]},{"osd":6,"uuid":"4a523512-de6b-49f0-be81-00f2f00c1c10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":1206337767},{"type":"v1","addr":"172.21.15.12:6819","nonce":1206337767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":1206337767},{"type":"v1","addr":"172.21.15.12:6821","nonce":1206337767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":1206337767},{"type":"v1","addr":"172.21.15.12:6825","nonce":1206337767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":1206337767},{"type":"v1","addr":"172.21.15.12:6823","nonce":1206337767}]},"public_addr":"172.21.15.12:6819/1206337767","cluster_addr":"172.21.15.12:6821/1206337767","heartbeat_back_addr":"172.21.15.12:6825/1206337767","heartbeat_front_addr":"172.21.15.12:6823/1206337767","state":["exists","up"]},{"osd":7,"uuid":"363271a6-df93-44c9-b0e5-3d29768aa96c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":3283400032},{"type":"v1","addr":"172.21.15.158:6817","nonce":3283400032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":3283400032},{"type":"v1","addr":"172.21.15.158:6819","nonce":3283400032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":3283400032},{"type":"v1","addr":"172.21.15.158:6823","nonce":3283400032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":3283400032},{"type":"v1","addr":"172.21.15.158:6821","nonce":3283400032}]},"public_addr":"172.21.15.158:6817/3283400032","cluster_addr":"172.21.15.158:6819/3283400032","heartbeat_back_addr":"172.21.15.158:6823/3283400032","heartbeat_front_addr":"172.21.15.158:6821/3283400032","state":["exists","up"]},{"osd":8,"uuid":"9a089528-3e8e-45d3-9279-ddb54088b96a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2110170682},{"type":"v1","addr":"172.21.15.188:6817","nonce":2110170682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2110170682},{"type":"v1","addr":"172.21.15.188:6819","nonce":2110170682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":2110170682},{"type":"v1","addr":"172.21.15.188:6823","nonce":2110170682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2110170682},{"type":"v1","addr":"172.21.15.188:6821","nonce":2110170682}]},"public_addr":"172.21.15.188:6817/2110170682","cluster_addr":"172.21.15.188:6819/2110170682","heartbeat_back_addr":"172.21.15.188:6823/2110170682","heartbeat_front_addr":"172.21.15.188:6821/2110170682","state":["exists","up"]},{"osd":9,"uuid":"b12e2727-d649-4464-90e1-fafbc1754832","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1825928986},{"type":"v1","addr":"172.21.15.12:6827","nonce":1825928986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1825928986},{"type":"v1","addr":"172.21.15.12:6829","nonce":1825928986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":1825928986},{"type":"v1","addr":"172.21.15.12:6833","nonce":1825928986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1825928986},{"type":"v1","addr":"172.21.15.12:6831","nonce":1825928986}]},"public_addr":"172.21.15.12:6827/1825928986","cluster_addr":"172.21.15.12:6829/1825928986","heartbeat_back_addr":"172.21.15.12:6833/1825928986","heartbeat_front_addr":"172.21.15.12:6831/1825928986","state":["exists","up"]},{"osd":10,"uuid":"518b4cbf-9d90-4e9a-8ac9-b5cb9b3dee4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":3239052167},{"type":"v1","addr":"172.21.15.158:6825","nonce":3239052167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":3239052167},{"type":"v1","addr":"172.21.15.158:6827","nonce":3239052167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":3239052167},{"type":"v1","addr":"172.21.15.158:6831","nonce":3239052167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":3239052167},{"type":"v1","addr":"172.21.15.158:6829","nonce":3239052167}]},"public_addr":"172.21.15.158:6825/3239052167","cluster_addr":"172.21.15.158:6827/3239052167","heartbeat_back_addr":"172.21.15.158:6831/3239052167","heartbeat_front_addr":"172.21.15.158:6829/3239052167","state":["exists","up"]},{"osd":11,"uuid":"98aa3a0d-4b2f-46c8-9341-375d13794c3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1135773350},{"type":"v1","addr":"172.21.15.188:6825","nonce":1135773350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1135773350},{"type":"v1","addr":"172.21.15.188:6827","nonce":1135773350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1135773350},{"type":"v1","addr":"172.21.15.188:6831","nonce":1135773350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1135773350},{"type":"v1","addr":"172.21.15.188:6829","nonce":1135773350}]},"public_addr":"172.21.15.188:6825/1135773350","cluster_addr":"172.21.15.188:6827/1135773350","heartbeat_back_addr":"172.21.15.188:6831/1135773350","heartbeat_front_addr":"172.21.15.188:6829/1135773350","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:33:24.280879+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:33:47.859461+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:34:11.725283+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:34:35.707629+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:35:00.024197+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:35:24.499803+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:35:48.953870+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:36:13.642456+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:36:38.936457+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:37:04.021466+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:37:29.073178+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:37:54.863919+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.37","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:6801/4135274411":"2024-02-19T11:31:50.936928+0000","172.21.15.12:6800/4135274411":"2024-02-19T11:31:50.936928+0000","172.21.15.12:0/118428584":"2024-02-19T11:31:50.936928+0000","172.21.15.12:0/4276810915":"2024-02-19T11:31:50.936928+0000","172.21.15.12:0/3777545262":"2024-02-19T11:31:24.000830+0000","172.21.15.12:0/114707743":"2024-02-19T11:31:24.000830+0000","172.21.15.12:6800/3925204723":"2024-02-19T11:31:24.000830+0000","172.21.15.12:6801/3925204723":"2024-02-19T11:31:24.000830+0000","172.21.15.12:0/74881618":"2024-02-19T11:31:24.000830+0000","172.21.15.12:0/1614805375":"2024-02-19T11:31:50.936928+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-18T11:54:02.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.644+0000 7fe7d9d3e640 1 -- 172.21.15.12:0/3609994178 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fe7bc0a2750 msgr2=0x7fe7bc0a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:54:02.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.644+0000 7fe7d9d3e640 1 --2- 172.21.15.12:0/3609994178 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fe7bc0a2750 0x7fe7bc0a4c10 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe7c4009230 tx=0x7fe7c4005fd0 comp rx=0 tx=0).stop 2024-02-18T11:54:02.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.644+0000 7fe7d9d3e640 1 -- 172.21.15.12:0/3609994178 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7d4106ee0 msgr2=0x7fe7d419e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:54:02.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.644+0000 7fe7d9d3e640 1 --2- 172.21.15.12:0/3609994178 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7d4106ee0 0x7fe7d419e7e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc009fd0 tx=0x7fe7cc00def0 comp rx=0 tx=0).stop 2024-02-18T11:54:02.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.644+0000 7fe7d9d3e640 1 -- 172.21.15.12:0/3609994178 shutdown_connections 2024-02-18T11:54:02.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.644+0000 7fe7d9d3e640 1 --2- 172.21.15.12:0/3609994178 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fe7bc0a2750 0x7fe7bc0a4c10 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:02.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.644+0000 7fe7d9d3e640 1 --2- 172.21.15.12:0/3609994178 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7d41045b0 0x7fe7d4197770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:02.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.644+0000 7fe7d9d3e640 1 --2- 172.21.15.12:0/3609994178 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7d4106ee0 0x7fe7d419e7e0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:02.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.644+0000 7fe7d9d3e640 1 --2- 172.21.15.12:0/3609994178 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe7d4101c80 0x7fe7d4197230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:02.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.644+0000 7fe7d9d3e640 1 -- 172.21.15.12:0/3609994178 >> 172.21.15.12:0/3609994178 conn(0x7fe7d40fb670 msgr2=0x7fe7d41025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:54:02.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.644+0000 7fe7d9d3e640 1 -- 172.21.15.12:0/3609994178 shutdown_connections 2024-02-18T11:54:02.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.644+0000 7fe7d9d3e640 1 -- 172.21.15.12:0/3609994178 wait complete. 2024-02-18T11:54:02.648 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-18T11:34:15.281138+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-18T11:38:44.802134+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-18T11:38:46.081188+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-18T11:54:02.649 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-18T11:54:02.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.764+0000 7fb9accdf640 1 -- 172.21.15.12:0/2114725570 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9a8101c60 msgr2=0x7fb9a8104050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:54:02.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.764+0000 7fb9accdf640 1 --2- 172.21.15.12:0/2114725570 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9a8101c60 0x7fb9a8104050 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb998009900 tx=0x7fb99802bc30 comp rx=0 tx=0).stop 2024-02-18T11:54:02.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.764+0000 7fb9accdf640 1 -- 172.21.15.12:0/2114725570 shutdown_connections 2024-02-18T11:54:02.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.764+0000 7fb9accdf640 1 --2- 172.21.15.12:0/2114725570 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9a8106ec0 0x7fb9a8109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:02.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.764+0000 7fb9accdf640 1 --2- 172.21.15.12:0/2114725570 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb9a8104590 0x7fb9a8106980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:02.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.764+0000 7fb9accdf640 1 --2- 172.21.15.12:0/2114725570 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9a8101c60 0x7fb9a8104050 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:02.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.764+0000 7fb9accdf640 1 -- 172.21.15.12:0/2114725570 >> 172.21.15.12:0/2114725570 conn(0x7fb9a80fb670 msgr2=0x7fb9a80fda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:54:02.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.764+0000 7fb9accdf640 1 -- 172.21.15.12:0/2114725570 shutdown_connections 2024-02-18T11:54:02.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.764+0000 7fb9accdf640 1 -- 172.21.15.12:0/2114725570 wait complete. 2024-02-18T11:54:02.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.765+0000 7fb9accdf640 1 Processor -- start 2024-02-18T11:54:02.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.765+0000 7fb9accdf640 1 -- start start 2024-02-18T11:54:02.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.766+0000 7fb9accdf640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9a8101c60 0x7fb9a81972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:54:02.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.766+0000 7fb9accdf640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb9a8104590 0x7fb9a8197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:54:02.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.766+0000 7fb9accdf640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9a8106ec0 0x7fb9a819e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:54:02.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.766+0000 7fb9accdf640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a819ed70 con 0x7fb9a8101c60 2024-02-18T11:54:02.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.766+0000 7fb9accdf640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a819eee0 con 0x7fb9a8104590 2024-02-18T11:54:02.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.766+0000 7fb9accdf640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a819f0a0 con 0x7fb9a8106ec0 2024-02-18T11:54:02.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.766+0000 7fb9a659c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9a8101c60 0x7fb9a81972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:54:02.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.766+0000 7fb9a659c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9a8101c60 0x7fb9a81972d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41948/0 (socket says 172.21.15.12:41948) 2024-02-18T11:54:02.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.766+0000 7fb9a659c640 1 -- 172.21.15.12:0/729812578 learned_addr learned my addr 172.21.15.12:0/729812578 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:54:02.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.766+0000 7fb9a5d9b640 1 --2- 172.21.15.12:0/729812578 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb9a8104590 0x7fb9a8197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:54:02.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.766+0000 7fb9a6d9d640 1 --2- 172.21.15.12:0/729812578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9a8106ec0 0x7fb9a819e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:54:02.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.766+0000 7fb9a659c640 1 -- 172.21.15.12:0/729812578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9a8106ec0 msgr2=0x7fb9a819e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:54:02.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.766+0000 7fb9a659c640 1 --2- 172.21.15.12:0/729812578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9a8106ec0 0x7fb9a819e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:02.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.766+0000 7fb9a659c640 1 -- 172.21.15.12:0/729812578 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb9a8104590 msgr2=0x7fb9a8197810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:54:02.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.766+0000 7fb9a659c640 1 --2- 172.21.15.12:0/729812578 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb9a8104590 0x7fb9a8197810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:02.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.766+0000 7fb9a659c640 1 -- 172.21.15.12:0/729812578 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb998009560 con 0x7fb9a8101c60 2024-02-18T11:54:02.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.766+0000 7fb9a659c640 1 --2- 172.21.15.12:0/729812578 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9a8101c60 0x7fb9a81972d0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fb998006010 tx=0x7fb9980040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:54:02.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.766+0000 7fb9977fe640 1 -- 172.21.15.12:0/729812578 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9980044c0 con 0x7fb9a8101c60 2024-02-18T11:54:02.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.766+0000 7fb9977fe640 1 -- 172.21.15.12:0/729812578 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb9980052d0 con 0x7fb9a8101c60 2024-02-18T11:54:02.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.766+0000 7fb9accdf640 1 -- 172.21.15.12:0/729812578 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9a819f320 con 0x7fb9a8101c60 2024-02-18T11:54:02.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.766+0000 7fb9977fe640 1 -- 172.21.15.12:0/729812578 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb998003e10 con 0x7fb9a8101c60 2024-02-18T11:54:02.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.766+0000 7fb9accdf640 1 -- 172.21.15.12:0/729812578 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9a819f830 con 0x7fb9a8101c60 2024-02-18T11:54:02.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.768+0000 7fb9accdf640 1 -- 172.21.15.12:0/729812578 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9a80fd160 con 0x7fb9a8101c60 2024-02-18T11:54:02.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.768+0000 7fb9977fe640 1 -- 172.21.15.12:0/729812578 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 20) v1 ==== 137245+0+0 (secure 0 0 0) 0x7fb998005440 con 0x7fb9a8101c60 2024-02-18T11:54:02.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.769+0000 7fb9977fe640 1 --2- 172.21.15.12:0/729812578 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fb9900a2750 0x7fb9900a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:54:02.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.769+0000 7fb9a5d9b640 1 --2- 172.21.15.12:0/729812578 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fb9900a2750 0x7fb9900a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:54:02.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.769+0000 7fb9977fe640 1 -- 172.21.15.12:0/729812578 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb9980f08f0 con 0x7fb9a8101c60 2024-02-18T11:54:02.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.770+0000 7fb9a5d9b640 1 --2- 172.21.15.12:0/729812578 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fb9900a2750 0x7fb9900a4c10 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb9a8198790 tx=0x7fb99c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:54:02.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.774+0000 7fb9977fe640 1 -- 172.21.15.12:0/729812578 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb9980b9690 con 0x7fb9a8101c60 2024-02-18T11:54:02.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.957+0000 7fb9accdf640 1 -- 172.21.15.12:0/729812578 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb9a802ce10 con 0x7fb9a8101c60 2024-02-18T11:54:02.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.957+0000 7fb9977fe640 1 -- 172.21.15.12:0/729812578 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb9980bcd50 con 0x7fb9a8101c60 2024-02-18T11:54:02.950 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-02-18T11:54:02.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.959+0000 7fb9accdf640 1 -- 172.21.15.12:0/729812578 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fb9900a2750 msgr2=0x7fb9900a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:54:02.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.959+0000 7fb9accdf640 1 --2- 172.21.15.12:0/729812578 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fb9900a2750 0x7fb9900a4c10 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb9a8198790 tx=0x7fb99c005f90 comp rx=0 tx=0).stop 2024-02-18T11:54:02.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.959+0000 7fb9accdf640 1 -- 172.21.15.12:0/729812578 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9a8101c60 msgr2=0x7fb9a81972d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:54:02.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.959+0000 7fb9accdf640 1 --2- 172.21.15.12:0/729812578 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9a8101c60 0x7fb9a81972d0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fb998006010 tx=0x7fb9980040c0 comp rx=0 tx=0).stop 2024-02-18T11:54:02.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.959+0000 7fb9accdf640 1 -- 172.21.15.12:0/729812578 shutdown_connections 2024-02-18T11:54:02.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.959+0000 7fb9accdf640 1 --2- 172.21.15.12:0/729812578 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fb9900a2750 0x7fb9900a4c10 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:02.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.959+0000 7fb9accdf640 1 --2- 172.21.15.12:0/729812578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9a8106ec0 0x7fb9a819e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:02.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.959+0000 7fb9accdf640 1 --2- 172.21.15.12:0/729812578 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb9a8104590 0x7fb9a8197810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:02.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.959+0000 7fb9accdf640 1 --2- 172.21.15.12:0/729812578 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9a8101c60 0x7fb9a81972d0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:02.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.959+0000 7fb9accdf640 1 -- 172.21.15.12:0/729812578 >> 172.21.15.12:0/729812578 conn(0x7fb9a80fb670 msgr2=0x7fb9a8102a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:54:02.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.959+0000 7fb9accdf640 1 -- 172.21.15.12:0/729812578 shutdown_connections 2024-02-18T11:54:02.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:02.959+0000 7fb9accdf640 1 -- 172.21.15.12:0/729812578 wait complete. 2024-02-18T11:54:02.963 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-18T11:54:03.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.079+0000 7fdd7e981640 1 -- 172.21.15.12:0/3422933592 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd78101c80 msgr2=0x7fdd78108510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:54:03.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.079+0000 7fdd7e981640 1 --2- 172.21.15.12:0/3422933592 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd78101c80 0x7fdd78108510 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fdd70009ca0 tx=0x7fdd7002c0f0 comp rx=0 tx=0).stop 2024-02-18T11:54:03.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.079+0000 7fdd7e981640 1 -- 172.21.15.12:0/3422933592 shutdown_connections 2024-02-18T11:54:03.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.079+0000 7fdd7e981640 1 --2- 172.21.15.12:0/3422933592 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd78101c80 0x7fdd78108510 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:03.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.079+0000 7fdd7e981640 1 --2- 172.21.15.12:0/3422933592 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdd781012c0 0x7fdd78101740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:03.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.079+0000 7fdd7e981640 1 --2- 172.21.15.12:0/3422933592 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdd781000d0 0x7fdd781004f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:03.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.079+0000 7fdd7e981640 1 -- 172.21.15.12:0/3422933592 >> 172.21.15.12:0/3422933592 conn(0x7fdd780fb660 msgr2=0x7fdd780fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:54:03.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.080+0000 7fdd7e981640 1 -- 172.21.15.12:0/3422933592 shutdown_connections 2024-02-18T11:54:03.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.080+0000 7fdd7e981640 1 -- 172.21.15.12:0/3422933592 wait complete. 2024-02-18T11:54:03.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.080+0000 7fdd7e981640 1 Processor -- start 2024-02-18T11:54:03.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.080+0000 7fdd7e981640 1 -- start start 2024-02-18T11:54:03.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.080+0000 7fdd7e981640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdd781000d0 0x7fdd78104060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:54:03.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.080+0000 7fdd7e981640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd781012c0 0x7fdd781026a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:54:03.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.080+0000 7fdd7e981640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdd78101c80 0x7fdd78102be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:54:03.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.080+0000 7fdd7e981640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd78103340 con 0x7fdd781012c0 2024-02-18T11:54:03.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.080+0000 7fdd7e981640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd781034b0 con 0x7fdd781000d0 2024-02-18T11:54:03.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.080+0000 7fdd7e981640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd781a1250 con 0x7fdd78101c80 2024-02-18T11:54:03.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.081+0000 7fdd777fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd781012c0 0x7fdd781026a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:54:03.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.081+0000 7fdd777fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd781012c0 0x7fdd781026a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41968/0 (socket says 172.21.15.12:41968) 2024-02-18T11:54:03.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.081+0000 7fdd777fe640 1 -- 172.21.15.12:0/3415615332 learned_addr learned my addr 172.21.15.12:0/3415615332 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:54:03.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.081+0000 7fdd7cf1e640 1 --2- 172.21.15.12:0/3415615332 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdd78101c80 0x7fdd78102be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:54:03.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.081+0000 7fdd77fff640 1 --2- 172.21.15.12:0/3415615332 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdd781000d0 0x7fdd78104060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:54:03.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.081+0000 7fdd777fe640 1 -- 172.21.15.12:0/3415615332 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdd78101c80 msgr2=0x7fdd78102be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:54:03.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.081+0000 7fdd777fe640 1 --2- 172.21.15.12:0/3415615332 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdd78101c80 0x7fdd78102be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:03.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.081+0000 7fdd777fe640 1 -- 172.21.15.12:0/3415615332 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdd781000d0 msgr2=0x7fdd78104060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:54:03.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.081+0000 7fdd777fe640 1 --2- 172.21.15.12:0/3415615332 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdd781000d0 0x7fdd78104060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:03.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.081+0000 7fdd777fe640 1 -- 172.21.15.12:0/3415615332 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd70009900 con 0x7fdd781012c0 2024-02-18T11:54:03.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.081+0000 7fdd7cf1e640 1 --2- 172.21.15.12:0/3415615332 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdd78101c80 0x7fdd78102be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:54:03.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.081+0000 7fdd777fe640 1 --2- 172.21.15.12:0/3415615332 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd781012c0 0x7fdd781026a0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fdd6800d980 tx=0x7fdd6800de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:54:03.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.081+0000 7fdd757fa640 1 -- 172.21.15.12:0/3415615332 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd68009a30 con 0x7fdd781012c0 2024-02-18T11:54:03.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.081+0000 7fdd757fa640 1 -- 172.21.15.12:0/3415615332 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdd68009b90 con 0x7fdd781012c0 2024-02-18T11:54:03.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.081+0000 7fdd7e981640 1 -- 172.21.15.12:0/3415615332 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd781a1500 con 0x7fdd781012c0 2024-02-18T11:54:03.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.081+0000 7fdd757fa640 1 -- 172.21.15.12:0/3415615332 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd68004a00 con 0x7fdd781012c0 2024-02-18T11:54:03.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.081+0000 7fdd7e981640 1 -- 172.21.15.12:0/3415615332 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd781a1a10 con 0x7fdd781012c0 2024-02-18T11:54:03.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.082+0000 7fdd7e981640 1 -- 172.21.15.12:0/3415615332 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd7810bae0 con 0x7fdd781012c0 2024-02-18T11:54:03.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.084+0000 7fdd757fa640 1 -- 172.21.15.12:0/3415615332 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 20) v1 ==== 137245+0+0 (secure 0 0 0) 0x7fdd68009d00 con 0x7fdd781012c0 2024-02-18T11:54:03.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.084+0000 7fdd757fa640 1 --2- 172.21.15.12:0/3415615332 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fdd640a2750 0x7fdd640a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:54:03.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.084+0000 7fdd77fff640 1 --2- 172.21.15.12:0/3415615332 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fdd640a2750 0x7fdd640a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:54:03.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.085+0000 7fdd757fa640 1 -- 172.21.15.12:0/3415615332 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fdd680ce600 con 0x7fdd781012c0 2024-02-18T11:54:03.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.085+0000 7fdd77fff640 1 --2- 172.21.15.12:0/3415615332 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fdd640a2750 0x7fdd640a4c10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fdd60005fd0 tx=0x7fdd60005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:54:03.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.088+0000 7fdd757fa640 1 -- 172.21.15.12:0/3415615332 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdd680973a0 con 0x7fdd781012c0 2024-02-18T11:54:03.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.273+0000 7fdd7e981640 1 -- 172.21.15.12:0/3415615332 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdd781a1f90 con 0x7fdd781012c0 2024-02-18T11:54:03.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.273+0000 7fdd757fa640 1 -- 172.21.15.12:0/3415615332 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdd680170a0 con 0x7fdd781012c0 2024-02-18T11:54:03.269 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-02-18T11:54:03.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.275+0000 7fdd7e981640 1 -- 172.21.15.12:0/3415615332 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fdd640a2750 msgr2=0x7fdd640a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:54:03.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.275+0000 7fdd7e981640 1 --2- 172.21.15.12:0/3415615332 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fdd640a2750 0x7fdd640a4c10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fdd60005fd0 tx=0x7fdd60005850 comp rx=0 tx=0).stop 2024-02-18T11:54:03.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.275+0000 7fdd7e981640 1 -- 172.21.15.12:0/3415615332 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd781012c0 msgr2=0x7fdd781026a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:54:03.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.275+0000 7fdd7e981640 1 --2- 172.21.15.12:0/3415615332 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd781012c0 0x7fdd781026a0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fdd6800d980 tx=0x7fdd6800de40 comp rx=0 tx=0).stop 2024-02-18T11:54:03.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.275+0000 7fdd7e981640 1 -- 172.21.15.12:0/3415615332 shutdown_connections 2024-02-18T11:54:03.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.275+0000 7fdd7e981640 1 --2- 172.21.15.12:0/3415615332 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fdd640a2750 0x7fdd640a4c10 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:03.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.275+0000 7fdd7e981640 1 --2- 172.21.15.12:0/3415615332 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdd78101c80 0x7fdd78102be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:03.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.275+0000 7fdd7e981640 1 --2- 172.21.15.12:0/3415615332 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd781012c0 0x7fdd781026a0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:03.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.275+0000 7fdd7e981640 1 --2- 172.21.15.12:0/3415615332 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdd781000d0 0x7fdd78104060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:03.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.275+0000 7fdd7e981640 1 -- 172.21.15.12:0/3415615332 >> 172.21.15.12:0/3415615332 conn(0x7fdd780fb660 msgr2=0x7fdd780fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:54:03.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.275+0000 7fdd7e981640 1 -- 172.21.15.12:0/3415615332 shutdown_connections 2024-02-18T11:54:03.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.275+0000 7fdd7e981640 1 -- 172.21.15.12:0/3415615332 wait complete. 2024-02-18T11:54:03.278 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:54:03 smithi012 ceph-mon[32460]: pgmap v744: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 27 MiB/s wr, 375 op/s 2024-02-18T11:54:03.279 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:54:03 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/3609994178' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-18T11:54:03.279 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:54:03 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/729812578' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-18T11:54:03.280 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-18T11:54:03.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.395+0000 7f1725481640 1 -- 172.21.15.12:0/2796189382 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1720102da0 msgr2=0x7f1720109630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:54:03.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.395+0000 7f1725481640 1 --2- 172.21.15.12:0/2796189382 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1720102da0 0x7f1720109630 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f1718009ca0 tx=0x7f171802c0f0 comp rx=0 tx=0).stop 2024-02-18T11:54:03.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.395+0000 7f1725481640 1 -- 172.21.15.12:0/2796189382 shutdown_connections 2024-02-18T11:54:03.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.395+0000 7f1725481640 1 --2- 172.21.15.12:0/2796189382 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1720102da0 0x7f1720109630 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:03.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.395+0000 7f1725481640 1 --2- 172.21.15.12:0/2796189382 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f17201023e0 0x7f1720102860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:03.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.395+0000 7f1725481640 1 --2- 172.21.15.12:0/2796189382 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f17201011f0 0x7f1720101610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:03.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.395+0000 7f1725481640 1 -- 172.21.15.12:0/2796189382 >> 172.21.15.12:0/2796189382 conn(0x7f17200fc760 msgr2=0x7f17200feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:54:03.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.396+0000 7f1725481640 1 -- 172.21.15.12:0/2796189382 shutdown_connections 2024-02-18T11:54:03.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.396+0000 7f1725481640 1 -- 172.21.15.12:0/2796189382 wait complete. 2024-02-18T11:54:03.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.396+0000 7f1725481640 1 Processor -- start 2024-02-18T11:54:03.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.396+0000 7f1725481640 1 -- start start 2024-02-18T11:54:03.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.396+0000 7f1725481640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f17201011f0 0x7f1720199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:54:03.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.396+0000 7f1725481640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f17201023e0 0x7f17201998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:54:03.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.396+0000 7f1725481640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1720102da0 0x7f17201a0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:54:03.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.396+0000 7f1725481640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17201a0e70 con 0x7f17201023e0 2024-02-18T11:54:03.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.396+0000 7f1725481640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17201a0fe0 con 0x7f1720102da0 2024-02-18T11:54:03.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.396+0000 7f1725481640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17201a11a0 con 0x7f17201011f0 2024-02-18T11:54:03.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.397+0000 7f171e7fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f17201023e0 0x7f17201998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:54:03.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.397+0000 7f171e7fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f17201023e0 0x7f17201998c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41982/0 (socket says 172.21.15.12:41982) 2024-02-18T11:54:03.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.397+0000 7f171e7fc640 1 -- 172.21.15.12:0/2991939083 learned_addr learned my addr 172.21.15.12:0/2991939083 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:54:03.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.397+0000 7f171f7fe640 1 --2- 172.21.15.12:0/2991939083 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1720102da0 0x7f17201a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:54:03.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.397+0000 7f171effd640 1 --2- 172.21.15.12:0/2991939083 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f17201011f0 0x7f1720199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:54:03.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.397+0000 7f171e7fc640 1 -- 172.21.15.12:0/2991939083 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f17201011f0 msgr2=0x7f1720199380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:54:03.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.397+0000 7f171e7fc640 1 --2- 172.21.15.12:0/2991939083 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f17201011f0 0x7f1720199380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:03.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.397+0000 7f171e7fc640 1 -- 172.21.15.12:0/2991939083 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1720102da0 msgr2=0x7f17201a0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:54:03.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.397+0000 7f171e7fc640 1 --2- 172.21.15.12:0/2991939083 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1720102da0 0x7f17201a0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:03.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.397+0000 7f171e7fc640 1 -- 172.21.15.12:0/2991939083 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1718009900 con 0x7f17201023e0 2024-02-18T11:54:03.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.397+0000 7f171effd640 1 --2- 172.21.15.12:0/2991939083 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f17201011f0 0x7f1720199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:54:03.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.397+0000 7f171e7fc640 1 --2- 172.21.15.12:0/2991939083 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f17201023e0 0x7f17201998c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f171000c920 tx=0x7f171000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:54:03.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.397+0000 7f1707fff640 1 -- 172.21.15.12:0/2991939083 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1710007c70 con 0x7f17201023e0 2024-02-18T11:54:03.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.397+0000 7f1725481640 1 -- 172.21.15.12:0/2991939083 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17201a1480 con 0x7f17201023e0 2024-02-18T11:54:03.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.397+0000 7f1725481640 1 -- 172.21.15.12:0/2991939083 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17201a19e0 con 0x7f17201023e0 2024-02-18T11:54:03.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.397+0000 7f1707fff640 1 -- 172.21.15.12:0/2991939083 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1710007dd0 con 0x7f17201023e0 2024-02-18T11:54:03.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.397+0000 7f1707fff640 1 -- 172.21.15.12:0/2991939083 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1710018860 con 0x7f17201023e0 2024-02-18T11:54:03.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.398+0000 7f1725481640 1 -- 172.21.15.12:0/2991939083 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f170c0052c0 con 0x7f17201023e0 2024-02-18T11:54:03.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.400+0000 7f1707fff640 1 -- 172.21.15.12:0/2991939083 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 20) v1 ==== 137245+0+0 (secure 0 0 0) 0x7f17100189c0 con 0x7f17201023e0 2024-02-18T11:54:03.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.401+0000 7f1707fff640 1 --2- 172.21.15.12:0/2991939083 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f17080a2750 0x7f17080a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:54:03.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.401+0000 7f171effd640 1 --2- 172.21.15.12:0/2991939083 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f17080a2750 0x7f17080a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:54:03.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.401+0000 7f1707fff640 1 -- 172.21.15.12:0/2991939083 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f17100ce8c0 con 0x7f17201023e0 2024-02-18T11:54:03.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.402+0000 7f171effd640 1 --2- 172.21.15.12:0/2991939083 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f17080a2750 0x7f17080a4c10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1720102240 tx=0x7f1714005d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:54:03.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.404+0000 7f1707fff640 1 -- 172.21.15.12:0/2991939083 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f17100976f0 con 0x7f17201023e0 2024-02-18T11:54:03.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:54:03 smithi158 ceph-mon[36626]: pgmap v744: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 27 MiB/s wr, 375 op/s 2024-02-18T11:54:03.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:54:03 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/3609994178' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-18T11:54:03.493 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:54:03 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/729812578' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-18T11:54:03.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.586+0000 7f1725481640 1 -- 172.21.15.12:0/2991939083 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f170c005af0 con 0x7f17201023e0 2024-02-18T11:54:03.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.586+0000 7f1707fff640 1 -- 172.21.15.12:0/2991939083 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+11 (secure 0 0 0) 0x7f171009adb0 con 0x7f17201023e0 2024-02-18T11:54:03.579 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2024-02-18T11:54:03.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.588+0000 7f1725481640 1 -- 172.21.15.12:0/2991939083 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f17080a2750 msgr2=0x7f17080a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:54:03.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.588+0000 7f1725481640 1 --2- 172.21.15.12:0/2991939083 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f17080a2750 0x7f17080a4c10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1720102240 tx=0x7f1714005d30 comp rx=0 tx=0).stop 2024-02-18T11:54:03.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.588+0000 7f1725481640 1 -- 172.21.15.12:0/2991939083 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f17201023e0 msgr2=0x7f17201998c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:54:03.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.588+0000 7f1725481640 1 --2- 172.21.15.12:0/2991939083 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f17201023e0 0x7f17201998c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f171000c920 tx=0x7f171000cde0 comp rx=0 tx=0).stop 2024-02-18T11:54:03.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.588+0000 7f1725481640 1 -- 172.21.15.12:0/2991939083 shutdown_connections 2024-02-18T11:54:03.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.588+0000 7f1725481640 1 --2- 172.21.15.12:0/2991939083 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f17080a2750 0x7f17080a4c10 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:03.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.588+0000 7f1725481640 1 --2- 172.21.15.12:0/2991939083 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1720102da0 0x7f17201a0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:03.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.588+0000 7f1725481640 1 --2- 172.21.15.12:0/2991939083 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f17201023e0 0x7f17201998c0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:03.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.588+0000 7f1725481640 1 --2- 172.21.15.12:0/2991939083 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f17201011f0 0x7f1720199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:03.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.588+0000 7f1725481640 1 -- 172.21.15.12:0/2991939083 >> 172.21.15.12:0/2991939083 conn(0x7f17200fc760 msgr2=0x7f17200733f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:54:03.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.588+0000 7f1725481640 1 -- 172.21.15.12:0/2991939083 shutdown_connections 2024-02-18T11:54:03.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.588+0000 7f1725481640 1 -- 172.21.15.12:0/2991939083 wait complete. 2024-02-18T11:54:03.593 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-18T11:54:03.602 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:54:03 smithi188 ceph-mon[36925]: pgmap v744: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 27 MiB/s wr, 375 op/s 2024-02-18T11:54:03.602 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:54:03 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/3609994178' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-18T11:54:03.602 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:54:03 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/729812578' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-18T11:54:03.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.705+0000 7ffb4e888640 1 -- 172.21.15.12:0/3625025213 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffb481045f0 msgr2=0x7ffb481069e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:54:03.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.705+0000 7ffb4e888640 1 --2- 172.21.15.12:0/3625025213 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffb481045f0 0x7ffb481069e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7ffb38009900 tx=0x7ffb3802bc30 comp rx=0 tx=0).stop 2024-02-18T11:54:03.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.705+0000 7ffb4e888640 1 -- 172.21.15.12:0/3625025213 shutdown_connections 2024-02-18T11:54:03.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.705+0000 7ffb4e888640 1 --2- 172.21.15.12:0/3625025213 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb48106f20 0x7ffb48109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:03.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.705+0000 7ffb4e888640 1 --2- 172.21.15.12:0/3625025213 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffb481045f0 0x7ffb481069e0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:03.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.705+0000 7ffb4e888640 1 --2- 172.21.15.12:0/3625025213 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ffb48101cc0 0x7ffb481040b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:03.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.705+0000 7ffb4e888640 1 -- 172.21.15.12:0/3625025213 >> 172.21.15.12:0/3625025213 conn(0x7ffb480fb650 msgr2=0x7ffb480fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:54:03.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.705+0000 7ffb4e888640 1 -- 172.21.15.12:0/3625025213 shutdown_connections 2024-02-18T11:54:03.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.705+0000 7ffb4e888640 1 -- 172.21.15.12:0/3625025213 wait complete. 2024-02-18T11:54:03.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.706+0000 7ffb4e888640 1 Processor -- start 2024-02-18T11:54:03.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.706+0000 7ffb4e888640 1 -- start start 2024-02-18T11:54:03.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.706+0000 7ffb4e888640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffb48101cc0 0x7ffb48199500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:54:03.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.706+0000 7ffb4e888640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb481045f0 0x7ffb48199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:54:03.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.706+0000 7ffb4e888640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ffb48106f20 0x7ffb481a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:54:03.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.706+0000 7ffb4e888640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb481a0ff0 con 0x7ffb48101cc0 2024-02-18T11:54:03.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.706+0000 7ffb4e888640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb481a1160 con 0x7ffb48106f20 2024-02-18T11:54:03.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.706+0000 7ffb4e888640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb481a1320 con 0x7ffb481045f0 2024-02-18T11:54:03.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.706+0000 7ffb47fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffb48101cc0 0x7ffb48199500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:54:03.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.707+0000 7ffb47fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffb48101cc0 0x7ffb48199500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41996/0 (socket says 172.21.15.12:41996) 2024-02-18T11:54:03.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.707+0000 7ffb47fff640 1 -- 172.21.15.12:0/3746815011 learned_addr learned my addr 172.21.15.12:0/3746815011 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:54:03.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.707+0000 7ffb477fe640 1 --2- 172.21.15.12:0/3746815011 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb481045f0 0x7ffb48199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:54:03.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.707+0000 7ffb4ce25640 1 --2- 172.21.15.12:0/3746815011 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ffb48106f20 0x7ffb481a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:54:03.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.707+0000 7ffb47fff640 1 -- 172.21.15.12:0/3746815011 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb481045f0 msgr2=0x7ffb48199a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:54:03.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.707+0000 7ffb47fff640 1 --2- 172.21.15.12:0/3746815011 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb481045f0 0x7ffb48199a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:03.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.707+0000 7ffb47fff640 1 -- 172.21.15.12:0/3746815011 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ffb48106f20 msgr2=0x7ffb481a0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:54:03.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.707+0000 7ffb47fff640 1 --2- 172.21.15.12:0/3746815011 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ffb48106f20 0x7ffb481a0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:03.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.707+0000 7ffb47fff640 1 -- 172.21.15.12:0/3746815011 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb38009560 con 0x7ffb48101cc0 2024-02-18T11:54:03.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.707+0000 7ffb4ce25640 1 --2- 172.21.15.12:0/3746815011 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ffb48106f20 0x7ffb481a0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:54:03.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.707+0000 7ffb47fff640 1 --2- 172.21.15.12:0/3746815011 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffb48101cc0 0x7ffb48199500 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ffb3000d950 tx=0x7ffb3000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:54:03.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.707+0000 7ffb457fa640 1 -- 172.21.15.12:0/3746815011 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb30009a30 con 0x7ffb48101cc0 2024-02-18T11:54:03.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.707+0000 7ffb4e888640 1 -- 172.21.15.12:0/3746815011 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb481a1600 con 0x7ffb48101cc0 2024-02-18T11:54:03.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.707+0000 7ffb457fa640 1 -- 172.21.15.12:0/3746815011 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffb30009b90 con 0x7ffb48101cc0 2024-02-18T11:54:03.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.707+0000 7ffb457fa640 1 -- 172.21.15.12:0/3746815011 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb30004a00 con 0x7ffb48101cc0 2024-02-18T11:54:03.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.707+0000 7ffb4e888640 1 -- 172.21.15.12:0/3746815011 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb481a1ae0 con 0x7ffb48101cc0 2024-02-18T11:54:03.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.708+0000 7ffb4e888640 1 -- 172.21.15.12:0/3746815011 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb48067fb0 con 0x7ffb48101cc0 2024-02-18T11:54:03.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.709+0000 7ffb457fa640 1 -- 172.21.15.12:0/3746815011 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 20) v1 ==== 137245+0+0 (secure 0 0 0) 0x7ffb30009d00 con 0x7ffb48101cc0 2024-02-18T11:54:03.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.710+0000 7ffb457fa640 1 --2- 172.21.15.12:0/3746815011 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7ffb340a2750 0x7ffb340a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:54:03.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.710+0000 7ffb477fe640 1 --2- 172.21.15.12:0/3746815011 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7ffb340a2750 0x7ffb340a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:54:03.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.710+0000 7ffb457fa640 1 -- 172.21.15.12:0/3746815011 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ffb300ce620 con 0x7ffb48101cc0 2024-02-18T11:54:03.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.711+0000 7ffb477fe640 1 --2- 172.21.15.12:0/3746815011 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7ffb340a2750 0x7ffb340a4c10 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ffb38005ab0 tx=0x7ffb38005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:54:03.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.714+0000 7ffb457fa640 1 -- 172.21.15.12:0/3746815011 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ffb300973c0 con 0x7ffb48101cc0 2024-02-18T11:54:03.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.908+0000 7ffb4e888640 1 -- 172.21.15.12:0/3746815011 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ffb48108a10 con 0x7ffb48101cc0 2024-02-18T11:54:03.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.909+0000 7ffb457fa640 1 -- 172.21.15.12:0/3746815011 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v89) v1 ==== 81+0+588 (secure 0 0 0) 0x7ffb30017020 con 0x7ffb48101cc0 2024-02-18T11:54:03.901 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6801/4135274411 2024-02-19T11:31:50.936928+0000 2024-02-18T11:54:03.902 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6800/4135274411 2024-02-19T11:31:50.936928+0000 2024-02-18T11:54:03.902 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/118428584 2024-02-19T11:31:50.936928+0000 2024-02-18T11:54:03.902 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/4276810915 2024-02-19T11:31:50.936928+0000 2024-02-18T11:54:03.902 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/3777545262 2024-02-19T11:31:24.000830+0000 2024-02-18T11:54:03.902 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/114707743 2024-02-19T11:31:24.000830+0000 2024-02-18T11:54:03.902 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6800/3925204723 2024-02-19T11:31:24.000830+0000 2024-02-18T11:54:03.902 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6801/3925204723 2024-02-19T11:31:24.000830+0000 2024-02-18T11:54:03.902 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/74881618 2024-02-19T11:31:24.000830+0000 2024-02-18T11:54:03.902 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/1614805375 2024-02-19T11:31:50.936928+0000 2024-02-18T11:54:03.902 INFO:teuthology.orchestra.run.smithi012.stderr:listed 10 entries 2024-02-18T11:54:03.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.910+0000 7ffb4e888640 1 -- 172.21.15.12:0/3746815011 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7ffb340a2750 msgr2=0x7ffb340a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:54:03.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.911+0000 7ffb4e888640 1 --2- 172.21.15.12:0/3746815011 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7ffb340a2750 0x7ffb340a4c10 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ffb38005ab0 tx=0x7ffb38005fd0 comp rx=0 tx=0).stop 2024-02-18T11:54:03.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.911+0000 7ffb4e888640 1 -- 172.21.15.12:0/3746815011 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffb48101cc0 msgr2=0x7ffb48199500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:54:03.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.911+0000 7ffb4e888640 1 --2- 172.21.15.12:0/3746815011 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffb48101cc0 0x7ffb48199500 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ffb3000d950 tx=0x7ffb3000de10 comp rx=0 tx=0).stop 2024-02-18T11:54:03.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.911+0000 7ffb4e888640 1 -- 172.21.15.12:0/3746815011 shutdown_connections 2024-02-18T11:54:03.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.911+0000 7ffb4e888640 1 --2- 172.21.15.12:0/3746815011 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7ffb340a2750 0x7ffb340a4c10 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:03.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.911+0000 7ffb4e888640 1 --2- 172.21.15.12:0/3746815011 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb481045f0 0x7ffb48199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:03.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.911+0000 7ffb4e888640 1 --2- 172.21.15.12:0/3746815011 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ffb48106f20 0x7ffb481a0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:03.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.911+0000 7ffb4e888640 1 --2- 172.21.15.12:0/3746815011 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffb48101cc0 0x7ffb48199500 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:03.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.911+0000 7ffb4e888640 1 -- 172.21.15.12:0/3746815011 >> 172.21.15.12:0/3746815011 conn(0x7ffb480fb650 msgr2=0x7ffb481014f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:54:03.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.911+0000 7ffb4e888640 1 -- 172.21.15.12:0/3746815011 shutdown_connections 2024-02-18T11:54:03.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:03.911+0000 7ffb4e888640 1 -- 172.21.15.12:0/3746815011 wait complete. 2024-02-18T11:54:03.915 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-02-18T11:54:03.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:54:03.915 DEBUG:teuthology.orchestra.run.smithi012:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-02-18T11:54:04.602 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:54:04 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/3415615332' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-18T11:54:04.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:54:04 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/2991939083' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-18T11:54:04.603 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:54:04 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/3746815011' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-18T11:54:04.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:54:04 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/3415615332' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-18T11:54:04.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:54:04 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/2991939083' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-18T11:54:04.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:54:04 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/3746815011' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-18T11:54:04.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:54:04 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/3415615332' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-18T11:54:04.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:54:04 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/2991939083' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-18T11:54:04.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:54:04 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/3746815011' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-18T11:54:05.407 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi012.front.sepia.ceph.com 2024-02-18T11:54:05.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:54:05.407 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-02-18T11:54:05.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:54:05.454 DEBUG:teuthology.orchestra.run.smithi012:> ip netns list 2024-02-18T11:54:05.495 INFO:teuthology.orchestra.run.smithi012.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-02-18T11:54:05.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:54:05.495 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-18T11:54:05.562 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:54:05.562 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link delete ceph-brx 2024-02-18T11:54:05.602 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:54:05 smithi188 ceph-mon[36925]: pgmap v745: 97 pgs: 97 active+clean; 4.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 19 MiB/s wr, 284 op/s 2024-02-18T11:54:05.626 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:54:05 smithi012 ceph-mon[32460]: pgmap v745: 97 pgs: 97 active+clean; 4.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 19 MiB/s wr, 284 op/s 2024-02-18T11:54:05.670 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-02-18T11:54:05.685 INFO:tasks.cephadm:Teardown begin 2024-02-18T11:54:05.686 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-02-18T11:54:05.686 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-18T11:54:05.742 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:54:05 smithi158 ceph-mon[36626]: pgmap v745: 97 pgs: 97 active+clean; 4.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 19 MiB/s wr, 284 op/s 2024-02-18T11:54:05.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:05.821+0000 7f63c280e640 1 -- 172.21.15.12:0/1812224418 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f63bc101cb0 msgr2=0x7f63bc108540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:54:05.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:05.821+0000 7f63c280e640 1 --2- 172.21.15.12:0/1812224418 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f63bc101cb0 0x7f63bc108540 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f63b4009ca0 tx=0x7f63b402c0f0 comp rx=0 tx=0).stop 2024-02-18T11:54:05.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:05.821+0000 7f63c280e640 1 -- 172.21.15.12:0/1812224418 shutdown_connections 2024-02-18T11:54:05.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:05.821+0000 7f63c280e640 1 --2- 172.21.15.12:0/1812224418 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f63bc101cb0 0x7f63bc108540 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:05.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:05.821+0000 7f63c280e640 1 --2- 172.21.15.12:0/1812224418 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f63bc1012f0 0x7f63bc101770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:05.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:05.821+0000 7f63c280e640 1 --2- 172.21.15.12:0/1812224418 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f63bc100100 0x7f63bc100520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:05.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:05.821+0000 7f63c280e640 1 -- 172.21.15.12:0/1812224418 >> 172.21.15.12:0/1812224418 conn(0x7f63bc0fb670 msgr2=0x7f63bc0fdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:54:05.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:05.821+0000 7f63c280e640 1 -- 172.21.15.12:0/1812224418 shutdown_connections 2024-02-18T11:54:05.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:05.821+0000 7f63c280e640 1 -- 172.21.15.12:0/1812224418 wait complete. 2024-02-18T11:54:05.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:05.822+0000 7f63c280e640 1 Processor -- start 2024-02-18T11:54:05.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:05.822+0000 7f63c280e640 1 -- start start 2024-02-18T11:54:05.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:05.822+0000 7f63c280e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f63bc100100 0x7f63bc06a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:54:05.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:05.822+0000 7f63c280e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f63bc1012f0 0x7f63bc19bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:54:05.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:05.822+0000 7f63c280e640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f63bc101cb0 0x7f63bc19e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:54:05.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:05.822+0000 7f63c280e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63bc19edc0 con 0x7f63bc1012f0 2024-02-18T11:54:05.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:05.822+0000 7f63c280e640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63bc19ef30 con 0x7f63bc101cb0 2024-02-18T11:54:05.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:05.822+0000 7f63c280e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63bc19f0f0 con 0x7f63bc100100 2024-02-18T11:54:05.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:05.822+0000 7f63bb7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f63bc1012f0 0x7f63bc19bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:54:05.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:05.822+0000 7f63bb7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f63bc1012f0 0x7f63bc19bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42014/0 (socket says 172.21.15.12:42014) 2024-02-18T11:54:05.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:05.822+0000 7f63bb7fe640 1 -- 172.21.15.12:0/1447666114 learned_addr learned my addr 172.21.15.12:0/1447666114 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:54:05.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:05.822+0000 7f63bbfff640 1 --2- 172.21.15.12:0/1447666114 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f63bc100100 0x7f63bc06a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:54:05.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:05.823+0000 7f63c0dab640 1 --2- 172.21.15.12:0/1447666114 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f63bc101cb0 0x7f63bc19e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:54:05.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:05.823+0000 7f63bb7fe640 1 -- 172.21.15.12:0/1447666114 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f63bc100100 msgr2=0x7f63bc06a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:54:05.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:05.823+0000 7f63bb7fe640 1 --2- 172.21.15.12:0/1447666114 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f63bc100100 0x7f63bc06a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:05.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:05.823+0000 7f63bb7fe640 1 -- 172.21.15.12:0/1447666114 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f63bc101cb0 msgr2=0x7f63bc19e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:54:05.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:05.823+0000 7f63bb7fe640 1 --2- 172.21.15.12:0/1447666114 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f63bc101cb0 0x7f63bc19e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:05.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:05.823+0000 7f63bb7fe640 1 -- 172.21.15.12:0/1447666114 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63b4009900 con 0x7f63bc1012f0 2024-02-18T11:54:05.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:05.823+0000 7f63bbfff640 1 --2- 172.21.15.12:0/1447666114 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f63bc100100 0x7f63bc06a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-18T11:54:05.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:05.823+0000 7f63bb7fe640 1 --2- 172.21.15.12:0/1447666114 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f63bc1012f0 0x7f63bc19bfc0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f63b000d950 tx=0x7f63b000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:54:05.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:05.823+0000 7f63b97fa640 1 -- 172.21.15.12:0/1447666114 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b0009c30 con 0x7f63bc1012f0 2024-02-18T11:54:05.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:05.823+0000 7f63b97fa640 1 -- 172.21.15.12:0/1447666114 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f63b0009d90 con 0x7f63bc1012f0 2024-02-18T11:54:05.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:05.823+0000 7f63c280e640 1 -- 172.21.15.12:0/1447666114 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63bc19f370 con 0x7f63bc1012f0 2024-02-18T11:54:05.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:05.823+0000 7f63b97fa640 1 -- 172.21.15.12:0/1447666114 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b000f7e0 con 0x7f63bc1012f0 2024-02-18T11:54:05.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:05.823+0000 7f63c280e640 1 -- 172.21.15.12:0/1447666114 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63bc19f8d0 con 0x7f63bc1012f0 2024-02-18T11:54:05.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:05.824+0000 7f63c280e640 1 -- 172.21.15.12:0/1447666114 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63bc1073c0 con 0x7f63bc1012f0 2024-02-18T11:54:05.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:05.825+0000 7f63b97fa640 1 -- 172.21.15.12:0/1447666114 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 20) v1 ==== 137245+0+0 (secure 0 0 0) 0x7f63b0010ad0 con 0x7f63bc1012f0 2024-02-18T11:54:05.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:05.826+0000 7f63b97fa640 1 --2- 172.21.15.12:0/1447666114 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f63a40a27d0 0x7f63a40a4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:54:05.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:05.826+0000 7f63bbfff640 1 --2- 172.21.15.12:0/1447666114 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f63a40a27d0 0x7f63a40a4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:54:05.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:05.826+0000 7f63b97fa640 1 -- 172.21.15.12:0/1447666114 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f63b00cdfe0 con 0x7f63bc1012f0 2024-02-18T11:54:05.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:05.826+0000 7f63bbfff640 1 --2- 172.21.15.12:0/1447666114 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f63a40a27d0 0x7f63a40a4c90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f63bc101150 tx=0x7f63ac005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:54:05.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:05.830+0000 7f63b97fa640 1 -- 172.21.15.12:0/1447666114 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f63b0096ed0 con 0x7f63bc1012f0 2024-02-18T11:54:06.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.051+0000 7f63c280e640 1 -- 172.21.15.12:0/1447666114 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f63bc106e30 con 0x7f63bc1012f0 2024-02-18T11:54:06.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.052+0000 7f63b97fa640 1 -- 172.21.15.12:0/1447666114 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+11349 (secure 0 0 0) 0x7f63b009a590 con 0x7f63bc1012f0 2024-02-18T11:54:06.046 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:54:06.046 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1788335302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1788335302},{"type":"v1","addr":"172.21.15.12:6835","nonce":1788335302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/2144692455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":2144692455},{"type":"v1","addr":"172.21.15.12:6837","nonce":2144692455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3569056036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3569056036},{"type":"v1","addr":"172.21.15.12:6839","nonce":3569056036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14622,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/1973234486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":1973234486},{"type":"v1","addr":"172.21.15.12:6841","nonce":1973234486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1207202465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1207202465},{"type":"v1","addr":"172.21.15.188:6833","nonce":1207202465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6833/2636348323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6832","nonce":2636348323},{"type":"v1","addr":"172.21.15.158:6833","nonce":2636348323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-18T11:38:46.888173+0000","modified":"2024-02-18T11:40:39.712523+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24524,"mds_1":24508,"mds_2":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.158:6835/3673253642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":3673253642},{"type":"v1","addr":"172.21.15.158:6835","nonce":3673253642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/806709644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":806709644},{"type":"v1","addr":"172.21.15.188:6835","nonce":806709644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.158:6837/813251825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6836","nonce":813251825},{"type":"v1","addr":"172.21.15.158:6837","nonce":813251825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.188:6837/1036630143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1036630143},{"type":"v1","addr":"172.21.15.188:6837","nonce":1036630143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.188:6839/1342671190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":1342671190},{"type":"v1","addr":"172.21.15.188:6839","nonce":1342671190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.158:6839/1127200849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6838","nonce":1127200849},{"type":"v1","addr":"172.21.15.158:6839","nonce":1127200849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-18T11:54:06.047 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 21 2024-02-18T11:54:06.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.054+0000 7f63c280e640 1 -- 172.21.15.12:0/1447666114 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f63a40a27d0 msgr2=0x7f63a40a4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:54:06.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.054+0000 7f63c280e640 1 --2- 172.21.15.12:0/1447666114 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f63a40a27d0 0x7f63a40a4c90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f63bc101150 tx=0x7f63ac005f90 comp rx=0 tx=0).stop 2024-02-18T11:54:06.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.054+0000 7f63c280e640 1 -- 172.21.15.12:0/1447666114 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f63bc1012f0 msgr2=0x7f63bc19bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:54:06.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.054+0000 7f63c280e640 1 --2- 172.21.15.12:0/1447666114 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f63bc1012f0 0x7f63bc19bfc0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f63b000d950 tx=0x7f63b000de10 comp rx=0 tx=0).stop 2024-02-18T11:54:06.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.054+0000 7f63c280e640 1 -- 172.21.15.12:0/1447666114 shutdown_connections 2024-02-18T11:54:06.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.054+0000 7f63c280e640 1 --2- 172.21.15.12:0/1447666114 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f63a40a27d0 0x7f63a40a4c90 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:06.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.054+0000 7f63c280e640 1 --2- 172.21.15.12:0/1447666114 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f63bc101cb0 0x7f63bc19e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:06.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.054+0000 7f63c280e640 1 --2- 172.21.15.12:0/1447666114 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f63bc1012f0 0x7f63bc19bfc0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:06.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.054+0000 7f63c280e640 1 --2- 172.21.15.12:0/1447666114 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f63bc100100 0x7f63bc06a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:06.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.054+0000 7f63c280e640 1 -- 172.21.15.12:0/1447666114 >> 172.21.15.12:0/1447666114 conn(0x7f63bc0fb670 msgr2=0x7f63bc104590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:54:06.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.054+0000 7f63c280e640 1 -- 172.21.15.12:0/1447666114 shutdown_connections 2024-02-18T11:54:06.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.054+0000 7f63c280e640 1 -- 172.21.15.12:0/1447666114 wait complete. 2024-02-18T11:54:06.059 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-18T11:54:06.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.169+0000 7f16c2f56640 1 -- 172.21.15.12:0/377161477 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16bc1000e0 msgr2=0x7f16bc100500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:54:06.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.169+0000 7f16c2f56640 1 --2- 172.21.15.12:0/377161477 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16bc1000e0 0x7f16bc100500 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f16b8009860 tx=0x7f16b802bb70 comp rx=0 tx=0).stop 2024-02-18T11:54:06.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.169+0000 7f16c2f56640 1 -- 172.21.15.12:0/377161477 shutdown_connections 2024-02-18T11:54:06.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.169+0000 7f16c2f56640 1 --2- 172.21.15.12:0/377161477 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f16bc101c90 0x7f16bc108520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:06.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.169+0000 7f16c2f56640 1 --2- 172.21.15.12:0/377161477 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f16bc1012d0 0x7f16bc101750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:06.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.169+0000 7f16c2f56640 1 --2- 172.21.15.12:0/377161477 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16bc1000e0 0x7f16bc100500 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:06.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.169+0000 7f16c2f56640 1 -- 172.21.15.12:0/377161477 >> 172.21.15.12:0/377161477 conn(0x7f16bc0fb650 msgr2=0x7f16bc0fda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:54:06.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.169+0000 7f16c2f56640 1 -- 172.21.15.12:0/377161477 shutdown_connections 2024-02-18T11:54:06.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.169+0000 7f16c2f56640 1 -- 172.21.15.12:0/377161477 wait complete. 2024-02-18T11:54:06.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.170+0000 7f16c2f56640 1 Processor -- start 2024-02-18T11:54:06.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.170+0000 7f16c2f56640 1 -- start start 2024-02-18T11:54:06.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.170+0000 7f16c2f56640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f16bc1000e0 0x7f16bc1994c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:54:06.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.170+0000 7f16c2f56640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16bc1012d0 0x7f16bc199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:54:06.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.170+0000 7f16c2f56640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f16bc101c90 0x7f16bc1a0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:54:06.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.170+0000 7f16c2f56640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16bc1a0fb0 con 0x7f16bc101c90 2024-02-18T11:54:06.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.170+0000 7f16c2f56640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16bc1a1120 con 0x7f16bc1000e0 2024-02-18T11:54:06.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.170+0000 7f16c2f56640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16bc1a12e0 con 0x7f16bc1012d0 2024-02-18T11:54:06.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.170+0000 7f16c14f3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f16bc101c90 0x7f16bc1a0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:54:06.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.170+0000 7f16c14f3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f16bc101c90 0x7f16bc1a0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42044/0 (socket says 172.21.15.12:42044) 2024-02-18T11:54:06.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.170+0000 7f16c14f3640 1 -- 172.21.15.12:0/1652843226 learned_addr learned my addr 172.21.15.12:0/1652843226 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:54:06.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.170+0000 7f16b3fff640 1 --2- 172.21.15.12:0/1652843226 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16bc1012d0 0x7f16bc199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:54:06.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.170+0000 7f16c0cf2640 1 --2- 172.21.15.12:0/1652843226 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f16bc1000e0 0x7f16bc1994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:54:06.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.170+0000 7f16c14f3640 1 -- 172.21.15.12:0/1652843226 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16bc1012d0 msgr2=0x7f16bc199a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:54:06.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.170+0000 7f16c14f3640 1 --2- 172.21.15.12:0/1652843226 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16bc1012d0 0x7f16bc199a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:06.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.170+0000 7f16c14f3640 1 -- 172.21.15.12:0/1652843226 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f16bc1000e0 msgr2=0x7f16bc1994c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:54:06.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.170+0000 7f16c14f3640 1 --2- 172.21.15.12:0/1652843226 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f16bc1000e0 0x7f16bc1994c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:06.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.171+0000 7f16c14f3640 1 -- 172.21.15.12:0/1652843226 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16b4009900 con 0x7f16bc101c90 2024-02-18T11:54:06.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.171+0000 7f16c14f3640 1 --2- 172.21.15.12:0/1652843226 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f16bc101c90 0x7f16bc1a0a70 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f16b400efc0 tx=0x7f16b400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:54:06.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.171+0000 7f16b1ffb640 1 -- 172.21.15.12:0/1652843226 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b4003970 con 0x7f16bc101c90 2024-02-18T11:54:06.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.171+0000 7f16b1ffb640 1 -- 172.21.15.12:0/1652843226 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f16b4004020 con 0x7f16bc101c90 2024-02-18T11:54:06.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.171+0000 7f16b1ffb640 1 -- 172.21.15.12:0/1652843226 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b400b9b0 con 0x7f16bc101c90 2024-02-18T11:54:06.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.171+0000 7f16c2f56640 1 -- 172.21.15.12:0/1652843226 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16b8009490 con 0x7f16bc101c90 2024-02-18T11:54:06.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.171+0000 7f16c2f56640 1 -- 172.21.15.12:0/1652843226 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16bc1a18f0 con 0x7f16bc101c90 2024-02-18T11:54:06.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.172+0000 7f16c2f56640 1 -- 172.21.15.12:0/1652843226 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16bc10baf0 con 0x7f16bc101c90 2024-02-18T11:54:06.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.173+0000 7f16b1ffb640 1 -- 172.21.15.12:0/1652843226 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 20) v1 ==== 137245+0+0 (secure 0 0 0) 0x7f16b400a040 con 0x7f16bc101c90 2024-02-18T11:54:06.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.173+0000 7f16b1ffb640 1 --2- 172.21.15.12:0/1652843226 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f16a40a2750 0x7f16a40a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:54:06.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.174+0000 7f16c0cf2640 1 --2- 172.21.15.12:0/1652843226 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f16a40a2750 0x7f16a40a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:54:06.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.174+0000 7f16b1ffb640 1 -- 172.21.15.12:0/1652843226 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f16b40ce5f0 con 0x7f16bc101c90 2024-02-18T11:54:06.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.174+0000 7f16c0cf2640 1 --2- 172.21.15.12:0/1652843226 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f16a40a2750 0x7f16a40a4c10 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f16b8009830 tx=0x7f16b8005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:54:06.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.178+0000 7f16b1ffb640 1 -- 172.21.15.12:0/1652843226 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f16b4097390 con 0x7f16bc101c90 2024-02-18T11:54:06.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.521+0000 7f16c2f56640 1 -- 172.21.15.12:0/1652843226 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f16bc1a1d20 con 0x7f16bc101c90 2024-02-18T11:54:06.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.522+0000 7f16b1ffb640 1 -- 172.21.15.12:0/1652843226 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+11349 (secure 0 0 0) 0x7f16b4004190 con 0x7f16bc101c90 2024-02-18T11:54:06.515 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:54:06.515 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1788335302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1788335302},{"type":"v1","addr":"172.21.15.12:6835","nonce":1788335302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/2144692455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":2144692455},{"type":"v1","addr":"172.21.15.12:6837","nonce":2144692455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3569056036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3569056036},{"type":"v1","addr":"172.21.15.12:6839","nonce":3569056036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14622,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/1973234486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":1973234486},{"type":"v1","addr":"172.21.15.12:6841","nonce":1973234486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1207202465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1207202465},{"type":"v1","addr":"172.21.15.188:6833","nonce":1207202465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6833/2636348323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6832","nonce":2636348323},{"type":"v1","addr":"172.21.15.158:6833","nonce":2636348323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-18T11:38:46.888173+0000","modified":"2024-02-18T11:40:39.712523+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24524,"mds_1":24508,"mds_2":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.158:6835/3673253642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":3673253642},{"type":"v1","addr":"172.21.15.158:6835","nonce":3673253642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/806709644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":806709644},{"type":"v1","addr":"172.21.15.188:6835","nonce":806709644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.158:6837/813251825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6836","nonce":813251825},{"type":"v1","addr":"172.21.15.158:6837","nonce":813251825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.188:6837/1036630143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1036630143},{"type":"v1","addr":"172.21.15.188:6837","nonce":1036630143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.188:6839/1342671190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":1342671190},{"type":"v1","addr":"172.21.15.188:6839","nonce":1342671190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.158:6839/1127200849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6838","nonce":1127200849},{"type":"v1","addr":"172.21.15.158:6839","nonce":1127200849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-18T11:54:06.516 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 21 2024-02-18T11:54:06.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.525+0000 7f16c2f56640 1 -- 172.21.15.12:0/1652843226 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f16a40a2750 msgr2=0x7f16a40a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:54:06.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.525+0000 7f16c2f56640 1 --2- 172.21.15.12:0/1652843226 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f16a40a2750 0x7f16a40a4c10 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f16b8009830 tx=0x7f16b8005da0 comp rx=0 tx=0).stop 2024-02-18T11:54:06.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.525+0000 7f16c2f56640 1 -- 172.21.15.12:0/1652843226 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f16bc101c90 msgr2=0x7f16bc1a0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:54:06.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.525+0000 7f16c2f56640 1 --2- 172.21.15.12:0/1652843226 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f16bc101c90 0x7f16bc1a0a70 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f16b400efc0 tx=0x7f16b400c800 comp rx=0 tx=0).stop 2024-02-18T11:54:06.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.525+0000 7f16c2f56640 1 -- 172.21.15.12:0/1652843226 shutdown_connections 2024-02-18T11:54:06.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.525+0000 7f16c2f56640 1 --2- 172.21.15.12:0/1652843226 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f16a40a2750 0x7f16a40a4c10 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:06.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.525+0000 7f16c2f56640 1 --2- 172.21.15.12:0/1652843226 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16bc1012d0 0x7f16bc199a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:06.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.525+0000 7f16c2f56640 1 --2- 172.21.15.12:0/1652843226 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f16bc101c90 0x7f16bc1a0a70 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:06.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.525+0000 7f16c2f56640 1 --2- 172.21.15.12:0/1652843226 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f16bc1000e0 0x7f16bc1994c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:06.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.525+0000 7f16c2f56640 1 -- 172.21.15.12:0/1652843226 >> 172.21.15.12:0/1652843226 conn(0x7f16bc0fb650 msgr2=0x7f16bc0fd150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:54:06.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.525+0000 7f16c2f56640 1 -- 172.21.15.12:0/1652843226 shutdown_connections 2024-02-18T11:54:06.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.525+0000 7f16c2f56640 1 -- 172.21.15.12:0/1652843226 wait complete. 2024-02-18T11:54:06.537 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-18T11:54:06.602 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:54:06 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/1447666114' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-18T11:54:06.671 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:54:06 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/1447666114' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-18T11:54:06.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.693+0000 7f5da5e15640 1 -- 172.21.15.12:0/1222808981 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5da0101d10 msgr2=0x7f5da01085a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:54:06.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.693+0000 7f5da5e15640 1 --2- 172.21.15.12:0/1222808981 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5da0101d10 0x7f5da01085a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5d98009ca0 tx=0x7f5d9802c120 comp rx=0 tx=0).stop 2024-02-18T11:54:06.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.693+0000 7f5da5e15640 1 -- 172.21.15.12:0/1222808981 shutdown_connections 2024-02-18T11:54:06.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.693+0000 7f5da5e15640 1 --2- 172.21.15.12:0/1222808981 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5da0101350 0x7f5da01017d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:06.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.693+0000 7f5da5e15640 1 --2- 172.21.15.12:0/1222808981 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5da0101d10 0x7f5da01085a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:06.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.693+0000 7f5da5e15640 1 --2- 172.21.15.12:0/1222808981 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5da0100160 0x7f5da0100580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:06.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.693+0000 7f5da5e15640 1 -- 172.21.15.12:0/1222808981 >> 172.21.15.12:0/1222808981 conn(0x7f5da00fb6f0 msgr2=0x7f5da00fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:54:06.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.693+0000 7f5da5e15640 1 -- 172.21.15.12:0/1222808981 shutdown_connections 2024-02-18T11:54:06.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.693+0000 7f5da5e15640 1 -- 172.21.15.12:0/1222808981 wait complete. 2024-02-18T11:54:06.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.693+0000 7f5da5e15640 1 Processor -- start 2024-02-18T11:54:06.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.694+0000 7f5da5e15640 1 -- start start 2024-02-18T11:54:06.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.694+0000 7f5da5e15640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5da0100160 0x7f5da01994c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:54:06.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.694+0000 7f5da5e15640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5da0101350 0x7f5da0199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:54:06.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.694+0000 7f5da5e15640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5da0101d10 0x7f5da01a0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:54:06.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.694+0000 7f5da5e15640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da01a0fb0 con 0x7f5da0101d10 2024-02-18T11:54:06.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.694+0000 7f5da5e15640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da01a1120 con 0x7f5da0100160 2024-02-18T11:54:06.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.694+0000 7f5da5e15640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da01a12e0 con 0x7f5da0101350 2024-02-18T11:54:06.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.694+0000 7f5d9ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5da0101d10 0x7f5da01a0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:54:06.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.694+0000 7f5d9f7fe640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5da0100160 0x7f5da01994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:54:06.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.694+0000 7f5d9ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5da0101d10 0x7f5da01a0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42072/0 (socket says 172.21.15.12:42072) 2024-02-18T11:54:06.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.694+0000 7f5d9ffff640 1 -- 172.21.15.12:0/1679864819 learned_addr learned my addr 172.21.15.12:0/1679864819 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:54:06.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.694+0000 7f5d9effd640 1 --2- 172.21.15.12:0/1679864819 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5da0101350 0x7f5da0199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:54:06.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.695+0000 7f5d9f7fe640 1 -- 172.21.15.12:0/1679864819 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5da0101350 msgr2=0x7f5da0199a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:54:06.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.695+0000 7f5d9f7fe640 1 --2- 172.21.15.12:0/1679864819 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5da0101350 0x7f5da0199a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:06.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.695+0000 7f5d9f7fe640 1 -- 172.21.15.12:0/1679864819 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5da0101d10 msgr2=0x7f5da01a0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:54:06.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.695+0000 7f5d9f7fe640 1 --2- 172.21.15.12:0/1679864819 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5da0101d10 0x7f5da01a0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:06.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.695+0000 7f5d9f7fe640 1 -- 172.21.15.12:0/1679864819 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d98009900 con 0x7f5da0100160 2024-02-18T11:54:06.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.695+0000 7f5d9ffff640 1 --2- 172.21.15.12:0/1679864819 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5da0101d10 0x7f5da01a0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-18T11:54:06.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.695+0000 7f5d9f7fe640 1 --2- 172.21.15.12:0/1679864819 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5da0100160 0x7f5da01994c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5da01011b0 tx=0x7f5d9000dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:54:06.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.696+0000 7f5d9cff9640 1 -- 172.21.15.12:0/1679864819 <== mon.2 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d90009c30 con 0x7f5da0100160 2024-02-18T11:54:06.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.696+0000 7f5da5e15640 1 -- 172.21.15.12:0/1679864819 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5da01a15c0 con 0x7f5da0100160 2024-02-18T11:54:06.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.696+0000 7f5da5e15640 1 -- 172.21.15.12:0/1679864819 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5da01a1b20 con 0x7f5da0100160 2024-02-18T11:54:06.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.696+0000 7f5d9cff9640 1 -- 172.21.15.12:0/1679864819 <== mon.2 v2:172.21.15.158:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5d90009d90 con 0x7f5da0100160 2024-02-18T11:54:06.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.696+0000 7f5d9cff9640 1 -- 172.21.15.12:0/1679864819 <== mon.2 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d9000f740 con 0x7f5da0100160 2024-02-18T11:54:06.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.698+0000 7f5da5e15640 1 -- 172.21.15.12:0/1679864819 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d8c0052c0 con 0x7f5da0100160 2024-02-18T11:54:06.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.699+0000 7f5d9cff9640 1 -- 172.21.15.12:0/1679864819 <== mon.2 v2:172.21.15.158:3300/0 4 ==== mgrmap(e 20) v1 ==== 137245+0+0 (secure 0 0 0) 0x7f5d900109d0 con 0x7f5da0100160 2024-02-18T11:54:06.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.700+0000 7f5d9cff9640 1 --2- 172.21.15.12:0/1679864819 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f5d880a27d0 0x7f5d880a4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:54:06.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.700+0000 7f5d9cff9640 1 -- 172.21.15.12:0/1679864819 <== mon.2 v2:172.21.15.158:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5d900ce4c0 con 0x7f5da0100160 2024-02-18T11:54:06.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.701+0000 7f5d9effd640 1 --2- 172.21.15.12:0/1679864819 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f5d880a27d0 0x7f5d880a4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:54:06.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.701+0000 7f5d9effd640 1 --2- 172.21.15.12:0/1679864819 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f5d880a27d0 0x7f5d880a4c90 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5da019a9d0 tx=0x7f5d940090e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:54:06.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:06.710+0000 7f5d9cff9640 1 -- 172.21.15.12:0/1679864819 <== mon.2 v2:172.21.15.158:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5d90097310 con 0x7f5da0100160 2024-02-18T11:54:06.742 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:54:06 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/1447666114' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-18T11:54:06.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.002+0000 7f5da5e15640 1 -- 172.21.15.12:0/1679864819 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5d8c005570 con 0x7f5da0100160 2024-02-18T11:54:06.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.005+0000 7f5d9cff9640 1 -- 172.21.15.12:0/1679864819 <== mon.2 v2:172.21.15.158:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+22662 (secure 0 0 0) 0x7f5d9009a9d0 con 0x7f5da0100160 2024-02-18T11:54:06.998 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-02-18T11:54:06.998 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":89,"fsid":"0c8bb0de-ce51-11ee-95ba-87774f69a715","created":"2024-02-18T11:30:49.110059+0000","modified":"2024-02-18T11:39:52.478480+0000","last_up_change":"2024-02-18T11:37:57.512337+0000","last_in_change":"2024-02-18T11:37:36.567969+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-18T11:34:15.281138+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-18T11:38:44.802134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-18T11:38:46.081188+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"35f15ea6-deb8-48a1-a7dd-ab649ec13533","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":536522426},{"type":"v1","addr":"172.21.15.12:6803","nonce":536522426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":536522426},{"type":"v1","addr":"172.21.15.12:6805","nonce":536522426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":536522426},{"type":"v1","addr":"172.21.15.12:6809","nonce":536522426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":536522426},{"type":"v1","addr":"172.21.15.12:6807","nonce":536522426}]},"public_addr":"172.21.15.12:6803/536522426","cluster_addr":"172.21.15.12:6805/536522426","heartbeat_back_addr":"172.21.15.12:6809/536522426","heartbeat_front_addr":"172.21.15.12:6807/536522426","state":["exists","up"]},{"osd":1,"uuid":"2af937ef-d827-44e6-b805-70cffbfe1126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":4249111488},{"type":"v1","addr":"172.21.15.158:6801","nonce":4249111488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":4249111488},{"type":"v1","addr":"172.21.15.158:6803","nonce":4249111488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":4249111488},{"type":"v1","addr":"172.21.15.158:6807","nonce":4249111488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":4249111488},{"type":"v1","addr":"172.21.15.158:6805","nonce":4249111488}]},"public_addr":"172.21.15.158:6801/4249111488","cluster_addr":"172.21.15.158:6803/4249111488","heartbeat_back_addr":"172.21.15.158:6807/4249111488","heartbeat_front_addr":"172.21.15.158:6805/4249111488","state":["exists","up"]},{"osd":2,"uuid":"17b804dd-31df-45ae-a637-4a5f65bbaf26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":627976150},{"type":"v1","addr":"172.21.15.188:6801","nonce":627976150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":627976150},{"type":"v1","addr":"172.21.15.188:6803","nonce":627976150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":627976150},{"type":"v1","addr":"172.21.15.188:6807","nonce":627976150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":627976150},{"type":"v1","addr":"172.21.15.188:6805","nonce":627976150}]},"public_addr":"172.21.15.188:6801/627976150","cluster_addr":"172.21.15.188:6803/627976150","heartbeat_back_addr":"172.21.15.188:6807/627976150","heartbeat_front_addr":"172.21.15.188:6805/627976150","state":["exists","up"]},{"osd":3,"uuid":"3f9cc423-cdfa-4acf-aaa2-b58b32ccb9cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":3758512503},{"type":"v1","addr":"172.21.15.12:6811","nonce":3758512503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":3758512503},{"type":"v1","addr":"172.21.15.12:6813","nonce":3758512503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3758512503},{"type":"v1","addr":"172.21.15.12:6817","nonce":3758512503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":3758512503},{"type":"v1","addr":"172.21.15.12:6815","nonce":3758512503}]},"public_addr":"172.21.15.12:6811/3758512503","cluster_addr":"172.21.15.12:6813/3758512503","heartbeat_back_addr":"172.21.15.12:6817/3758512503","heartbeat_front_addr":"172.21.15.12:6815/3758512503","state":["exists","up"]},{"osd":4,"uuid":"a92e20ec-af1c-4ef4-a0cb-caba5fcc71aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":3892820997},{"type":"v1","addr":"172.21.15.158:6809","nonce":3892820997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":3892820997},{"type":"v1","addr":"172.21.15.158:6811","nonce":3892820997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":3892820997},{"type":"v1","addr":"172.21.15.158:6815","nonce":3892820997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":3892820997},{"type":"v1","addr":"172.21.15.158:6813","nonce":3892820997}]},"public_addr":"172.21.15.158:6809/3892820997","cluster_addr":"172.21.15.158:6811/3892820997","heartbeat_back_addr":"172.21.15.158:6815/3892820997","heartbeat_front_addr":"172.21.15.158:6813/3892820997","state":["exists","up"]},{"osd":5,"uuid":"d9c2e0c8-85f5-4ae2-bf3e-125c1be3a2db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3939495896},{"type":"v1","addr":"172.21.15.188:6809","nonce":3939495896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3939495896},{"type":"v1","addr":"172.21.15.188:6811","nonce":3939495896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3939495896},{"type":"v1","addr":"172.21.15.188:6815","nonce":3939495896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3939495896},{"type":"v1","addr":"172.21.15.188:6813","nonce":3939495896}]},"public_addr":"172.21.15.188:6809/3939495896","cluster_addr":"172.21.15.188:6811/3939495896","heartbeat_back_addr":"172.21.15.188:6815/3939495896","heartbeat_front_addr":"172.21.15.188:6813/3939495896","state":["exists","up"]},{"osd":6,"uuid":"4a523512-de6b-49f0-be81-00f2f00c1c10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":1206337767},{"type":"v1","addr":"172.21.15.12:6819","nonce":1206337767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":1206337767},{"type":"v1","addr":"172.21.15.12:6821","nonce":1206337767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":1206337767},{"type":"v1","addr":"172.21.15.12:6825","nonce":1206337767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":1206337767},{"type":"v1","addr":"172.21.15.12:6823","nonce":1206337767}]},"public_addr":"172.21.15.12:6819/1206337767","cluster_addr":"172.21.15.12:6821/1206337767","heartbeat_back_addr":"172.21.15.12:6825/1206337767","heartbeat_front_addr":"172.21.15.12:6823/1206337767","state":["exists","up"]},{"osd":7,"uuid":"363271a6-df93-44c9-b0e5-3d29768aa96c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":3283400032},{"type":"v1","addr":"172.21.15.158:6817","nonce":3283400032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":3283400032},{"type":"v1","addr":"172.21.15.158:6819","nonce":3283400032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":3283400032},{"type":"v1","addr":"172.21.15.158:6823","nonce":3283400032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":3283400032},{"type":"v1","addr":"172.21.15.158:6821","nonce":3283400032}]},"public_addr":"172.21.15.158:6817/3283400032","cluster_addr":"172.21.15.158:6819/3283400032","heartbeat_back_addr":"172.21.15.158:6823/3283400032","heartbeat_front_addr":"172.21.15.158:6821/3283400032","state":["exists","up"]},{"osd":8,"uuid":"9a089528-3e8e-45d3-9279-ddb54088b96a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2110170682},{"type":"v1","addr":"172.21.15.188:6817","nonce":2110170682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2110170682},{"type":"v1","addr":"172.21.15.188:6819","nonce":2110170682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":2110170682},{"type":"v1","addr":"172.21.15.188:6823","nonce":2110170682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2110170682},{"type":"v1","addr":"172.21.15.188:6821","nonce":2110170682}]},"public_addr":"172.21.15.188:6817/2110170682","cluster_addr":"172.21.15.188:6819/2110170682","heartbeat_back_addr":"172.21.15.188:6823/2110170682","heartbeat_front_addr":"172.21.15.188:6821/2110170682","state":["exists","up"]},{"osd":9,"uuid":"b12e2727-d649-4464-90e1-fafbc1754832","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1825928986},{"type":"v1","addr":"172.21.15.12:6827","nonce":1825928986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1825928986},{"type":"v1","addr":"172.21.15.12:6829","nonce":1825928986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":1825928986},{"type":"v1","addr":"172.21.15.12:6833","nonce":1825928986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1825928986},{"type":"v1","addr":"172.21.15.12:6831","nonce":1825928986}]},"public_addr":"172.21.15.12:6827/1825928986","cluster_addr":"172.21.15.12:6829/1825928986","heartbeat_back_addr":"172.21.15.12:6833/1825928986","heartbeat_front_addr":"172.21.15.12:6831/1825928986","state":["exists","up"]},{"osd":10,"uuid":"518b4cbf-9d90-4e9a-8ac9-b5cb9b3dee4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":3239052167},{"type":"v1","addr":"172.21.15.158:6825","nonce":3239052167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":3239052167},{"type":"v1","addr":"172.21.15.158:6827","nonce":3239052167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":3239052167},{"type":"v1","addr":"172.21.15.158:6831","nonce":3239052167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":3239052167},{"type":"v1","addr":"172.21.15.158:6829","nonce":3239052167}]},"public_addr":"172.21.15.158:6825/3239052167","cluster_addr":"172.21.15.158:6827/3239052167","heartbeat_back_addr":"172.21.15.158:6831/3239052167","heartbeat_front_addr":"172.21.15.158:6829/3239052167","state":["exists","up"]},{"osd":11,"uuid":"98aa3a0d-4b2f-46c8-9341-375d13794c3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1135773350},{"type":"v1","addr":"172.21.15.188:6825","nonce":1135773350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1135773350},{"type":"v1","addr":"172.21.15.188:6827","nonce":1135773350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1135773350},{"type":"v1","addr":"172.21.15.188:6831","nonce":1135773350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1135773350},{"type":"v1","addr":"172.21.15.188:6829","nonce":1135773350}]},"public_addr":"172.21.15.188:6825/1135773350","cluster_addr":"172.21.15.188:6827/1135773350","heartbeat_back_addr":"172.21.15.188:6831/1135773350","heartbeat_front_addr":"172.21.15.188:6829/1135773350","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:33:24.280879+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:33:47.859461+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:34:11.725283+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:34:35.707629+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:35:00.024197+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:35:24.499803+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:35:48.953870+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:36:13.642456+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:36:38.936457+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:37:04.021466+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:37:29.073178+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:37:54.863919+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.37","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:6801/4135274411":"2024-02-19T11:31:50.936928+0000","172.21.15.12:6800/4135274411":"2024-02-19T11:31:50.936928+0000","172.21.15.12:0/118428584":"2024-02-19T11:31:50.936928+0000","172.21.15.12:0/4276810915":"2024-02-19T11:31:50.936928+0000","172.21.15.12:0/3777545262":"2024-02-19T11:31:24.000830+0000","172.21.15.12:0/114707743":"2024-02-19T11:31:24.000830+0000","172.21.15.12:6800/3925204723":"2024-02-19T11:31:24.000830+0000","172.21.15.12:6801/3925204723":"2024-02-19T11:31:24.000830+0000","172.21.15.12:0/74881618":"2024-02-19T11:31:24.000830+0000","172.21.15.12:0/1614805375":"2024-02-19T11:31:50.936928+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-18T11:54:07.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.008+0000 7f5da5e15640 1 -- 172.21.15.12:0/1679864819 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f5d880a27d0 msgr2=0x7f5d880a4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:54:07.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.008+0000 7f5da5e15640 1 --2- 172.21.15.12:0/1679864819 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f5d880a27d0 0x7f5d880a4c90 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5da019a9d0 tx=0x7f5d940090e0 comp rx=0 tx=0).stop 2024-02-18T11:54:07.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.008+0000 7f5da5e15640 1 -- 172.21.15.12:0/1679864819 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5da0100160 msgr2=0x7f5da01994c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:54:07.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.008+0000 7f5da5e15640 1 --2- 172.21.15.12:0/1679864819 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5da0100160 0x7f5da01994c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5da01011b0 tx=0x7f5d9000dde0 comp rx=0 tx=0).stop 2024-02-18T11:54:07.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.008+0000 7f5da5e15640 1 -- 172.21.15.12:0/1679864819 shutdown_connections 2024-02-18T11:54:07.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.008+0000 7f5da5e15640 1 --2- 172.21.15.12:0/1679864819 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f5d880a27d0 0x7f5d880a4c90 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:07.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.008+0000 7f5da5e15640 1 --2- 172.21.15.12:0/1679864819 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5da0101350 0x7f5da0199a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:07.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.008+0000 7f5da5e15640 1 --2- 172.21.15.12:0/1679864819 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5da0101d10 0x7f5da01a0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:07.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.008+0000 7f5da5e15640 1 --2- 172.21.15.12:0/1679864819 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5da0100160 0x7f5da01994c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:07.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.008+0000 7f5da5e15640 1 -- 172.21.15.12:0/1679864819 >> 172.21.15.12:0/1679864819 conn(0x7f5da00fb6f0 msgr2=0x7f5da00fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:54:07.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.008+0000 7f5da5e15640 1 -- 172.21.15.12:0/1679864819 shutdown_connections 2024-02-18T11:54:07.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.008+0000 7f5da5e15640 1 -- 172.21.15.12:0/1679864819 wait complete. 2024-02-18T11:54:07.014 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-02-18T11:54:07.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.129+0000 7fa03292c640 1 -- 172.21.15.12:0/2150572312 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa02c1024f0 msgr2=0x7fa02c102970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:54:07.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.129+0000 7fa03292c640 1 --2- 172.21.15.12:0/2150572312 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa02c1024f0 0x7fa02c102970 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fa020009900 tx=0x7fa02002bc30 comp rx=0 tx=0).stop 2024-02-18T11:54:07.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.129+0000 7fa03292c640 1 -- 172.21.15.12:0/2150572312 shutdown_connections 2024-02-18T11:54:07.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.129+0000 7fa03292c640 1 --2- 172.21.15.12:0/2150572312 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa02c102eb0 0x7fa02c109740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:07.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.129+0000 7fa03292c640 1 --2- 172.21.15.12:0/2150572312 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa02c1024f0 0x7fa02c102970 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:07.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.129+0000 7fa03292c640 1 --2- 172.21.15.12:0/2150572312 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa02c0ff480 0x7fa02c0ff8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:07.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.129+0000 7fa03292c640 1 -- 172.21.15.12:0/2150572312 >> 172.21.15.12:0/2150572312 conn(0x7fa02c0fadb0 msgr2=0x7fa02c0fd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:54:07.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.129+0000 7fa03292c640 1 -- 172.21.15.12:0/2150572312 shutdown_connections 2024-02-18T11:54:07.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.129+0000 7fa03292c640 1 -- 172.21.15.12:0/2150572312 wait complete. 2024-02-18T11:54:07.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.130+0000 7fa03292c640 1 Processor -- start 2024-02-18T11:54:07.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.130+0000 7fa03292c640 1 -- start start 2024-02-18T11:54:07.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.130+0000 7fa03292c640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa02c0ff480 0x7fa02c101e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:54:07.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.130+0000 7fa03292c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa02c1024f0 0x7fa02c1004a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:54:07.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.130+0000 7fa03292c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa02c102eb0 0x7fa02c1009e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:54:07.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.130+0000 7fa03292c640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa02c101140 con 0x7fa02c102eb0 2024-02-18T11:54:07.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.130+0000 7fa03292c640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa02c1012b0 con 0x7fa02c0ff480 2024-02-18T11:54:07.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.130+0000 7fa03292c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa02c1a12c0 con 0x7fa02c1024f0 2024-02-18T11:54:07.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.130+0000 7fa030ec9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa02c102eb0 0x7fa02c1009e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:54:07.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.130+0000 7fa02bfff640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa02c0ff480 0x7fa02c101e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:54:07.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.130+0000 7fa030ec9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa02c102eb0 0x7fa02c1009e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42090/0 (socket says 172.21.15.12:42090) 2024-02-18T11:54:07.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.130+0000 7fa030ec9640 1 -- 172.21.15.12:0/3562465032 learned_addr learned my addr 172.21.15.12:0/3562465032 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:54:07.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.130+0000 7fa02b7fe640 1 --2- 172.21.15.12:0/3562465032 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa02c1024f0 0x7fa02c1004a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:54:07.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.130+0000 7fa02bfff640 1 -- 172.21.15.12:0/3562465032 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa02c1024f0 msgr2=0x7fa02c1004a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:54:07.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.130+0000 7fa02bfff640 1 --2- 172.21.15.12:0/3562465032 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa02c1024f0 0x7fa02c1004a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:07.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.130+0000 7fa02bfff640 1 -- 172.21.15.12:0/3562465032 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa02c102eb0 msgr2=0x7fa02c1009e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:54:07.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.130+0000 7fa02bfff640 1 --2- 172.21.15.12:0/3562465032 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa02c102eb0 0x7fa02c1009e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:07.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.130+0000 7fa02bfff640 1 -- 172.21.15.12:0/3562465032 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa020009560 con 0x7fa02c0ff480 2024-02-18T11:54:07.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.130+0000 7fa030ec9640 1 --2- 172.21.15.12:0/3562465032 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa02c102eb0 0x7fa02c1009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-18T11:54:07.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.131+0000 7fa02bfff640 1 --2- 172.21.15.12:0/3562465032 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa02c0ff480 0x7fa02c101e60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa01c00e9f0 tx=0x7fa01c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:54:07.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.131+0000 7fa0297fa640 1 -- 172.21.15.12:0/3562465032 <== mon.2 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa01c00ccc0 con 0x7fa02c0ff480 2024-02-18T11:54:07.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.131+0000 7fa03292c640 1 -- 172.21.15.12:0/3562465032 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa02c1a1570 con 0x7fa02c0ff480 2024-02-18T11:54:07.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.131+0000 7fa0297fa640 1 -- 172.21.15.12:0/3562465032 <== mon.2 v2:172.21.15.158:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa01c00ce20 con 0x7fa02c0ff480 2024-02-18T11:54:07.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.131+0000 7fa0297fa640 1 -- 172.21.15.12:0/3562465032 <== mon.2 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa01c010430 con 0x7fa02c0ff480 2024-02-18T11:54:07.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.131+0000 7fa03292c640 1 -- 172.21.15.12:0/3562465032 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa02c1a1aa0 con 0x7fa02c0ff480 2024-02-18T11:54:07.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.132+0000 7fa03292c640 1 -- 172.21.15.12:0/3562465032 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa02c108620 con 0x7fa02c0ff480 2024-02-18T11:54:07.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.133+0000 7fa0297fa640 1 -- 172.21.15.12:0/3562465032 <== mon.2 v2:172.21.15.158:3300/0 4 ==== mgrmap(e 20) v1 ==== 137245+0+0 (secure 0 0 0) 0x7fa01c0035f0 con 0x7fa02c0ff480 2024-02-18T11:54:07.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.134+0000 7fa0297fa640 1 --2- 172.21.15.12:0/3562465032 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fa0140a2750 0x7fa0140a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:54:07.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.134+0000 7fa02b7fe640 1 --2- 172.21.15.12:0/3562465032 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fa0140a2750 0x7fa0140a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:54:07.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.134+0000 7fa0297fa640 1 -- 172.21.15.12:0/3562465032 <== mon.2 v2:172.21.15.158:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa01c014070 con 0x7fa02c0ff480 2024-02-18T11:54:07.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.134+0000 7fa02b7fe640 1 --2- 172.21.15.12:0/3562465032 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fa0140a2750 0x7fa0140a4c10 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa020009230 tx=0x7fa020005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:54:07.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.139+0000 7fa0297fa640 1 -- 172.21.15.12:0/3562465032 <== mon.2 v2:172.21.15.158:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa01c097870 con 0x7fa02c0ff480 2024-02-18T11:54:07.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.357+0000 7fa03292c640 1 -- 172.21.15.12:0/3562465032 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fa02c10cda0 con 0x7fa02c0ff480 2024-02-18T11:54:07.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.374+0000 7fa0297fa640 1 -- 172.21.15.12:0/3562465032 <== mon.2 v2:172.21.15.158:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v22) v1 ==== 161+0+0 (secure 0 0 0) 0x7fa02c10cda0 con 0x7fa02c0ff480 2024-02-18T11:54:07.367 INFO:teuthology.orchestra.run.smithi012.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-02-18T11:54:07.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.376+0000 7fa03292c640 1 -- 172.21.15.12:0/3562465032 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fa0140a2750 msgr2=0x7fa0140a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:54:07.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.376+0000 7fa03292c640 1 --2- 172.21.15.12:0/3562465032 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fa0140a2750 0x7fa0140a4c10 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa020009230 tx=0x7fa020005fd0 comp rx=0 tx=0).stop 2024-02-18T11:54:07.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.376+0000 7fa03292c640 1 -- 172.21.15.12:0/3562465032 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa02c0ff480 msgr2=0x7fa02c101e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:54:07.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.376+0000 7fa03292c640 1 --2- 172.21.15.12:0/3562465032 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa02c0ff480 0x7fa02c101e60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa01c00e9f0 tx=0x7fa01c00eeb0 comp rx=0 tx=0).stop 2024-02-18T11:54:07.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.376+0000 7fa03292c640 1 -- 172.21.15.12:0/3562465032 shutdown_connections 2024-02-18T11:54:07.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.376+0000 7fa03292c640 1 --2- 172.21.15.12:0/3562465032 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7fa0140a2750 0x7fa0140a4c10 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:07.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.376+0000 7fa03292c640 1 --2- 172.21.15.12:0/3562465032 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa02c1024f0 0x7fa02c1004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:07.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.376+0000 7fa03292c640 1 --2- 172.21.15.12:0/3562465032 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa02c102eb0 0x7fa02c1009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:07.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.376+0000 7fa03292c640 1 --2- 172.21.15.12:0/3562465032 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa02c0ff480 0x7fa02c101e60 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:07.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.376+0000 7fa03292c640 1 -- 172.21.15.12:0/3562465032 >> 172.21.15.12:0/3562465032 conn(0x7fa02c0fadb0 msgr2=0x7fa02c1057f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:54:07.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.376+0000 7fa03292c640 1 -- 172.21.15.12:0/3562465032 shutdown_connections 2024-02-18T11:54:07.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.376+0000 7fa03292c640 1 -- 172.21.15.12:0/3562465032 wait complete. 2024-02-18T11:54:07.381 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-02-18T11:54:07.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.496+0000 7f8f8fb25640 1 -- 172.21.15.12:0/706323285 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8f881012b0 msgr2=0x7f8f88101730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:54:07.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.496+0000 7f8f8fb25640 1 --2- 172.21.15.12:0/706323285 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8f881012b0 0x7f8f88101730 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c009900 tx=0x7f8f7c02bc30 comp rx=0 tx=0).stop 2024-02-18T11:54:07.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.496+0000 7f8f8fb25640 1 -- 172.21.15.12:0/706323285 shutdown_connections 2024-02-18T11:54:07.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.496+0000 7f8f8fb25640 1 --2- 172.21.15.12:0/706323285 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f88101c70 0x7f8f88108500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:07.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.496+0000 7f8f8fb25640 1 --2- 172.21.15.12:0/706323285 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8f881012b0 0x7f8f88101730 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:07.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.496+0000 7f8f8fb25640 1 --2- 172.21.15.12:0/706323285 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f881000c0 0x7f8f881004e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:07.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.496+0000 7f8f8fb25640 1 -- 172.21.15.12:0/706323285 >> 172.21.15.12:0/706323285 conn(0x7f8f880fb650 msgr2=0x7f8f880fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:54:07.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.496+0000 7f8f8fb25640 1 -- 172.21.15.12:0/706323285 shutdown_connections 2024-02-18T11:54:07.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.496+0000 7f8f8fb25640 1 -- 172.21.15.12:0/706323285 wait complete. 2024-02-18T11:54:07.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.496+0000 7f8f8fb25640 1 Processor -- start 2024-02-18T11:54:07.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.496+0000 7f8f8fb25640 1 -- start start 2024-02-18T11:54:07.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.497+0000 7f8f8fb25640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f881000c0 0x7f8f88068ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:54:07.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.497+0000 7f8f8fb25640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8f881012b0 0x7f8f8819bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:54:07.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.497+0000 7f8f8fb25640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f88101c70 0x7f8f8819e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:54:07.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.497+0000 7f8f8fb25640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8819ed70 con 0x7f8f881000c0 2024-02-18T11:54:07.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.497+0000 7f8f8fb25640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8819eee0 con 0x7f8f881012b0 2024-02-18T11:54:07.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.497+0000 7f8f8fb25640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8819f0a0 con 0x7f8f88101c70 2024-02-18T11:54:07.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.497+0000 7f8f8d8c1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f881000c0 0x7f8f88068ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:54:07.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.497+0000 7f8f8d0c0640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8f881012b0 0x7f8f8819bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:54:07.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.497+0000 7f8f8d8c1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f881000c0 0x7f8f88068ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42104/0 (socket says 172.21.15.12:42104) 2024-02-18T11:54:07.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.497+0000 7f8f8e0c2640 1 --2- 172.21.15.12:0/2152206934 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f88101c70 0x7f8f8819e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:54:07.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.497+0000 7f8f8d8c1640 1 -- 172.21.15.12:0/2152206934 learned_addr learned my addr 172.21.15.12:0/2152206934 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:54:07.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.497+0000 7f8f8d8c1640 1 -- 172.21.15.12:0/2152206934 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f88101c70 msgr2=0x7f8f8819e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:54:07.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.497+0000 7f8f8d8c1640 1 --2- 172.21.15.12:0/2152206934 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f88101c70 0x7f8f8819e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:07.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.497+0000 7f8f8d8c1640 1 -- 172.21.15.12:0/2152206934 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8f881012b0 msgr2=0x7f8f8819bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:54:07.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.497+0000 7f8f8d8c1640 1 --2- 172.21.15.12:0/2152206934 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8f881012b0 0x7f8f8819bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:07.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.497+0000 7f8f8d8c1640 1 -- 172.21.15.12:0/2152206934 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f78009840 con 0x7f8f881000c0 2024-02-18T11:54:07.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.497+0000 7f8f8d8c1640 1 --2- 172.21.15.12:0/2152206934 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f881000c0 0x7f8f88068ef0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f8f8000d950 tx=0x7f8f8000dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:54:07.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.497+0000 7f8f86ffd640 1 -- 172.21.15.12:0/2152206934 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f80009c30 con 0x7f8f881000c0 2024-02-18T11:54:07.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.497+0000 7f8f86ffd640 1 -- 172.21.15.12:0/2152206934 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8f8000be10 con 0x7f8f881000c0 2024-02-18T11:54:07.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.497+0000 7f8f8fb25640 1 -- 172.21.15.12:0/2152206934 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f7c009560 con 0x7f8f881000c0 2024-02-18T11:54:07.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.497+0000 7f8f86ffd640 1 -- 172.21.15.12:0/2152206934 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f8000f6b0 con 0x7f8f881000c0 2024-02-18T11:54:07.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.497+0000 7f8f8fb25640 1 -- 172.21.15.12:0/2152206934 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f8819f680 con 0x7f8f881000c0 2024-02-18T11:54:07.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.498+0000 7f8f8fb25640 1 -- 172.21.15.12:0/2152206934 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f88107500 con 0x7f8f881000c0 2024-02-18T11:54:07.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.499+0000 7f8f86ffd640 1 -- 172.21.15.12:0/2152206934 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 20) v1 ==== 137245+0+0 (secure 0 0 0) 0x7f8f80010460 con 0x7f8f881000c0 2024-02-18T11:54:07.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.500+0000 7f8f86ffd640 1 --2- 172.21.15.12:0/2152206934 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f8f700a2750 0x7f8f700a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:54:07.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.500+0000 7f8f8d0c0640 1 --2- 172.21.15.12:0/2152206934 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f8f700a2750 0x7f8f700a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:54:07.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.500+0000 7f8f86ffd640 1 -- 172.21.15.12:0/2152206934 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f8f800ce3b0 con 0x7f8f881000c0 2024-02-18T11:54:07.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.500+0000 7f8f8d0c0640 1 --2- 172.21.15.12:0/2152206934 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f8f700a2750 0x7f8f700a4c10 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c009230 tx=0x7f8f7c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:54:07.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.504+0000 7f8f86ffd640 1 -- 172.21.15.12:0/2152206934 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8f80097050 con 0x7f8f881000c0 2024-02-18T11:54:07.602 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:54:07 smithi188 ceph-mon[36925]: pgmap v746: 97 pgs: 97 active+clean; 4.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6.0 MiB/s wr, 132 op/s 2024-02-18T11:54:07.602 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:54:07 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/1652843226' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-18T11:54:07.602 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:54:07 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/1679864819' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-18T11:54:07.603 INFO:journalctl@ceph.mds.c.smithi188.stdout:Feb 18 11:54:07 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-c[67595]: ignoring --setuser ceph since I am not root 2024-02-18T11:54:07.603 INFO:journalctl@ceph.mds.c.smithi188.stdout:Feb 18 11:54:07 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-c[67595]: ignoring --setgroup ceph since I am not root 2024-02-18T11:54:07.603 INFO:journalctl@ceph.mds.c.smithi188.stdout:Feb 18 11:54:07 smithi188 ceph-mds[67599]: -- 172.21.15.188:0/1000766904 <== mon.1 v2:172.21.15.188:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5620d77801a0 con 0x5620d7757000 2024-02-18T11:54:07.603 INFO:journalctl@ceph.mds.c.smithi188.stdout:Feb 18 11:54:07 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-c[67595]: starting mds.c at 2024-02-18T11:54:07.603 INFO:journalctl@ceph.mds.f.smithi188.stdout:Feb 18 11:54:07 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-f[67292]: ignoring --setuser ceph since I am not root 2024-02-18T11:54:07.603 INFO:journalctl@ceph.mds.f.smithi188.stdout:Feb 18 11:54:07 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-f[67292]: ignoring --setgroup ceph since I am not root 2024-02-18T11:54:07.604 INFO:journalctl@ceph.mds.f.smithi188.stdout:Feb 18 11:54:07 smithi188 ceph-mds[67296]: -- 172.21.15.188:0/1961220717 <== mon.1 v2:172.21.15.188:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559fa5b421a0 con 0x559fa5b18800 2024-02-18T11:54:07.604 INFO:journalctl@ceph.mds.f.smithi188.stdout:Feb 18 11:54:07 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-f[67292]: starting mds.f at 2024-02-18T11:54:07.604 INFO:journalctl@ceph.mds.i.smithi188.stdout:Feb 18 11:54:07 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-i[67902]: ignoring --setuser ceph since I am not root 2024-02-18T11:54:07.604 INFO:journalctl@ceph.mds.i.smithi188.stdout:Feb 18 11:54:07 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-i[67902]: ignoring --setgroup ceph since I am not root 2024-02-18T11:54:07.604 INFO:journalctl@ceph.mds.i.smithi188.stdout:Feb 18 11:54:07 smithi188 ceph-mds[67907]: -- 172.21.15.188:0/2446572195 <== mon.1 v2:172.21.15.188:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x557ff82be1a0 con 0x557ff8294c00 2024-02-18T11:54:07.604 INFO:journalctl@ceph.mds.i.smithi188.stdout:Feb 18 11:54:07 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-i[67902]: starting mds.i at 2024-02-18T11:54:07.671 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:54:07 smithi012 ceph-mon[32460]: pgmap v746: 97 pgs: 97 active+clean; 4.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6.0 MiB/s wr, 132 op/s 2024-02-18T11:54:07.671 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:54:07 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/1652843226' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-18T11:54:07.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:54:07 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/1679864819' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-18T11:54:07.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:54:07 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mon-a[32456]: 2024-02-18T11:54:07.365+0000 7f156a0d5700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-18T11:54:07.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:07.726+0000 7f8f8fb25640 1 -- 172.21.15.12:0/2152206934 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f8f8802ce80 con 0x7f8f881000c0 2024-02-18T11:54:07.742 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:54:07 smithi158 ceph-mon[36626]: pgmap v746: 97 pgs: 97 active+clean; 4.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6.0 MiB/s wr, 132 op/s 2024-02-18T11:54:07.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:54:07 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/1652843226' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-18T11:54:07.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:54:07 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/1679864819' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-18T11:54:07.743 INFO:journalctl@ceph.mds.b.smithi158.stdout:Feb 18 11:54:07 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-b[66835]: ignoring --setuser ceph since I am not root 2024-02-18T11:54:07.743 INFO:journalctl@ceph.mds.b.smithi158.stdout:Feb 18 11:54:07 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-b[66835]: ignoring --setgroup ceph since I am not root 2024-02-18T11:54:07.743 INFO:journalctl@ceph.mds.b.smithi158.stdout:Feb 18 11:54:07 smithi158 ceph-mds[66839]: -- 172.21.15.158:0/1842514968 <== mon.2 v2:172.21.15.158:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b33fe641a0 con 0x55b33fe3ac00 2024-02-18T11:54:07.744 INFO:journalctl@ceph.mds.b.smithi158.stdout:Feb 18 11:54:07 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-b[66835]: starting mds.b at 2024-02-18T11:54:07.744 INFO:journalctl@ceph.mds.h.smithi158.stdout:Feb 18 11:54:07 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-h[66515]: ignoring --setuser ceph since I am not root 2024-02-18T11:54:07.744 INFO:journalctl@ceph.mds.h.smithi158.stdout:Feb 18 11:54:07 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-h[66515]: ignoring --setgroup ceph since I am not root 2024-02-18T11:54:07.744 INFO:journalctl@ceph.mds.h.smithi158.stdout:Feb 18 11:54:07 smithi158 ceph-mds[66519]: -- 172.21.15.158:0/3764839593 <== mon.0 v2:172.21.15.12:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5653cea221a0 con 0x5653ce9f8800 2024-02-18T11:54:07.744 INFO:journalctl@ceph.mds.h.smithi158.stdout:Feb 18 11:54:07 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-h[66515]: starting mds.h at 2024-02-18T11:54:07.744 INFO:journalctl@ceph.mds.e.smithi158.stdout:Feb 18 11:54:07 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-e[66203]: ignoring --setuser ceph since I am not root 2024-02-18T11:54:07.744 INFO:journalctl@ceph.mds.e.smithi158.stdout:Feb 18 11:54:07 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-e[66203]: ignoring --setgroup ceph since I am not root 2024-02-18T11:54:07.745 INFO:journalctl@ceph.mds.e.smithi158.stdout:Feb 18 11:54:07 smithi158 ceph-mds[66207]: -- 172.21.15.158:0/2380350281 <== mon.1 v2:172.21.15.188:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x564d523561a0 con 0x564d5232cc00 2024-02-18T11:54:07.745 INFO:journalctl@ceph.mds.e.smithi158.stdout:Feb 18 11:54:07 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-e[66203]: starting mds.e at 2024-02-18T11:54:08.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:08.376+0000 7f8f86ffd640 1 -- 172.21.15.12:0/2152206934 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v23) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8f800170a0 con 0x7f8f881000c0 2024-02-18T11:54:08.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:08.378+0000 7f8f8fb25640 1 -- 172.21.15.12:0/2152206934 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f8f700a2750 msgr2=0x7f8f700a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:54:08.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:08.378+0000 7f8f8fb25640 1 --2- 172.21.15.12:0/2152206934 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f8f700a2750 0x7f8f700a4c10 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c009230 tx=0x7f8f7c005fd0 comp rx=0 tx=0).stop 2024-02-18T11:54:08.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:08.378+0000 7f8f8fb25640 1 -- 172.21.15.12:0/2152206934 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f881000c0 msgr2=0x7f8f88068ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:54:08.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:08.378+0000 7f8f8fb25640 1 --2- 172.21.15.12:0/2152206934 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f881000c0 0x7f8f88068ef0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f8f8000d950 tx=0x7f8f8000dd50 comp rx=0 tx=0).stop 2024-02-18T11:54:08.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:08.378+0000 7f8f8fb25640 1 -- 172.21.15.12:0/2152206934 shutdown_connections 2024-02-18T11:54:08.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:08.378+0000 7f8f8fb25640 1 --2- 172.21.15.12:0/2152206934 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f8f700a2750 0x7f8f700a4c10 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:08.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:08.378+0000 7f8f8fb25640 1 --2- 172.21.15.12:0/2152206934 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f88101c70 0x7f8f8819e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:08.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:08.378+0000 7f8f8fb25640 1 --2- 172.21.15.12:0/2152206934 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8f881012b0 0x7f8f8819bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:08.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:08.378+0000 7f8f8fb25640 1 --2- 172.21.15.12:0/2152206934 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f881000c0 0x7f8f88068ef0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:08.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:08.378+0000 7f8f8fb25640 1 -- 172.21.15.12:0/2152206934 >> 172.21.15.12:0/2152206934 conn(0x7f8f880fb650 msgr2=0x7f8f881046d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:54:08.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:08.378+0000 7f8f8fb25640 1 -- 172.21.15.12:0/2152206934 shutdown_connections 2024-02-18T11:54:08.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:08.378+0000 7f8f8fb25640 1 -- 172.21.15.12:0/2152206934 wait complete. 2024-02-18T11:54:08.383 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-02-18T11:54:08.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:08.498+0000 7f3402a88640 1 -- 172.21.15.12:0/3766568568 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33fc101c70 msgr2=0x7f33fc108500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:54:08.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:08.498+0000 7f3402a88640 1 --2- 172.21.15.12:0/3766568568 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33fc101c70 0x7f33fc108500 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f33f4009ca0 tx=0x7f33f402c0f0 comp rx=0 tx=0).stop 2024-02-18T11:54:08.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:08.498+0000 7f3402a88640 1 -- 172.21.15.12:0/3766568568 shutdown_connections 2024-02-18T11:54:08.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:08.498+0000 7f3402a88640 1 --2- 172.21.15.12:0/3766568568 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33fc101c70 0x7f33fc108500 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:08.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:08.498+0000 7f3402a88640 1 --2- 172.21.15.12:0/3766568568 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f33fc1012b0 0x7f33fc101730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:08.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:08.498+0000 7f3402a88640 1 --2- 172.21.15.12:0/3766568568 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f33fc1000c0 0x7f33fc1004e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:08.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:08.498+0000 7f3402a88640 1 -- 172.21.15.12:0/3766568568 >> 172.21.15.12:0/3766568568 conn(0x7f33fc0fb650 msgr2=0x7f33fc0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:54:08.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:08.498+0000 7f3402a88640 1 -- 172.21.15.12:0/3766568568 shutdown_connections 2024-02-18T11:54:08.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:08.498+0000 7f3402a88640 1 -- 172.21.15.12:0/3766568568 wait complete. 2024-02-18T11:54:08.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:08.498+0000 7f3402a88640 1 Processor -- start 2024-02-18T11:54:08.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:08.498+0000 7f3402a88640 1 -- start start 2024-02-18T11:54:08.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:08.498+0000 7f3402a88640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f33fc1000c0 0x7f33fc1971a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:54:08.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:08.498+0000 7f3402a88640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f33fc1012b0 0x7f33fc1976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:54:08.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:08.498+0000 7f3402a88640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33fc101c70 0x7f33fc19e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:54:08.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:08.498+0000 7f3402a88640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33fc19ec90 con 0x7f33fc101c70 2024-02-18T11:54:08.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:08.498+0000 7f3402a88640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33fc19ee00 con 0x7f33fc1012b0 2024-02-18T11:54:08.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:08.498+0000 7f3402a88640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33fc19efc0 con 0x7f33fc1000c0 2024-02-18T11:54:08.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:08.499+0000 7f3401025640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33fc101c70 0x7f33fc19e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:54:08.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:08.499+0000 7f3401025640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33fc101c70 0x7f33fc19e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42128/0 (socket says 172.21.15.12:42128) 2024-02-18T11:54:08.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:08.499+0000 7f3401025640 1 -- 172.21.15.12:0/2274711504 learned_addr learned my addr 172.21.15.12:0/2274711504 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:54:08.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:08.499+0000 7f3400824640 1 --2- 172.21.15.12:0/2274711504 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f33fc1000c0 0x7f33fc1971a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:54:08.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:08.499+0000 7f33fbfff640 1 --2- 172.21.15.12:0/2274711504 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f33fc1012b0 0x7f33fc1976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:54:08.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:08.499+0000 7f3401025640 1 -- 172.21.15.12:0/2274711504 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f33fc1000c0 msgr2=0x7f33fc1971a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:54:08.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:08.499+0000 7f3401025640 1 --2- 172.21.15.12:0/2274711504 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f33fc1000c0 0x7f33fc1971a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:08.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:08.499+0000 7f3401025640 1 -- 172.21.15.12:0/2274711504 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f33fc1012b0 msgr2=0x7f33fc1976e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:54:08.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:08.499+0000 7f3401025640 1 --2- 172.21.15.12:0/2274711504 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f33fc1012b0 0x7f33fc1976e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:08.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:08.499+0000 7f3401025640 1 -- 172.21.15.12:0/2274711504 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33f4009900 con 0x7f33fc101c70 2024-02-18T11:54:08.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:08.499+0000 7f3400824640 1 --2- 172.21.15.12:0/2274711504 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f33fc1000c0 0x7f33fc1971a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:54:08.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:08.499+0000 7f33fbfff640 1 --2- 172.21.15.12:0/2274711504 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f33fc1012b0 0x7f33fc1976e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:54:08.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:08.499+0000 7f3401025640 1 --2- 172.21.15.12:0/2274711504 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33fc101c70 0x7f33fc19e750 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f33f4037040 tx=0x7f33f4034d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:54:08.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:08.499+0000 7f33f9ffb640 1 -- 172.21.15.12:0/2274711504 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f4032eb0 con 0x7f33fc101c70 2024-02-18T11:54:08.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:08.499+0000 7f33f9ffb640 1 -- 172.21.15.12:0/2274711504 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f33f4003e50 con 0x7f33fc101c70 2024-02-18T11:54:08.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:08.499+0000 7f33f9ffb640 1 -- 172.21.15.12:0/2274711504 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f4040450 con 0x7f33fc101c70 2024-02-18T11:54:08.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:08.499+0000 7f3402a88640 1 -- 172.21.15.12:0/2274711504 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33fc19f240 con 0x7f33fc101c70 2024-02-18T11:54:08.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:08.500+0000 7f3402a88640 1 -- 172.21.15.12:0/2274711504 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33fc19f740 con 0x7f33fc101c70 2024-02-18T11:54:08.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:08.500+0000 7f3402a88640 1 -- 172.21.15.12:0/2274711504 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33fc10ba90 con 0x7f33fc101c70 2024-02-18T11:54:08.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:08.501+0000 7f33f9ffb640 1 -- 172.21.15.12:0/2274711504 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 20) v1 ==== 137245+0+0 (secure 0 0 0) 0x7f33f400a040 con 0x7f33fc101c70 2024-02-18T11:54:08.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:08.502+0000 7f33f9ffb640 1 --2- 172.21.15.12:0/2274711504 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f33ec0a27d0 0x7f33ec0a4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:54:08.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:08.502+0000 7f3400824640 1 --2- 172.21.15.12:0/2274711504 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f33ec0a27d0 0x7f33ec0a4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:54:08.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:08.502+0000 7f33f9ffb640 1 -- 172.21.15.12:0/2274711504 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f33f40f1a40 con 0x7f33fc101c70 2024-02-18T11:54:08.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:08.503+0000 7f3400824640 1 --2- 172.21.15.12:0/2274711504 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f33ec0a27d0 0x7f33ec0a4c90 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f33fc101110 tx=0x7f33f000a5e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:54:08.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:08.506+0000 7f33f9ffb640 1 -- 172.21.15.12:0/2274711504 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f33f40ba880 con 0x7f33fc101c70 2024-02-18T11:54:08.671 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:54:08 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/3562465032' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-02-18T11:54:08.671 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:54:08 smithi012 ceph-mon[32460]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-02-18T11:54:08.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:54:08 smithi012 ceph-mon[32460]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-02-18T11:54:08.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:54:08 smithi012 ceph-mon[32460]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-18T11:54:08.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:54:08 smithi012 ceph-mon[32460]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-02-18T11:54:08.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:54:08 smithi012 ceph-mon[32460]: osdmap e90: 12 total, 12 up, 12 in 2024-02-18T11:54:08.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:54:08 smithi012 ceph-mon[32460]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-02-18T11:54:08.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:54:08 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/2152206934' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-02-18T11:54:08.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:08.689+0000 7f3402a88640 1 -- 172.21.15.12:0/2274711504 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f33fc10bc20 con 0x7f33fc101c70 2024-02-18T11:54:08.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:54:08 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/3562465032' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-02-18T11:54:08.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:54:08 smithi158 ceph-mon[36626]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-02-18T11:54:08.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:54:08 smithi158 ceph-mon[36626]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-02-18T11:54:08.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:54:08 smithi158 ceph-mon[36626]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-18T11:54:08.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:54:08 smithi158 ceph-mon[36626]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-02-18T11:54:08.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:54:08 smithi158 ceph-mon[36626]: osdmap e90: 12 total, 12 up, 12 in 2024-02-18T11:54:08.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:54:08 smithi158 ceph-mon[36626]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-02-18T11:54:08.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:54:08 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/2152206934' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-02-18T11:54:08.852 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:54:08 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/3562465032' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-02-18T11:54:08.852 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:54:08 smithi188 ceph-mon[36925]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-02-18T11:54:08.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:54:08 smithi188 ceph-mon[36925]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-02-18T11:54:08.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:54:08 smithi188 ceph-mon[36925]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-18T11:54:08.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:54:08 smithi188 ceph-mon[36925]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-02-18T11:54:08.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:54:08 smithi188 ceph-mon[36925]: osdmap e90: 12 total, 12 up, 12 in 2024-02-18T11:54:08.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:54:08 smithi188 ceph-mon[36925]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-02-18T11:54:08.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:54:08 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/2152206934' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-02-18T11:54:09.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:09.380+0000 7f33f9ffb640 1 -- 172.21.15.12:0/2274711504 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v91) v1 ==== 181+0+0 (secure 0 0 0) 0x7f33f40bdf40 con 0x7f33fc101c70 2024-02-18T11:54:09.373 INFO:teuthology.orchestra.run.smithi012.stderr:pool 'cephfs_metadata' removed 2024-02-18T11:54:09.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:09.382+0000 7f3402a88640 1 -- 172.21.15.12:0/2274711504 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f33ec0a27d0 msgr2=0x7f33ec0a4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:54:09.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:09.382+0000 7f3402a88640 1 --2- 172.21.15.12:0/2274711504 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f33ec0a27d0 0x7f33ec0a4c90 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f33fc101110 tx=0x7f33f000a5e0 comp rx=0 tx=0).stop 2024-02-18T11:54:09.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:09.382+0000 7f3402a88640 1 -- 172.21.15.12:0/2274711504 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33fc101c70 msgr2=0x7f33fc19e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:54:09.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:09.382+0000 7f3402a88640 1 --2- 172.21.15.12:0/2274711504 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33fc101c70 0x7f33fc19e750 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f33f4037040 tx=0x7f33f4034d40 comp rx=0 tx=0).stop 2024-02-18T11:54:09.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:09.382+0000 7f3402a88640 1 -- 172.21.15.12:0/2274711504 shutdown_connections 2024-02-18T11:54:09.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:09.382+0000 7f3402a88640 1 --2- 172.21.15.12:0/2274711504 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f33ec0a27d0 0x7f33ec0a4c90 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:09.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:09.382+0000 7f3402a88640 1 --2- 172.21.15.12:0/2274711504 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33fc101c70 0x7f33fc19e750 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:09.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:09.382+0000 7f3402a88640 1 --2- 172.21.15.12:0/2274711504 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f33fc1012b0 0x7f33fc1976e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:09.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:09.382+0000 7f3402a88640 1 --2- 172.21.15.12:0/2274711504 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f33fc1000c0 0x7f33fc1971a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:09.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:09.382+0000 7f3402a88640 1 -- 172.21.15.12:0/2274711504 >> 172.21.15.12:0/2274711504 conn(0x7f33fc0fb650 msgr2=0x7f33fc0fd010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:54:09.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:09.382+0000 7f3402a88640 1 -- 172.21.15.12:0/2274711504 shutdown_connections 2024-02-18T11:54:09.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:09.382+0000 7f3402a88640 1 -- 172.21.15.12:0/2274711504 wait complete. 2024-02-18T11:54:09.389 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-02-18T11:54:09.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:09.508+0000 7f48d770c640 1 -- 172.21.15.12:0/356009605 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f48d01000c0 msgr2=0x7f48d01004e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:54:09.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:09.508+0000 7f48d770c640 1 --2- 172.21.15.12:0/356009605 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f48d01000c0 0x7f48d01004e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f48cc009900 tx=0x7f48cc02bc30 comp rx=0 tx=0).stop 2024-02-18T11:54:09.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:09.508+0000 7f48d770c640 1 -- 172.21.15.12:0/356009605 shutdown_connections 2024-02-18T11:54:09.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:09.508+0000 7f48d770c640 1 --2- 172.21.15.12:0/356009605 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48d01012b0 0x7f48d0101730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:09.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:09.508+0000 7f48d770c640 1 --2- 172.21.15.12:0/356009605 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f48d0101c70 0x7f48d0108500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:09.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:09.508+0000 7f48d770c640 1 --2- 172.21.15.12:0/356009605 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f48d01000c0 0x7f48d01004e0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:09.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:09.508+0000 7f48d770c640 1 -- 172.21.15.12:0/356009605 >> 172.21.15.12:0/356009605 conn(0x7f48d00fb650 msgr2=0x7f48d00fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:54:09.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:09.508+0000 7f48d770c640 1 -- 172.21.15.12:0/356009605 shutdown_connections 2024-02-18T11:54:09.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:09.508+0000 7f48d770c640 1 -- 172.21.15.12:0/356009605 wait complete. 2024-02-18T11:54:09.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:09.508+0000 7f48d770c640 1 Processor -- start 2024-02-18T11:54:09.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:09.509+0000 7f48d770c640 1 -- start start 2024-02-18T11:54:09.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:09.509+0000 7f48d770c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f48d01000c0 0x7f48d01972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:54:09.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:09.509+0000 7f48d770c640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f48d01012b0 0x7f48d01977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:54:09.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:09.509+0000 7f48d770c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48d0101c70 0x7f48d019e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:54:09.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:09.509+0000 7f48d770c640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d019eda0 con 0x7f48d01000c0 2024-02-18T11:54:09.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:09.509+0000 7f48d770c640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d019ef10 con 0x7f48d01012b0 2024-02-18T11:54:09.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:09.509+0000 7f48d770c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d019f0d0 con 0x7f48d0101c70 2024-02-18T11:54:09.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:09.509+0000 7f48d54a8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f48d01000c0 0x7f48d01972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:54:09.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:09.509+0000 7f48d54a8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f48d01000c0 0x7f48d01972b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42146/0 (socket says 172.21.15.12:42146) 2024-02-18T11:54:09.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:09.509+0000 7f48d54a8640 1 -- 172.21.15.12:0/2553038485 learned_addr learned my addr 172.21.15.12:0/2553038485 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-02-18T11:54:09.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:09.509+0000 7f48d4ca7640 1 --2- 172.21.15.12:0/2553038485 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f48d01012b0 0x7f48d01977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:54:09.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:09.509+0000 7f48d5ca9640 1 --2- 172.21.15.12:0/2553038485 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48d0101c70 0x7f48d019e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:54:09.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:09.509+0000 7f48d54a8640 1 -- 172.21.15.12:0/2553038485 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48d0101c70 msgr2=0x7f48d019e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:54:09.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:09.509+0000 7f48d54a8640 1 --2- 172.21.15.12:0/2553038485 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48d0101c70 0x7f48d019e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:09.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:09.509+0000 7f48d54a8640 1 -- 172.21.15.12:0/2553038485 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f48d01012b0 msgr2=0x7f48d01977f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:54:09.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:09.509+0000 7f48d54a8640 1 --2- 172.21.15.12:0/2553038485 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f48d01012b0 0x7f48d01977f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:09.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:09.509+0000 7f48d54a8640 1 -- 172.21.15.12:0/2553038485 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48cc009560 con 0x7f48d01000c0 2024-02-18T11:54:09.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:09.509+0000 7f48d54a8640 1 --2- 172.21.15.12:0/2553038485 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f48d01000c0 0x7f48d01972b0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f48cc036040 tx=0x7f48cc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:54:09.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:09.509+0000 7f48c67fc640 1 -- 172.21.15.12:0/2553038485 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48cc0044e0 con 0x7f48d01000c0 2024-02-18T11:54:09.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:09.509+0000 7f48c67fc640 1 -- 172.21.15.12:0/2553038485 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f48cc0054e0 con 0x7f48d01000c0 2024-02-18T11:54:09.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:09.509+0000 7f48d770c640 1 -- 172.21.15.12:0/2553038485 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48d019f350 con 0x7f48d01000c0 2024-02-18T11:54:09.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:09.509+0000 7f48c67fc640 1 -- 172.21.15.12:0/2553038485 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48cc03e450 con 0x7f48d01000c0 2024-02-18T11:54:09.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:09.510+0000 7f48d770c640 1 -- 172.21.15.12:0/2553038485 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48d019f8e0 con 0x7f48d01000c0 2024-02-18T11:54:09.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:09.510+0000 7f48d770c640 1 -- 172.21.15.12:0/2553038485 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48d010bb10 con 0x7f48d01000c0 2024-02-18T11:54:09.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:09.511+0000 7f48c67fc640 1 -- 172.21.15.12:0/2553038485 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 20) v1 ==== 137245+0+0 (secure 0 0 0) 0x7f48cc003710 con 0x7f48d01000c0 2024-02-18T11:54:09.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:09.512+0000 7f48c67fc640 1 --2- 172.21.15.12:0/2553038485 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f48bc0a2750 0x7f48bc0a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:54:09.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:09.512+0000 7f48d4ca7640 1 --2- 172.21.15.12:0/2553038485 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f48bc0a2750 0x7f48bc0a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:54:09.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:09.512+0000 7f48c67fc640 1 -- 172.21.15.12:0/2553038485 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 7973+0+0 (secure 0 0 0) 0x7f48cc0f0ea0 con 0x7f48d01000c0 2024-02-18T11:54:09.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:09.513+0000 7f48d4ca7640 1 --2- 172.21.15.12:0/2553038485 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f48bc0a2750 0x7f48bc0a4c10 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f48c0005e10 tx=0x7f48c0005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:54:09.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:09.516+0000 7f48c67fc640 1 -- 172.21.15.12:0/2553038485 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f48cc0b9d80 con 0x7f48d01000c0 2024-02-18T11:54:09.671 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:54:09 smithi012 ceph-mon[32460]: pgmap v748: 97 pgs: 97 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 62 MiB/s wr, 576 op/s 2024-02-18T11:54:09.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:54:09 smithi012 ceph-mon[32460]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-02-18T11:54:09.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:54:09 smithi012 ceph-mon[32460]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-18T11:54:09.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:54:09 smithi012 ceph-mon[32460]: Cluster is now healthy 2024-02-18T11:54:09.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:54:09 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/2152206934' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-02-18T11:54:09.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:54:09 smithi012 ceph-mon[32460]: mds.? [v2:172.21.15.188:6834/249487569,v1:172.21.15.188:6835/249487569] up:boot 2024-02-18T11:54:09.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:54:09 smithi012 ceph-mon[32460]: mds.? [v2:172.21.15.188:6837/399184081,v1:172.21.15.188:6839/399184081] up:boot 2024-02-18T11:54:09.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:54:09 smithi012 ceph-mon[32460]: mds.? [v2:172.21.15.188:6836/3434121333,v1:172.21.15.188:6838/3434121333] up:boot 2024-02-18T11:54:09.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:54:09 smithi012 ceph-mon[32460]: mds.? [v2:172.21.15.158:6834/931080983,v1:172.21.15.158:6835/931080983] up:boot 2024-02-18T11:54:09.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:54:09 smithi012 ceph-mon[32460]: mds.? [v2:172.21.15.158:6838/3562592644,v1:172.21.15.158:6839/3562592644] up:boot 2024-02-18T11:54:09.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:54:09 smithi012 ceph-mon[32460]: mds.? [v2:172.21.15.158:6836/3673767939,v1:172.21.15.158:6837/3673767939] up:boot 2024-02-18T11:54:09.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:54:09 smithi012 ceph-mon[32460]: fsmap 12 up:standby 2024-02-18T11:54:09.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:54:09 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-18T11:54:09.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:54:09 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-18T11:54:09.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:54:09 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-18T11:54:09.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:54:09 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-18T11:54:09.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:54:09 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-02-18T11:54:09.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:54:09 smithi012 ceph-mon[32460]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-18T11:54:09.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:54:09 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/2274711504' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-18T11:54:09.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:09.696+0000 7f48d770c640 1 -- 172.21.15.12:0/2553038485 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f48d010bca0 con 0x7f48d01000c0 2024-02-18T11:54:09.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:54:09 smithi158 ceph-mon[36626]: pgmap v748: 97 pgs: 97 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 62 MiB/s wr, 576 op/s 2024-02-18T11:54:09.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:54:09 smithi158 ceph-mon[36626]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-02-18T11:54:09.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:54:09 smithi158 ceph-mon[36626]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-18T11:54:09.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:54:09 smithi158 ceph-mon[36626]: Cluster is now healthy 2024-02-18T11:54:09.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:54:09 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/2152206934' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-02-18T11:54:09.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:54:09 smithi158 ceph-mon[36626]: mds.? [v2:172.21.15.188:6834/249487569,v1:172.21.15.188:6835/249487569] up:boot 2024-02-18T11:54:09.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:54:09 smithi158 ceph-mon[36626]: mds.? [v2:172.21.15.188:6837/399184081,v1:172.21.15.188:6839/399184081] up:boot 2024-02-18T11:54:09.744 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:54:09 smithi158 ceph-mon[36626]: mds.? [v2:172.21.15.188:6836/3434121333,v1:172.21.15.188:6838/3434121333] up:boot 2024-02-18T11:54:09.744 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:54:09 smithi158 ceph-mon[36626]: mds.? [v2:172.21.15.158:6834/931080983,v1:172.21.15.158:6835/931080983] up:boot 2024-02-18T11:54:09.744 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:54:09 smithi158 ceph-mon[36626]: mds.? [v2:172.21.15.158:6838/3562592644,v1:172.21.15.158:6839/3562592644] up:boot 2024-02-18T11:54:09.744 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:54:09 smithi158 ceph-mon[36626]: mds.? [v2:172.21.15.158:6836/3673767939,v1:172.21.15.158:6837/3673767939] up:boot 2024-02-18T11:54:09.744 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:54:09 smithi158 ceph-mon[36626]: fsmap 12 up:standby 2024-02-18T11:54:09.744 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:54:09 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-18T11:54:09.744 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:54:09 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-18T11:54:09.744 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:54:09 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-18T11:54:09.744 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:54:09 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-18T11:54:09.744 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:54:09 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-02-18T11:54:09.744 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:54:09 smithi158 ceph-mon[36626]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-18T11:54:09.744 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:54:09 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/2274711504' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-18T11:54:09.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:54:09 smithi188 ceph-mon[36925]: pgmap v748: 97 pgs: 97 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 62 MiB/s wr, 576 op/s 2024-02-18T11:54:09.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:54:09 smithi188 ceph-mon[36925]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-02-18T11:54:09.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:54:09 smithi188 ceph-mon[36925]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-18T11:54:09.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:54:09 smithi188 ceph-mon[36925]: Cluster is now healthy 2024-02-18T11:54:09.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:54:09 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/2152206934' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-02-18T11:54:09.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:54:09 smithi188 ceph-mon[36925]: mds.? [v2:172.21.15.188:6834/249487569,v1:172.21.15.188:6835/249487569] up:boot 2024-02-18T11:54:09.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:54:09 smithi188 ceph-mon[36925]: mds.? [v2:172.21.15.188:6837/399184081,v1:172.21.15.188:6839/399184081] up:boot 2024-02-18T11:54:09.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:54:09 smithi188 ceph-mon[36925]: mds.? [v2:172.21.15.188:6836/3434121333,v1:172.21.15.188:6838/3434121333] up:boot 2024-02-18T11:54:09.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:54:09 smithi188 ceph-mon[36925]: mds.? [v2:172.21.15.158:6834/931080983,v1:172.21.15.158:6835/931080983] up:boot 2024-02-18T11:54:09.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:54:09 smithi188 ceph-mon[36925]: mds.? [v2:172.21.15.158:6838/3562592644,v1:172.21.15.158:6839/3562592644] up:boot 2024-02-18T11:54:09.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:54:09 smithi188 ceph-mon[36925]: mds.? [v2:172.21.15.158:6836/3673767939,v1:172.21.15.158:6837/3673767939] up:boot 2024-02-18T11:54:09.853 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:54:09 smithi188 ceph-mon[36925]: fsmap 12 up:standby 2024-02-18T11:54:09.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:54:09 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-18T11:54:09.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:54:09 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-18T11:54:09.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:54:09 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-18T11:54:09.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:54:09 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-18T11:54:09.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:54:09 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-02-18T11:54:09.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:54:09 smithi188 ceph-mon[36925]: from='mgr.14152 172.21.15.12:0/4035763835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-18T11:54:09.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:54:09 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/2274711504' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-18T11:54:10.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:10.387+0000 7f48c67fc640 1 -- 172.21.15.12:0/2553038485 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v92) v1 ==== 169+0+0 (secure 0 0 0) 0x7f48cc0bd440 con 0x7f48d01000c0 2024-02-18T11:54:10.380 INFO:teuthology.orchestra.run.smithi012.stderr:pool 'cephfs_data' removed 2024-02-18T11:54:10.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:10.391+0000 7f48d770c640 1 -- 172.21.15.12:0/2553038485 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f48bc0a2750 msgr2=0x7f48bc0a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:54:10.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:10.391+0000 7f48d770c640 1 --2- 172.21.15.12:0/2553038485 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f48bc0a2750 0x7f48bc0a4c10 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f48c0005e10 tx=0x7f48c0005da0 comp rx=0 tx=0).stop 2024-02-18T11:54:10.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:10.391+0000 7f48d770c640 1 -- 172.21.15.12:0/2553038485 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f48d01000c0 msgr2=0x7f48d01972b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:54:10.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:10.391+0000 7f48d770c640 1 --2- 172.21.15.12:0/2553038485 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f48d01000c0 0x7f48d01972b0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f48cc036040 tx=0x7f48cc0042f0 comp rx=0 tx=0).stop 2024-02-18T11:54:10.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:10.391+0000 7f48d770c640 1 -- 172.21.15.12:0/2553038485 shutdown_connections 2024-02-18T11:54:10.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:10.391+0000 7f48d770c640 1 --2- 172.21.15.12:0/2553038485 >> [v2:172.21.15.12:6800/1153736685,v1:172.21.15.12:6801/1153736685] conn(0x7f48bc0a2750 0x7f48bc0a4c10 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:10.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:10.391+0000 7f48d770c640 1 --2- 172.21.15.12:0/2553038485 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48d0101c70 0x7f48d019e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:10.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:10.391+0000 7f48d770c640 1 --2- 172.21.15.12:0/2553038485 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f48d01012b0 0x7f48d01977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:10.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:10.391+0000 7f48d770c640 1 --2- 172.21.15.12:0/2553038485 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f48d01000c0 0x7f48d01972b0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:54:10.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:10.391+0000 7f48d770c640 1 -- 172.21.15.12:0/2553038485 >> 172.21.15.12:0/2553038485 conn(0x7f48d00fb650 msgr2=0x7f48d00fd0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:54:10.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:10.391+0000 7f48d770c640 1 -- 172.21.15.12:0/2553038485 shutdown_connections 2024-02-18T11:54:10.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-02-18T11:54:10.391+0000 7f48d770c640 1 -- 172.21.15.12:0/2553038485 wait complete. 2024-02-18T11:54:10.403 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-18T11:54:10.434 DEBUG:teuthology.orchestra.run.smithi158:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-18T11:54:10.467 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-18T11:54:10.498 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-02-18T11:54:10.498 DEBUG:teuthology.orchestra.run.smithi012:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-18T11:54:10.513 DEBUG:teuthology.orchestra.run.smithi158:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-18T11:54:10.539 DEBUG:teuthology.orchestra.run.smithi188:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-18T11:54:10.554 INFO:tasks.cephadm:Stopping all daemons... 2024-02-18T11:54:10.554 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-02-18T11:54:10.554 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@mon.a 2024-02-18T11:54:10.671 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:54:10 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/2274711504' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-02-18T11:54:10.671 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:54:10 smithi012 ceph-mon[32460]: osdmap e91: 12 total, 12 up, 12 in 2024-02-18T11:54:10.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:54:10 smithi012 ceph-mon[32460]: from='client.? 172.21.15.12:0/2553038485' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-18T11:54:10.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:54:10 smithi012 systemd[1]: Stopping Ceph mon.a for 0c8bb0de-ce51-11ee-95ba-87774f69a715... 2024-02-18T11:54:10.742 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:54:10 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/2274711504' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-02-18T11:54:10.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:54:10 smithi158 ceph-mon[36626]: osdmap e91: 12 total, 12 up, 12 in 2024-02-18T11:54:10.743 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:54:10 smithi158 ceph-mon[36626]: from='client.? 172.21.15.12:0/2553038485' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-18T11:54:10.852 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:54:10 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/2274711504' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-02-18T11:54:10.852 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:54:10 smithi188 ceph-mon[36925]: osdmap e91: 12 total, 12 up, 12 in 2024-02-18T11:54:10.852 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:54:10 smithi188 ceph-mon[36925]: from='client.? 172.21.15.12:0/2553038485' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-18T11:54:11.172 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:54:10 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mon-a[32456]: 2024-02-18T11:54:10.851+0000 7f156f8e0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-18T11:54:11.172 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:54:10 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mon-a[32456]: 2024-02-18T11:54:10.851+0000 7f156f8e0700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-02-18T11:54:11.172 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:54:10 smithi012 podman[87070]: 2024-02-18 11:54:10.872808252 +0000 UTC m=+0.118816047 container died 74f2d580255e5a641a85532de12b838833c585e2189c06128eeb7fad38e2d61f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mon-a, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-02-18T11:54:11.603 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:54:11 smithi012 podman[87070]: 2024-02-18 11:54:11.326967104 +0000 UTC m=+0.572974899 container cleanup 74f2d580255e5a641a85532de12b838833c585e2189c06128eeb7fad38e2d61f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mon-a, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-02-18T11:54:11.604 INFO:journalctl@ceph.mon.a.smithi012.stdout:Feb 18 11:54:11 smithi012 bash[87070]: ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mon-a 2024-02-18T11:54:11.727 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@mon.a.service' 2024-02-18T11:54:11.767 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-18T11:54:11.767 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-02-18T11:54:11.767 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-02-18T11:54:11.768 DEBUG:teuthology.orchestra.run.smithi158:> sudo systemctl stop ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@mon.b 2024-02-18T11:54:12.134 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:54:11 smithi158 systemd[1]: Stopping Ceph mon.b for 0c8bb0de-ce51-11ee-95ba-87774f69a715... 2024-02-18T11:54:12.399 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:54:12 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mon-b[36622]: 2024-02-18T11:54:12.141+0000 7fee16ff1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-18T11:54:12.400 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:54:12 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mon-b[36622]: 2024-02-18T11:54:12.141+0000 7fee16ff1700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-02-18T11:54:12.694 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:54:12 smithi158 podman[71966]: 2024-02-18 11:54:12.407804897 +0000 UTC m=+0.393696745 container died bceb8ce2e0f307e0a94029737af1fc45a62ec71d8c9a2697b7c45ef81ef282ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mon-b, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240217.015652-1c920af) 2024-02-18T11:54:12.993 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:54:12 smithi158 podman[71966]: 2024-02-18 11:54:12.784320783 +0000 UTC m=+0.770212655 container cleanup bceb8ce2e0f307e0a94029737af1fc45a62ec71d8c9a2697b7c45ef81ef282ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mon-b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True) 2024-02-18T11:54:12.993 INFO:journalctl@ceph.mon.b.smithi158.stdout:Feb 18 11:54:12 smithi158 bash[71966]: ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mon-b 2024-02-18T11:54:13.244 DEBUG:teuthology.orchestra.run.smithi158:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@mon.b.service' 2024-02-18T11:54:13.304 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-18T11:54:13.305 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-02-18T11:54:13.305 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-02-18T11:54:13.305 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@mon.c 2024-02-18T11:54:13.602 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:54:13 smithi188 systemd[1]: Stopping Ceph mon.c for 0c8bb0de-ce51-11ee-95ba-87774f69a715... 2024-02-18T11:54:14.102 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:54:13 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mon-c[36913]: 2024-02-18T11:54:13.635+0000 7f91d8bde700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-18T11:54:14.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:54:13 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mon-c[36913]: 2024-02-18T11:54:13.635+0000 7f91d8bde700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-02-18T11:54:14.103 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:54:13 smithi188 podman[72613]: 2024-02-18 11:54:13.859861053 +0000 UTC m=+0.353418839 container died 7875018d620ce6f669d4e4488106f1078a142ed107f42cda925fa5c6fdc76223 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mon-c, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-02-18T11:54:14.495 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:54:14 smithi188 podman[72613]: 2024-02-18 11:54:14.24349614 +0000 UTC m=+0.737053911 container cleanup 7875018d620ce6f669d4e4488106f1078a142ed107f42cda925fa5c6fdc76223 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mon-c, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux ) 2024-02-18T11:54:14.495 INFO:journalctl@ceph.mon.c.smithi188.stdout:Feb 18 11:54:14 smithi188 bash[72613]: ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mon-c 2024-02-18T11:54:14.649 DEBUG:teuthology.orchestra.run.smithi188:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@mon.c.service' 2024-02-18T11:54:14.700 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-18T11:54:14.700 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-02-18T11:54:14.701 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-02-18T11:54:14.701 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@mgr.x 2024-02-18T11:54:15.007 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:54:14 smithi012 systemd[1]: Stopping Ceph mgr.x for 0c8bb0de-ce51-11ee-95ba-87774f69a715... 2024-02-18T11:54:15.307 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:54:15 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x[32692]: 2024-02-18T11:54:15.014+0000 7fb3dc873700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-18T11:54:15.307 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:54:15 smithi012 podman[87193]: 2024-02-18 11:54:15.045477568 +0000 UTC m=+0.135422043 container died d9409129c6e15042d5cc90ddd4a9178a7b2f1278cace3b6e7b20c6e80622f50c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, ceph=True, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-18T11:54:15.671 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:54:15 smithi012 podman[87193]: 2024-02-18 11:54:15.390304473 +0000 UTC m=+0.480248942 container cleanup d9409129c6e15042d5cc90ddd4a9178a7b2f1278cace3b6e7b20c6e80622f50c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c920afa8316ea21877f3186850254947531ea22, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240214) 2024-02-18T11:54:15.672 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Feb 18 11:54:15 smithi012 bash[87193]: ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-x 2024-02-18T11:54:15.854 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@mgr.x.service' 2024-02-18T11:54:15.890 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-18T11:54:15.890 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-02-18T11:54:15.890 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-02-18T11:54:15.891 DEBUG:teuthology.orchestra.run.smithi158:> sudo systemctl stop ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@mgr.y 2024-02-18T11:54:16.202 INFO:journalctl@ceph.mgr.y.smithi158.stdout:Feb 18 11:54:15 smithi158 systemd[1]: Stopping Ceph mgr.y for 0c8bb0de-ce51-11ee-95ba-87774f69a715... 2024-02-18T11:54:16.493 INFO:journalctl@ceph.mgr.y.smithi158.stdout:Feb 18 11:54:16 smithi158 podman[72088]: 2024-02-18 11:54:16.209550204 +0000 UTC m=+0.123832977 container died 7fc0f5c3029b2bd800258c2143e4d71ec90f08a71c72491e0b15cc8957462650 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, ceph=True) 2024-02-18T11:54:16.941 INFO:journalctl@ceph.mgr.y.smithi158.stdout:Feb 18 11:54:16 smithi158 podman[72088]: 2024-02-18 11:54:16.76428337 +0000 UTC m=+0.678566145 container cleanup 7fc0f5c3029b2bd800258c2143e4d71ec90f08a71c72491e0b15cc8957462650 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-y, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240217.015652-1c920af) 2024-02-18T11:54:16.941 INFO:journalctl@ceph.mgr.y.smithi158.stdout:Feb 18 11:54:16 smithi158 bash[72088]: ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-y 2024-02-18T11:54:16.944 INFO:journalctl@ceph.mgr.y.smithi158.stdout:Feb 18 11:54:16 smithi158 podman[72102]: 2024-02-18 11:54:16.949082105 +0000 UTC m=+0.736446678 container remove 7fc0f5c3029b2bd800258c2143e4d71ec90f08a71c72491e0b15cc8957462650 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-y, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-02-18T11:54:16.944 INFO:journalctl@ceph.mgr.y.smithi158.stdout:Feb 18 11:54:16 smithi158 systemd[1]: ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-02-18T11:54:17.173 DEBUG:teuthology.orchestra.run.smithi158:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@mgr.y.service' 2024-02-18T11:54:17.196 INFO:journalctl@ceph.mgr.y.smithi158.stdout:Feb 18 11:54:17 smithi158 systemd[1]: ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@mgr.y.service: Failed with result 'exit-code'. 2024-02-18T11:54:17.196 INFO:journalctl@ceph.mgr.y.smithi158.stdout:Feb 18 11:54:17 smithi158 systemd[1]: Stopped Ceph mgr.y for 0c8bb0de-ce51-11ee-95ba-87774f69a715. 2024-02-18T11:54:17.197 INFO:journalctl@ceph.mgr.y.smithi158.stdout:Feb 18 11:54:17 smithi158 systemd[1]: ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@mgr.y.service: Consumed 14.960s CPU time. 2024-02-18T11:54:17.210 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-18T11:54:17.210 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-02-18T11:54:17.210 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-02-18T11:54:17.211 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@mgr.z 2024-02-18T11:54:17.536 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Feb 18 11:54:17 smithi188 systemd[1]: Stopping Ceph mgr.z for 0c8bb0de-ce51-11ee-95ba-87774f69a715... 2024-02-18T11:54:17.801 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Feb 18 11:54:17 smithi188 podman[72736]: 2024-02-18 11:54:17.543752838 +0000 UTC m=+0.138229920 container died 9be720b49c12510eca4c7f89186401828e2a11c54adfb1ba092b0f421af76b8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-z, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-02-18T11:54:18.102 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Feb 18 11:54:17 smithi188 podman[72736]: 2024-02-18 11:54:17.889310776 +0000 UTC m=+0.483787847 container cleanup 9be720b49c12510eca4c7f89186401828e2a11c54adfb1ba092b0f421af76b8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-z, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-02-18T11:54:18.103 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Feb 18 11:54:17 smithi188 bash[72736]: ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mgr-z 2024-02-18T11:54:18.354 DEBUG:teuthology.orchestra.run.smithi188:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@mgr.z.service' 2024-02-18T11:54:18.391 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-18T11:54:18.391 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-02-18T11:54:18.391 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-02-18T11:54:18.392 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@osd.0 2024-02-18T11:54:18.671 INFO:journalctl@ceph.osd.0.smithi012.stdout:Feb 18 11:54:18 smithi012 systemd[1]: Stopping Ceph osd.0 for 0c8bb0de-ce51-11ee-95ba-87774f69a715... 2024-02-18T11:54:19.171 INFO:journalctl@ceph.osd.0.smithi012.stdout:Feb 18 11:54:18 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-0[44875]: 2024-02-18T11:54:18.685+0000 7f86db3f5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-18T11:54:19.172 INFO:journalctl@ceph.osd.0.smithi012.stdout:Feb 18 11:54:18 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-0[44875]: 2024-02-18T11:54:18.685+0000 7f86db3f5700 -1 osd.0 92 *** Got signal Terminated *** 2024-02-18T11:54:19.172 INFO:journalctl@ceph.osd.0.smithi012.stdout:Feb 18 11:54:18 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-0[44875]: 2024-02-18T11:54:18.685+0000 7f86db3f5700 -1 osd.0 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-18T11:54:24.412 INFO:journalctl@ceph.osd.0.smithi012.stdout:Feb 18 11:54:24 smithi012 podman[87316]: 2024-02-18 11:54:24.163663665 +0000 UTC m=+5.569875351 container died a7a69fd149bbcef0c803ddf2bbe82f8ea217afb85af40d963e4c58635df40c6a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-02-18T11:54:24.671 INFO:journalctl@ceph.osd.0.smithi012.stdout:Feb 18 11:54:24 smithi012 podman[87316]: 2024-02-18 11:54:24.502025606 +0000 UTC m=+5.908237300 container cleanup a7a69fd149bbcef0c803ddf2bbe82f8ea217afb85af40d963e4c58635df40c6a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True) 2024-02-18T11:54:24.671 INFO:journalctl@ceph.osd.0.smithi012.stdout:Feb 18 11:54:24 smithi012 bash[87316]: ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-0 2024-02-18T11:54:24.921 INFO:journalctl@ceph.osd.0.smithi012.stdout:Feb 18 11:54:24 smithi012 podman[87335]: 2024-02-18 11:54:24.678177569 +0000 UTC m=+0.518980888 container remove a7a69fd149bbcef0c803ddf2bbe82f8ea217afb85af40d963e4c58635df40c6a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-0, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, org.label-schema.build-date=20240214, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-18T11:54:25.408 INFO:journalctl@ceph.osd.0.smithi012.stdout:Feb 18 11:54:25 smithi012 podman[87425]: 2024-02-18 11:54:25.11224333 +0000 UTC m=+0.107981912 container create 8e5ddf7399e0d9707a9947c886c85644d94db9edff8f9bdd4e59083eb841153d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240214, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, org.label-schema.vendor=CentOS) 2024-02-18T11:54:25.408 INFO:journalctl@ceph.osd.0.smithi012.stdout:Feb 18 11:54:25 smithi012 podman[87425]: 2024-02-18 11:54:25.022564891 +0000 UTC m=+0.018303448 image pull 53c33ee3bf62dc411b650f86eedbbee7698efb1001b4c063271dcd8d65b41165 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1 2024-02-18T11:54:25.671 INFO:journalctl@ceph.osd.0.smithi012.stdout:Feb 18 11:54:25 smithi012 podman[87425]: 2024-02-18 11:54:25.414631975 +0000 UTC m=+0.410370516 container init 8e5ddf7399e0d9707a9947c886c85644d94db9edff8f9bdd4e59083eb841153d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-02-18T11:54:25.672 INFO:journalctl@ceph.osd.0.smithi012.stdout:Feb 18 11:54:25 smithi012 podman[87425]: 2024-02-18 11:54:25.419000231 +0000 UTC m=+0.414738778 container start 8e5ddf7399e0d9707a9947c886c85644d94db9edff8f9bdd4e59083eb841153d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-0-deactivate, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-02-18T11:54:25.672 INFO:journalctl@ceph.osd.0.smithi012.stdout:Feb 18 11:54:25 smithi012 podman[87425]: 2024-02-18 11:54:25.481446537 +0000 UTC m=+0.477185086 container attach 8e5ddf7399e0d9707a9947c886c85644d94db9edff8f9bdd4e59083eb841153d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-0-deactivate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, org.label-schema.build-date=20240214) 2024-02-18T11:54:25.985 INFO:journalctl@ceph.osd.0.smithi012.stdout:Feb 18 11:54:25 smithi012 podman[87425]: 2024-02-18 11:54:25.711718085 +0000 UTC m=+0.707456626 container died 8e5ddf7399e0d9707a9947c886c85644d94db9edff8f9bdd4e59083eb841153d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-0-deactivate, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, GIT_BRANCH=HEAD) 2024-02-18T11:54:26.226 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@osd.0.service' 2024-02-18T11:54:26.251 INFO:journalctl@ceph.osd.0.smithi012.stdout:Feb 18 11:54:26 smithi012 podman[87453]: 2024-02-18 11:54:26.218356091 +0000 UTC m=+0.501456835 container remove 8e5ddf7399e0d9707a9947c886c85644d94db9edff8f9bdd4e59083eb841153d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-0-deactivate, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True) 2024-02-18T11:54:26.251 INFO:journalctl@ceph.osd.0.smithi012.stdout:Feb 18 11:54:26 smithi012 systemd[1]: ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@osd.0.service: Deactivated successfully. 2024-02-18T11:54:26.251 INFO:journalctl@ceph.osd.0.smithi012.stdout:Feb 18 11:54:26 smithi012 systemd[1]: Stopped Ceph osd.0 for 0c8bb0de-ce51-11ee-95ba-87774f69a715. 2024-02-18T11:54:26.251 INFO:journalctl@ceph.osd.0.smithi012.stdout:Feb 18 11:54:26 smithi012 systemd[1]: ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@osd.0.service: Consumed 9min 11.840s CPU time. 2024-02-18T11:54:26.266 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-18T11:54:26.267 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-02-18T11:54:26.267 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-02-18T11:54:26.267 DEBUG:teuthology.orchestra.run.smithi158:> sudo systemctl stop ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@osd.1 2024-02-18T11:54:26.570 INFO:journalctl@ceph.osd.1.smithi158.stdout:Feb 18 11:54:26 smithi158 systemd[1]: Stopping Ceph osd.1 for 0c8bb0de-ce51-11ee-95ba-87774f69a715... 2024-02-18T11:54:26.993 INFO:journalctl@ceph.osd.1.smithi158.stdout:Feb 18 11:54:26 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-1[41251]: 2024-02-18T11:54:26.577+0000 7f0a6115b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-18T11:54:26.993 INFO:journalctl@ceph.osd.1.smithi158.stdout:Feb 18 11:54:26 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-1[41251]: 2024-02-18T11:54:26.577+0000 7f0a6115b700 -1 osd.1 92 *** Got signal Terminated *** 2024-02-18T11:54:26.993 INFO:journalctl@ceph.osd.1.smithi158.stdout:Feb 18 11:54:26 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-1[41251]: 2024-02-18T11:54:26.577+0000 7f0a6115b700 -1 osd.1 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-18T11:54:32.296 INFO:journalctl@ceph.osd.1.smithi158.stdout:Feb 18 11:54:32 smithi158 podman[72224]: 2024-02-18 11:54:32.016528782 +0000 UTC m=+5.549429067 container died ae98728565fae580488d1126c21fbc96873aa5df601c1c005d4245247312c8d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-1, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240217.015652-1c920af) 2024-02-18T11:54:32.629 INFO:journalctl@ceph.osd.1.smithi158.stdout:Feb 18 11:54:32 smithi158 podman[72224]: 2024-02-18 11:54:32.419378003 +0000 UTC m=+5.952278274 container cleanup ae98728565fae580488d1126c21fbc96873aa5df601c1c005d4245247312c8d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-1, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240217.015652-1c920af) 2024-02-18T11:54:32.629 INFO:journalctl@ceph.osd.1.smithi158.stdout:Feb 18 11:54:32 smithi158 bash[72224]: ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-1 2024-02-18T11:54:32.952 INFO:journalctl@ceph.osd.1.smithi158.stdout:Feb 18 11:54:32 smithi158 podman[72240]: 2024-02-18 11:54:32.637408097 +0000 UTC m=+0.623679854 container remove ae98728565fae580488d1126c21fbc96873aa5df601c1c005d4245247312c8d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-1, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-02-18T11:54:33.243 INFO:journalctl@ceph.osd.1.smithi158.stdout:Feb 18 11:54:32 smithi158 podman[72332]: 2024-02-18 11:54:32.862839193 +0000 UTC m=+0.017792136 image pull 53c33ee3bf62dc411b650f86eedbbee7698efb1001b4c063271dcd8d65b41165 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1 2024-02-18T11:54:33.243 INFO:journalctl@ceph.osd.1.smithi158.stdout:Feb 18 11:54:32 smithi158 podman[72332]: 2024-02-18 11:54:32.980087751 +0000 UTC m=+0.135040692 container create 1d3011da46a8e7aefd0b81e098c0faddc43b8e3e14caa27196e0844d620a54ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-1-deactivate, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240214) 2024-02-18T11:54:33.243 INFO:journalctl@ceph.osd.1.smithi158.stdout:Feb 18 11:54:33 smithi158 podman[72332]: 2024-02-18 11:54:33.200001202 +0000 UTC m=+0.354954151 container init 1d3011da46a8e7aefd0b81e098c0faddc43b8e3e14caa27196e0844d620a54ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-1-deactivate, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240217.015652-1c920af) 2024-02-18T11:54:33.244 INFO:journalctl@ceph.osd.1.smithi158.stdout:Feb 18 11:54:33 smithi158 podman[72332]: 2024-02-18 11:54:33.203472151 +0000 UTC m=+0.358425089 container start 1d3011da46a8e7aefd0b81e098c0faddc43b8e3e14caa27196e0844d620a54ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-1-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-02-18T11:54:33.559 INFO:journalctl@ceph.osd.1.smithi158.stdout:Feb 18 11:54:33 smithi158 podman[72332]: 2024-02-18 11:54:33.287913004 +0000 UTC m=+0.442865943 container attach 1d3011da46a8e7aefd0b81e098c0faddc43b8e3e14caa27196e0844d620a54ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-1-deactivate, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-18T11:54:33.853 INFO:journalctl@ceph.osd.1.smithi158.stdout:Feb 18 11:54:33 smithi158 podman[72332]: 2024-02-18 11:54:33.567215876 +0000 UTC m=+0.722168815 container died 1d3011da46a8e7aefd0b81e098c0faddc43b8e3e14caa27196e0844d620a54ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-1-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-02-18T11:54:34.147 DEBUG:teuthology.orchestra.run.smithi158:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@osd.1.service' 2024-02-18T11:54:34.189 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-18T11:54:34.189 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-02-18T11:54:34.189 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-02-18T11:54:34.189 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@osd.2 2024-02-18T11:54:34.493 INFO:journalctl@ceph.osd.2.smithi188.stdout:Feb 18 11:54:34 smithi188 systemd[1]: Stopping Ceph osd.2 for 0c8bb0de-ce51-11ee-95ba-87774f69a715... 2024-02-18T11:54:34.853 INFO:journalctl@ceph.osd.2.smithi188.stdout:Feb 18 11:54:34 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-2[42103]: 2024-02-18T11:54:34.499+0000 7fc427d36700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-18T11:54:34.853 INFO:journalctl@ceph.osd.2.smithi188.stdout:Feb 18 11:54:34 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-2[42103]: 2024-02-18T11:54:34.499+0000 7fc427d36700 -1 osd.2 92 *** Got signal Terminated *** 2024-02-18T11:54:34.853 INFO:journalctl@ceph.osd.2.smithi188.stdout:Feb 18 11:54:34 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-2[42103]: 2024-02-18T11:54:34.499+0000 7fc427d36700 -1 osd.2 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-18T11:54:40.151 INFO:journalctl@ceph.osd.2.smithi188.stdout:Feb 18 11:54:39 smithi188 podman[72875]: 2024-02-18 11:54:39.895435057 +0000 UTC m=+5.503121872 container died e3754460562c832f53c10324cadabfcfc00b7c7c955762dbd33324e30c628cc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-2, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.build-date=20240214) 2024-02-18T11:54:40.417 INFO:journalctl@ceph.osd.2.smithi188.stdout:Feb 18 11:54:40 smithi188 podman[72875]: 2024-02-18 11:54:40.232588311 +0000 UTC m=+5.840275117 container cleanup e3754460562c832f53c10324cadabfcfc00b7c7c955762dbd33324e30c628cc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-2, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-18T11:54:40.418 INFO:journalctl@ceph.osd.2.smithi188.stdout:Feb 18 11:54:40 smithi188 bash[72875]: ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-2 2024-02-18T11:54:40.818 INFO:journalctl@ceph.osd.2.smithi188.stdout:Feb 18 11:54:40 smithi188 podman[72894]: 2024-02-18 11:54:40.425518357 +0000 UTC m=+0.527007372 container remove e3754460562c832f53c10324cadabfcfc00b7c7c955762dbd33324e30c628cc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, GIT_BRANCH=HEAD) 2024-02-18T11:54:41.103 INFO:journalctl@ceph.osd.2.smithi188.stdout:Feb 18 11:54:40 smithi188 podman[72983]: 2024-02-18 11:54:40.826281658 +0000 UTC m=+0.109119528 container create 77e7ccd8a841bc777dd7bdd1166bfaf8e60153d9eff841dc3de8a366286dd188 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-2-deactivate, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-02-18T11:54:41.103 INFO:journalctl@ceph.osd.2.smithi188.stdout:Feb 18 11:54:40 smithi188 podman[72983]: 2024-02-18 11:54:40.73478084 +0000 UTC m=+0.017618700 image pull 53c33ee3bf62dc411b650f86eedbbee7698efb1001b4c063271dcd8d65b41165 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1 2024-02-18T11:54:41.396 INFO:journalctl@ceph.osd.2.smithi188.stdout:Feb 18 11:54:41 smithi188 podman[72983]: 2024-02-18 11:54:41.112083444 +0000 UTC m=+0.394921303 container init 77e7ccd8a841bc777dd7bdd1166bfaf8e60153d9eff841dc3de8a366286dd188 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-2-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-18T11:54:41.396 INFO:journalctl@ceph.osd.2.smithi188.stdout:Feb 18 11:54:41 smithi188 podman[72983]: 2024-02-18 11:54:41.115364089 +0000 UTC m=+0.398201940 container start 77e7ccd8a841bc777dd7bdd1166bfaf8e60153d9eff841dc3de8a366286dd188 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-2-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-vshankar-testing-20240217.015652-1c920af) 2024-02-18T11:54:41.396 INFO:journalctl@ceph.osd.2.smithi188.stdout:Feb 18 11:54:41 smithi188 podman[72983]: 2024-02-18 11:54:41.220794792 +0000 UTC m=+0.503632658 container attach 77e7ccd8a841bc777dd7bdd1166bfaf8e60153d9eff841dc3de8a366286dd188 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-2-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-18T11:54:41.396 INFO:journalctl@ceph.osd.2.smithi188.stdout:Feb 18 11:54:41 smithi188 podman[72983]: 2024-02-18 11:54:41.403377892 +0000 UTC m=+0.686215742 container died 77e7ccd8a841bc777dd7bdd1166bfaf8e60153d9eff841dc3de8a366286dd188 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-2-deactivate, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, org.label-schema.license=GPLv2) 2024-02-18T11:54:41.962 INFO:journalctl@ceph.osd.2.smithi188.stdout:Feb 18 11:54:41 smithi188 podman[73009]: 2024-02-18 11:54:41.949233 +0000 UTC m=+0.537788549 container remove 77e7ccd8a841bc777dd7bdd1166bfaf8e60153d9eff841dc3de8a366286dd188 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-2-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-02-18T11:54:41.967 DEBUG:teuthology.orchestra.run.smithi188:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@osd.2.service' 2024-02-18T11:54:42.009 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-18T11:54:42.010 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-02-18T11:54:42.010 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-02-18T11:54:42.010 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@osd.3 2024-02-18T11:54:42.295 INFO:journalctl@ceph.osd.3.smithi012.stdout:Feb 18 11:54:42 smithi012 systemd[1]: Stopping Ceph osd.3 for 0c8bb0de-ce51-11ee-95ba-87774f69a715... 2024-02-18T11:54:42.566 INFO:journalctl@ceph.osd.3.smithi012.stdout:Feb 18 11:54:42 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-3[51430]: 2024-02-18T11:54:42.302+0000 7f856b21f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-18T11:54:42.566 INFO:journalctl@ceph.osd.3.smithi012.stdout:Feb 18 11:54:42 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-3[51430]: 2024-02-18T11:54:42.302+0000 7f856b21f700 -1 osd.3 92 *** Got signal Terminated *** 2024-02-18T11:54:42.566 INFO:journalctl@ceph.osd.3.smithi012.stdout:Feb 18 11:54:42 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-3[51430]: 2024-02-18T11:54:42.302+0000 7f856b21f700 -1 osd.3 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-18T11:54:46.353 INFO:journalctl@ceph.osd.5.smithi188.stdout:Feb 18 11:54:45 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-5[48668]: 2024-02-18T11:54:45.965+0000 7f7791b5f700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:20.524067+0000 front 2024-02-18T11:54:20.523988+0000 (oldest deadline 2024-02-18T11:54:45.823829+0000) 2024-02-18T11:54:46.422 INFO:journalctl@ceph.osd.3.smithi012.stdout:Feb 18 11:54:46 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-3[51430]: 2024-02-18T11:54:46.168+0000 7f856400c700 -1 osd.3 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:22.897477+0000 front 2024-02-18T11:54:22.897448+0000 (oldest deadline 2024-02-18T11:54:45.797584+0000) 2024-02-18T11:54:46.493 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:54:46 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:54:46.176+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:22.525039+0000 front 2024-02-18T11:54:22.525199+0000 (oldest deadline 2024-02-18T11:54:46.024887+0000) 2024-02-18T11:54:47.353 INFO:journalctl@ceph.osd.5.smithi188.stdout:Feb 18 11:54:46 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-5[48668]: 2024-02-18T11:54:46.997+0000 7f7791b5f700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:20.524067+0000 front 2024-02-18T11:54:20.523988+0000 (oldest deadline 2024-02-18T11:54:45.823829+0000) 2024-02-18T11:54:47.394 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:54:47 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:54:47.130+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:22.525039+0000 front 2024-02-18T11:54:22.525199+0000 (oldest deadline 2024-02-18T11:54:46.024887+0000) 2024-02-18T11:54:47.422 INFO:journalctl@ceph.osd.3.smithi012.stdout:Feb 18 11:54:47 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-3[51430]: 2024-02-18T11:54:47.154+0000 7f856400c700 -1 osd.3 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:22.897477+0000 front 2024-02-18T11:54:22.897448+0000 (oldest deadline 2024-02-18T11:54:45.797584+0000) 2024-02-18T11:54:47.853 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:54:47 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:54:47.360+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.164300+0000 front 2024-02-18T11:54:23.164278+0000 (oldest deadline 2024-02-18T11:54:46.664258+0000) 2024-02-18T11:54:48.162 INFO:journalctl@ceph.osd.3.smithi012.stdout:Feb 18 11:54:47 smithi012 podman[87533]: 2024-02-18 11:54:47.891847884 +0000 UTC m=+5.682269445 container died fe248724d54471a99a6bb827f4649d323c1da21ea1fc2b07778c47498f8060f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-3, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-02-18T11:54:48.353 INFO:journalctl@ceph.osd.5.smithi188.stdout:Feb 18 11:54:47 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-5[48668]: 2024-02-18T11:54:47.996+0000 7f7791b5f700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:20.524067+0000 front 2024-02-18T11:54:20.523988+0000 (oldest deadline 2024-02-18T11:54:45.823829+0000) 2024-02-18T11:54:48.422 INFO:journalctl@ceph.osd.3.smithi012.stdout:Feb 18 11:54:48 smithi012 podman[87533]: 2024-02-18 11:54:48.260800857 +0000 UTC m=+6.051222396 container cleanup fe248724d54471a99a6bb827f4649d323c1da21ea1fc2b07778c47498f8060f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-3, ceph=True, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-18T11:54:48.422 INFO:journalctl@ceph.osd.3.smithi012.stdout:Feb 18 11:54:48 smithi012 bash[87533]: ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-3 2024-02-18T11:54:48.493 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:54:48 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:54:48.134+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:22.525039+0000 front 2024-02-18T11:54:22.525199+0000 (oldest deadline 2024-02-18T11:54:46.024887+0000) 2024-02-18T11:54:48.853 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:54:48 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:54:48.379+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.164300+0000 front 2024-02-18T11:54:23.164278+0000 (oldest deadline 2024-02-18T11:54:46.664258+0000) 2024-02-18T11:54:48.922 INFO:journalctl@ceph.osd.3.smithi012.stdout:Feb 18 11:54:48 smithi012 podman[87561]: 2024-02-18 11:54:48.587162714 +0000 UTC m=+0.691381934 container remove fe248724d54471a99a6bb827f4649d323c1da21ea1fc2b07778c47498f8060f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-3, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.vendor=CentOS) 2024-02-18T11:54:49.183 INFO:journalctl@ceph.osd.3.smithi012.stdout:Feb 18 11:54:48 smithi012 podman[87651]: 2024-02-18 11:54:48.836778021 +0000 UTC m=+0.018666043 image pull 53c33ee3bf62dc411b650f86eedbbee7698efb1001b4c063271dcd8d65b41165 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1 2024-02-18T11:54:49.183 INFO:journalctl@ceph.osd.3.smithi012.stdout:Feb 18 11:54:48 smithi012 podman[87651]: 2024-02-18 11:54:48.963186389 +0000 UTC m=+0.145074446 container create 5c742bf377bc42abc9f28ee4077b17db7978caaed7f32c66bc62b1d926093134 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-3-deactivate, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-18T11:54:49.183 INFO:journalctl@ceph.osd.3.smithi012.stdout:Feb 18 11:54:49 smithi012 podman[87651]: 2024-02-18 11:54:49.190564007 +0000 UTC m=+0.372452022 container init 5c742bf377bc42abc9f28ee4077b17db7978caaed7f32c66bc62b1d926093134 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-3-deactivate, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-02-18T11:54:49.353 INFO:journalctl@ceph.osd.5.smithi188.stdout:Feb 18 11:54:48 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-5[48668]: 2024-02-18T11:54:48.967+0000 7f7791b5f700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:20.524067+0000 front 2024-02-18T11:54:20.523988+0000 (oldest deadline 2024-02-18T11:54:45.823829+0000) 2024-02-18T11:54:49.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:54:48 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:54:48.978+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:21.400001+0000 front 2024-02-18T11:54:21.400128+0000 (oldest deadline 2024-02-18T11:54:48.400621+0000) 2024-02-18T11:54:49.485 INFO:journalctl@ceph.osd.3.smithi012.stdout:Feb 18 11:54:49 smithi012 podman[87651]: 2024-02-18 11:54:49.193742988 +0000 UTC m=+0.375631003 container start 5c742bf377bc42abc9f28ee4077b17db7978caaed7f32c66bc62b1d926093134 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-02-18T11:54:49.485 INFO:journalctl@ceph.osd.3.smithi012.stdout:Feb 18 11:54:49 smithi012 podman[87651]: 2024-02-18 11:54:49.290384644 +0000 UTC m=+0.472272659 container attach 5c742bf377bc42abc9f28ee4077b17db7978caaed7f32c66bc62b1d926093134 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-3-deactivate, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, ceph=True, maintainer=Guillaume Abrioux ) 2024-02-18T11:54:49.493 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:54:49 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:54:49.169+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:22.525039+0000 front 2024-02-18T11:54:22.525199+0000 (oldest deadline 2024-02-18T11:54:46.024887+0000) 2024-02-18T11:54:49.769 INFO:journalctl@ceph.osd.3.smithi012.stdout:Feb 18 11:54:49 smithi012 podman[87651]: 2024-02-18 11:54:49.492810264 +0000 UTC m=+0.674698295 container died 5c742bf377bc42abc9f28ee4077b17db7978caaed7f32c66bc62b1d926093134 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-3-deactivate, ceph=True, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-02-18T11:54:49.853 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:54:49 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:54:49.370+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.164300+0000 front 2024-02-18T11:54:23.164278+0000 (oldest deadline 2024-02-18T11:54:46.664258+0000) 2024-02-18T11:54:50.088 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@osd.3.service' 2024-02-18T11:54:50.132 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-18T11:54:50.132 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-02-18T11:54:50.133 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-02-18T11:54:50.133 DEBUG:teuthology.orchestra.run.smithi158:> sudo systemctl stop ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@osd.4 2024-02-18T11:54:50.317 INFO:journalctl@ceph.osd.5.smithi188.stdout:Feb 18 11:54:49 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-5[48668]: 2024-02-18T11:54:49.918+0000 7f7791b5f700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:20.524067+0000 front 2024-02-18T11:54:20.523988+0000 (oldest deadline 2024-02-18T11:54:45.823829+0000) 2024-02-18T11:54:50.318 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:54:50 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:54:50.007+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:21.400001+0000 front 2024-02-18T11:54:21.400128+0000 (oldest deadline 2024-02-18T11:54:48.400621+0000) 2024-02-18T11:54:50.422 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:54:50 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:54:50.058+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.279478+0000 front 2024-02-18T11:54:23.279513+0000 (oldest deadline 2024-02-18T11:54:49.179497+0000) 2024-02-18T11:54:50.422 INFO:journalctl@ceph.osd.6.smithi012.stdout:Feb 18 11:54:50 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-6[58474]: 2024-02-18T11:54:50.181+0000 7fcbe0990700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:19.389184+0000 front 2024-02-18T11:54:19.389074+0000 (oldest deadline 2024-02-18T11:54:49.989726+0000) 2024-02-18T11:54:50.432 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:54:50 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:54:50.131+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:22.525039+0000 front 2024-02-18T11:54:22.525199+0000 (oldest deadline 2024-02-18T11:54:46.024887+0000) 2024-02-18T11:54:50.432 INFO:journalctl@ceph.osd.4.smithi158.stdout:Feb 18 11:54:50 smithi158 systemd[1]: Stopping Ceph osd.4 for 0c8bb0de-ce51-11ee-95ba-87774f69a715... 2024-02-18T11:54:50.603 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:54:50 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:54:50.324+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.164300+0000 front 2024-02-18T11:54:23.164278+0000 (oldest deadline 2024-02-18T11:54:46.664258+0000) 2024-02-18T11:54:50.743 INFO:journalctl@ceph.osd.4.smithi158.stdout:Feb 18 11:54:50 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-4[47822]: 2024-02-18T11:54:50.439+0000 7fac7809d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-18T11:54:50.744 INFO:journalctl@ceph.osd.4.smithi158.stdout:Feb 18 11:54:50 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-4[47822]: 2024-02-18T11:54:50.439+0000 7fac7809d700 -1 osd.4 92 *** Got signal Terminated *** 2024-02-18T11:54:50.744 INFO:journalctl@ceph.osd.4.smithi158.stdout:Feb 18 11:54:50 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-4[47822]: 2024-02-18T11:54:50.439+0000 7fac7809d700 -1 osd.4 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-18T11:54:51.286 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:54:51 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:54:51.051+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:21.400001+0000 front 2024-02-18T11:54:21.400128+0000 (oldest deadline 2024-02-18T11:54:48.400621+0000) 2024-02-18T11:54:51.286 INFO:journalctl@ceph.osd.5.smithi188.stdout:Feb 18 11:54:50 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-5[48668]: 2024-02-18T11:54:50.963+0000 7f7791b5f700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:20.524067+0000 front 2024-02-18T11:54:20.523988+0000 (oldest deadline 2024-02-18T11:54:45.823829+0000) 2024-02-18T11:54:51.422 INFO:journalctl@ceph.osd.6.smithi012.stdout:Feb 18 11:54:51 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-6[58474]: 2024-02-18T11:54:51.185+0000 7fcbe0990700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:19.389184+0000 front 2024-02-18T11:54:19.389074+0000 (oldest deadline 2024-02-18T11:54:49.989726+0000) 2024-02-18T11:54:51.422 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:54:51 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:54:51.092+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.279478+0000 front 2024-02-18T11:54:23.279513+0000 (oldest deadline 2024-02-18T11:54:49.179497+0000) 2024-02-18T11:54:51.493 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:54:51 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:54:51.163+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:22.525039+0000 front 2024-02-18T11:54:22.525199+0000 (oldest deadline 2024-02-18T11:54:46.024887+0000) 2024-02-18T11:54:51.603 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:54:51 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:54:51.293+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.164300+0000 front 2024-02-18T11:54:23.164278+0000 (oldest deadline 2024-02-18T11:54:46.664258+0000) 2024-02-18T11:54:52.320 INFO:journalctl@ceph.osd.5.smithi188.stdout:Feb 18 11:54:51 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-5[48668]: 2024-02-18T11:54:51.929+0000 7f7791b5f700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:20.524067+0000 front 2024-02-18T11:54:20.523988+0000 (oldest deadline 2024-02-18T11:54:45.823829+0000) 2024-02-18T11:54:52.320 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:54:52 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:54:52.032+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:21.400001+0000 front 2024-02-18T11:54:21.400128+0000 (oldest deadline 2024-02-18T11:54:48.400621+0000) 2024-02-18T11:54:52.422 INFO:journalctl@ceph.osd.6.smithi012.stdout:Feb 18 11:54:52 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-6[58474]: 2024-02-18T11:54:52.159+0000 7fcbe0990700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:19.389184+0000 front 2024-02-18T11:54:19.389074+0000 (oldest deadline 2024-02-18T11:54:49.989726+0000) 2024-02-18T11:54:52.422 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:54:52 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:54:52.090+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.279478+0000 front 2024-02-18T11:54:23.279513+0000 (oldest deadline 2024-02-18T11:54:49.179497+0000) 2024-02-18T11:54:52.422 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:54:52 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:54:52.090+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:29.180115+0000 front 2024-02-18T11:54:29.180243+0000 (oldest deadline 2024-02-18T11:54:52.080171+0000) 2024-02-18T11:54:52.493 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:54:52 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:54:52.177+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:22.525039+0000 front 2024-02-18T11:54:22.525199+0000 (oldest deadline 2024-02-18T11:54:46.024887+0000) 2024-02-18T11:54:52.603 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:54:52 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:54:52.327+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.164300+0000 front 2024-02-18T11:54:23.164278+0000 (oldest deadline 2024-02-18T11:54:46.664258+0000) 2024-02-18T11:54:53.248 INFO:journalctl@ceph.osd.5.smithi188.stdout:Feb 18 11:54:52 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-5[48668]: 2024-02-18T11:54:52.952+0000 7f7791b5f700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:20.524067+0000 front 2024-02-18T11:54:20.523988+0000 (oldest deadline 2024-02-18T11:54:45.823829+0000) 2024-02-18T11:54:53.248 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:54:53 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:54:53.065+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:21.400001+0000 front 2024-02-18T11:54:21.400128+0000 (oldest deadline 2024-02-18T11:54:48.400621+0000) 2024-02-18T11:54:53.248 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:54:53 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:54:53.065+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.901802+0000 front 2024-02-18T11:54:28.901977+0000 (oldest deadline 2024-02-18T11:54:53.001759+0000) 2024-02-18T11:54:53.422 INFO:journalctl@ceph.osd.6.smithi012.stdout:Feb 18 11:54:53 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-6[58474]: 2024-02-18T11:54:53.180+0000 7fcbe0990700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:19.389184+0000 front 2024-02-18T11:54:19.389074+0000 (oldest deadline 2024-02-18T11:54:49.989726+0000) 2024-02-18T11:54:53.422 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:54:53 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:54:53.060+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.279478+0000 front 2024-02-18T11:54:23.279513+0000 (oldest deadline 2024-02-18T11:54:49.179497+0000) 2024-02-18T11:54:53.422 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:54:53 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:54:53.060+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:29.180115+0000 front 2024-02-18T11:54:29.180243+0000 (oldest deadline 2024-02-18T11:54:52.080171+0000) 2024-02-18T11:54:53.493 INFO:journalctl@ceph.osd.4.smithi158.stdout:Feb 18 11:54:53 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-4[47822]: 2024-02-18T11:54:53.156+0000 7fac70e8a700 -1 osd.4 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.583357+0000 front 2024-02-18T11:54:28.583386+0000 (oldest deadline 2024-02-18T11:54:52.683483+0000) 2024-02-18T11:54:53.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:54:53 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:54:53.203+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:22.525039+0000 front 2024-02-18T11:54:22.525199+0000 (oldest deadline 2024-02-18T11:54:46.024887+0000) 2024-02-18T11:54:53.603 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:54:53 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:54:53.329+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.164300+0000 front 2024-02-18T11:54:23.164278+0000 (oldest deadline 2024-02-18T11:54:46.664258+0000) 2024-02-18T11:54:54.305 INFO:journalctl@ceph.osd.5.smithi188.stdout:Feb 18 11:54:53 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-5[48668]: 2024-02-18T11:54:53.930+0000 7f7791b5f700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:20.524067+0000 front 2024-02-18T11:54:20.523988+0000 (oldest deadline 2024-02-18T11:54:45.823829+0000) 2024-02-18T11:54:54.305 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:54:54 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:54:54.105+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:21.400001+0000 front 2024-02-18T11:54:21.400128+0000 (oldest deadline 2024-02-18T11:54:48.400621+0000) 2024-02-18T11:54:54.305 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:54:54 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:54:54.105+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.901802+0000 front 2024-02-18T11:54:28.901977+0000 (oldest deadline 2024-02-18T11:54:53.001759+0000) 2024-02-18T11:54:54.422 INFO:journalctl@ceph.osd.6.smithi012.stdout:Feb 18 11:54:54 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-6[58474]: 2024-02-18T11:54:54.132+0000 7fcbe0990700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:19.389184+0000 front 2024-02-18T11:54:19.389074+0000 (oldest deadline 2024-02-18T11:54:49.989726+0000) 2024-02-18T11:54:54.423 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:54:54 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:54:54.017+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.279478+0000 front 2024-02-18T11:54:23.279513+0000 (oldest deadline 2024-02-18T11:54:49.179497+0000) 2024-02-18T11:54:54.423 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:54:54 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:54:54.017+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:29.180115+0000 front 2024-02-18T11:54:29.180243+0000 (oldest deadline 2024-02-18T11:54:52.080171+0000) 2024-02-18T11:54:54.493 INFO:journalctl@ceph.osd.4.smithi158.stdout:Feb 18 11:54:54 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-4[47822]: 2024-02-18T11:54:54.125+0000 7fac70e8a700 -1 osd.4 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.583357+0000 front 2024-02-18T11:54:28.583386+0000 (oldest deadline 2024-02-18T11:54:52.683483+0000) 2024-02-18T11:54:54.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:54:54 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:54:54.194+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:22.525039+0000 front 2024-02-18T11:54:22.525199+0000 (oldest deadline 2024-02-18T11:54:46.024887+0000) 2024-02-18T11:54:54.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:54:54 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:54:54.194+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:31.127112+0000 front 2024-02-18T11:54:31.127151+0000 (oldest deadline 2024-02-18T11:54:54.027319+0000) 2024-02-18T11:54:54.604 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:54:54 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:54:54.311+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.164300+0000 front 2024-02-18T11:54:23.164278+0000 (oldest deadline 2024-02-18T11:54:46.664258+0000) 2024-02-18T11:54:54.604 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:54:54 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:54:54.311+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.365211+0000 front 2024-02-18T11:54:28.365390+0000 (oldest deadline 2024-02-18T11:54:53.665226+0000) 2024-02-18T11:54:55.258 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:54:55 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:54:55.083+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:21.400001+0000 front 2024-02-18T11:54:21.400128+0000 (oldest deadline 2024-02-18T11:54:48.400621+0000) 2024-02-18T11:54:55.259 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:54:55 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:54:55.083+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.901802+0000 front 2024-02-18T11:54:28.901977+0000 (oldest deadline 2024-02-18T11:54:53.001759+0000) 2024-02-18T11:54:55.259 INFO:journalctl@ceph.osd.5.smithi188.stdout:Feb 18 11:54:54 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-5[48668]: 2024-02-18T11:54:54.955+0000 7f7791b5f700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:20.524067+0000 front 2024-02-18T11:54:20.523988+0000 (oldest deadline 2024-02-18T11:54:45.823829+0000) 2024-02-18T11:54:55.259 INFO:journalctl@ceph.osd.5.smithi188.stdout:Feb 18 11:54:54 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-5[48668]: 2024-02-18T11:54:54.955+0000 7f7791b5f700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:29.324890+0000 front 2024-02-18T11:54:29.324961+0000 (oldest deadline 2024-02-18T11:54:54.624902+0000) 2024-02-18T11:54:55.422 INFO:journalctl@ceph.osd.6.smithi012.stdout:Feb 18 11:54:55 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-6[58474]: 2024-02-18T11:54:55.134+0000 7fcbe0990700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:19.389184+0000 front 2024-02-18T11:54:19.389074+0000 (oldest deadline 2024-02-18T11:54:49.989726+0000) 2024-02-18T11:54:55.422 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:54:55 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:54:55.051+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.279478+0000 front 2024-02-18T11:54:23.279513+0000 (oldest deadline 2024-02-18T11:54:49.179497+0000) 2024-02-18T11:54:55.423 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:54:55 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:54:55.051+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:29.180115+0000 front 2024-02-18T11:54:29.180243+0000 (oldest deadline 2024-02-18T11:54:52.080171+0000) 2024-02-18T11:54:55.493 INFO:journalctl@ceph.osd.4.smithi158.stdout:Feb 18 11:54:55 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-4[47822]: 2024-02-18T11:54:55.111+0000 7fac70e8a700 -1 osd.4 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.583357+0000 front 2024-02-18T11:54:28.583386+0000 (oldest deadline 2024-02-18T11:54:52.683483+0000) 2024-02-18T11:54:55.494 INFO:journalctl@ceph.osd.7.smithi158.stdout:Feb 18 11:54:55 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-7[54444]: 2024-02-18T11:54:55.357+0000 7f24cf624700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:31.308446+0000 front 2024-02-18T11:54:31.308475+0000 (oldest deadline 2024-02-18T11:54:54.808583+0000) 2024-02-18T11:54:55.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:54:55 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:54:55.157+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:22.525039+0000 front 2024-02-18T11:54:22.525199+0000 (oldest deadline 2024-02-18T11:54:46.024887+0000) 2024-02-18T11:54:55.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:54:55 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:54:55.157+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:31.127112+0000 front 2024-02-18T11:54:31.127151+0000 (oldest deadline 2024-02-18T11:54:54.027319+0000) 2024-02-18T11:54:55.603 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:54:55 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:54:55.265+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.164300+0000 front 2024-02-18T11:54:23.164278+0000 (oldest deadline 2024-02-18T11:54:46.664258+0000) 2024-02-18T11:54:55.604 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:54:55 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:54:55.265+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.365211+0000 front 2024-02-18T11:54:28.365390+0000 (oldest deadline 2024-02-18T11:54:53.665226+0000) 2024-02-18T11:54:56.238 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:54:56 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:54:56.074+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:21.400001+0000 front 2024-02-18T11:54:21.400128+0000 (oldest deadline 2024-02-18T11:54:48.400621+0000) 2024-02-18T11:54:56.238 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:54:56 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:54:56.074+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.901802+0000 front 2024-02-18T11:54:28.901977+0000 (oldest deadline 2024-02-18T11:54:53.001759+0000) 2024-02-18T11:54:56.238 INFO:journalctl@ceph.osd.5.smithi188.stdout:Feb 18 11:54:55 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-5[48668]: 2024-02-18T11:54:55.961+0000 7f7791b5f700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:20.524067+0000 front 2024-02-18T11:54:20.523988+0000 (oldest deadline 2024-02-18T11:54:45.823829+0000) 2024-02-18T11:54:56.239 INFO:journalctl@ceph.osd.5.smithi188.stdout:Feb 18 11:54:55 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-5[48668]: 2024-02-18T11:54:55.961+0000 7f7791b5f700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:29.324890+0000 front 2024-02-18T11:54:29.324961+0000 (oldest deadline 2024-02-18T11:54:54.624902+0000) 2024-02-18T11:54:56.365 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:54:56 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:54:56.156+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:22.525039+0000 front 2024-02-18T11:54:22.525199+0000 (oldest deadline 2024-02-18T11:54:46.024887+0000) 2024-02-18T11:54:56.365 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:54:56 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:54:56.156+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:31.127112+0000 front 2024-02-18T11:54:31.127151+0000 (oldest deadline 2024-02-18T11:54:54.027319+0000) 2024-02-18T11:54:56.365 INFO:journalctl@ceph.osd.4.smithi158.stdout:Feb 18 11:54:56 smithi158 podman[72447]: 2024-02-18 11:54:56.051144919 +0000 UTC m=+5.715926880 container died fd54360331aec8ace4cad9b720e859f15ed62fd8bbe55faee64700f599b38839 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-4, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, org.label-schema.license=GPLv2) 2024-02-18T11:54:56.365 INFO:journalctl@ceph.osd.4.smithi158.stdout:Feb 18 11:54:56 smithi158 podman[72447]: 2024-02-18 11:54:56.372374229 +0000 UTC m=+6.037156178 container cleanup fd54360331aec8ace4cad9b720e859f15ed62fd8bbe55faee64700f599b38839 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-4, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214) 2024-02-18T11:54:56.422 INFO:journalctl@ceph.osd.6.smithi012.stdout:Feb 18 11:54:56 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-6[58474]: 2024-02-18T11:54:56.105+0000 7fcbe0990700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:19.389184+0000 front 2024-02-18T11:54:19.389074+0000 (oldest deadline 2024-02-18T11:54:49.989726+0000) 2024-02-18T11:54:56.422 INFO:journalctl@ceph.osd.6.smithi012.stdout:Feb 18 11:54:56 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-6[58474]: 2024-02-18T11:54:56.105+0000 7fcbe0990700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:29.990442+0000 front 2024-02-18T11:54:29.990459+0000 (oldest deadline 2024-02-18T11:54:55.290283+0000) 2024-02-18T11:54:56.423 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:54:56 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:54:56.053+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.279478+0000 front 2024-02-18T11:54:23.279513+0000 (oldest deadline 2024-02-18T11:54:49.179497+0000) 2024-02-18T11:54:56.423 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:54:56 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:54:56.053+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:29.180115+0000 front 2024-02-18T11:54:29.180243+0000 (oldest deadline 2024-02-18T11:54:52.080171+0000) 2024-02-18T11:54:56.529 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:54:56 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:54:56.245+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.164300+0000 front 2024-02-18T11:54:23.164278+0000 (oldest deadline 2024-02-18T11:54:46.664258+0000) 2024-02-18T11:54:56.529 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:54:56 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:54:56.245+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.365211+0000 front 2024-02-18T11:54:28.365390+0000 (oldest deadline 2024-02-18T11:54:53.665226+0000) 2024-02-18T11:54:56.743 INFO:journalctl@ceph.osd.7.smithi158.stdout:Feb 18 11:54:56 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-7[54444]: 2024-02-18T11:54:56.374+0000 7f24cf624700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:31.308446+0000 front 2024-02-18T11:54:31.308475+0000 (oldest deadline 2024-02-18T11:54:54.808583+0000) 2024-02-18T11:54:56.743 INFO:journalctl@ceph.osd.4.smithi158.stdout:Feb 18 11:54:56 smithi158 bash[72447]: ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-4 2024-02-18T11:54:56.744 INFO:journalctl@ceph.osd.4.smithi158.stdout:Feb 18 11:54:56 smithi158 podman[72464]: 2024-02-18 11:54:56.59065246 +0000 UTC m=+0.536701432 container remove fd54360331aec8ace4cad9b720e859f15ed62fd8bbe55faee64700f599b38839 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-4, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240217.015652-1c920af) 2024-02-18T11:54:57.237 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:54:57 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:54:57.095+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:21.400001+0000 front 2024-02-18T11:54:21.400128+0000 (oldest deadline 2024-02-18T11:54:48.400621+0000) 2024-02-18T11:54:57.237 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:54:57 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:54:57.095+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.901802+0000 front 2024-02-18T11:54:28.901977+0000 (oldest deadline 2024-02-18T11:54:53.001759+0000) 2024-02-18T11:54:57.237 INFO:journalctl@ceph.osd.5.smithi188.stdout:Feb 18 11:54:56 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-5[48668]: 2024-02-18T11:54:56.956+0000 7f7791b5f700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:20.524067+0000 front 2024-02-18T11:54:20.523988+0000 (oldest deadline 2024-02-18T11:54:45.823829+0000) 2024-02-18T11:54:57.237 INFO:journalctl@ceph.osd.5.smithi188.stdout:Feb 18 11:54:56 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-5[48668]: 2024-02-18T11:54:56.956+0000 7f7791b5f700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:29.324890+0000 front 2024-02-18T11:54:29.324961+0000 (oldest deadline 2024-02-18T11:54:54.624902+0000) 2024-02-18T11:54:57.243 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:54:57 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:54:57.182+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:22.525039+0000 front 2024-02-18T11:54:22.525199+0000 (oldest deadline 2024-02-18T11:54:46.024887+0000) 2024-02-18T11:54:57.243 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:54:57 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:54:57.182+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:31.127112+0000 front 2024-02-18T11:54:31.127151+0000 (oldest deadline 2024-02-18T11:54:54.027319+0000) 2024-02-18T11:54:57.244 INFO:journalctl@ceph.osd.4.smithi158.stdout:Feb 18 11:54:56 smithi158 podman[72556]: 2024-02-18 11:54:56.89461659 +0000 UTC m=+0.017849418 image pull 53c33ee3bf62dc411b650f86eedbbee7698efb1001b4c063271dcd8d65b41165 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1 2024-02-18T11:54:57.244 INFO:journalctl@ceph.osd.4.smithi158.stdout:Feb 18 11:54:57 smithi158 podman[72556]: 2024-02-18 11:54:57.008178626 +0000 UTC m=+0.131411464 container create cda2b3431ebaec4707e1cf0a0d7e44e8917a8da7cb4c23df2ed2e7e1f8fc6739 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-4-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, org.label-schema.build-date=20240214) 2024-02-18T11:54:57.244 INFO:journalctl@ceph.osd.4.smithi158.stdout:Feb 18 11:54:57 smithi158 podman[72556]: 2024-02-18 11:54:57.219059279 +0000 UTC m=+0.342292113 container init cda2b3431ebaec4707e1cf0a0d7e44e8917a8da7cb4c23df2ed2e7e1f8fc6739 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-02-18T11:54:57.244 INFO:journalctl@ceph.osd.4.smithi158.stdout:Feb 18 11:54:57 smithi158 podman[72556]: 2024-02-18 11:54:57.222155584 +0000 UTC m=+0.345388403 container start cda2b3431ebaec4707e1cf0a0d7e44e8917a8da7cb4c23df2ed2e7e1f8fc6739 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-4-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-02-18T11:54:57.422 INFO:journalctl@ceph.osd.6.smithi012.stdout:Feb 18 11:54:57 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-6[58474]: 2024-02-18T11:54:57.148+0000 7fcbe0990700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:19.389184+0000 front 2024-02-18T11:54:19.389074+0000 (oldest deadline 2024-02-18T11:54:49.989726+0000) 2024-02-18T11:54:57.422 INFO:journalctl@ceph.osd.6.smithi012.stdout:Feb 18 11:54:57 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-6[58474]: 2024-02-18T11:54:57.148+0000 7fcbe0990700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:29.990442+0000 front 2024-02-18T11:54:29.990459+0000 (oldest deadline 2024-02-18T11:54:55.290283+0000) 2024-02-18T11:54:57.422 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:54:57 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:54:57.032+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.279478+0000 front 2024-02-18T11:54:23.279513+0000 (oldest deadline 2024-02-18T11:54:49.179497+0000) 2024-02-18T11:54:57.423 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:54:57 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:54:57.032+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:29.180115+0000 front 2024-02-18T11:54:29.180243+0000 (oldest deadline 2024-02-18T11:54:52.080171+0000) 2024-02-18T11:54:57.603 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:54:57 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:54:57.243+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.164300+0000 front 2024-02-18T11:54:23.164278+0000 (oldest deadline 2024-02-18T11:54:46.664258+0000) 2024-02-18T11:54:57.603 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:54:57 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:54:57.243+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.365211+0000 front 2024-02-18T11:54:28.365390+0000 (oldest deadline 2024-02-18T11:54:53.665226+0000) 2024-02-18T11:54:57.743 INFO:journalctl@ceph.osd.7.smithi158.stdout:Feb 18 11:54:57 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-7[54444]: 2024-02-18T11:54:57.366+0000 7f24cf624700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:31.308446+0000 front 2024-02-18T11:54:31.308475+0000 (oldest deadline 2024-02-18T11:54:54.808583+0000) 2024-02-18T11:54:57.744 INFO:journalctl@ceph.osd.4.smithi158.stdout:Feb 18 11:54:57 smithi158 podman[72556]: 2024-02-18 11:54:57.302734885 +0000 UTC m=+0.425967710 container attach cda2b3431ebaec4707e1cf0a0d7e44e8917a8da7cb4c23df2ed2e7e1f8fc6739 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-4-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.build-date=20240214) 2024-02-18T11:54:57.744 INFO:journalctl@ceph.osd.4.smithi158.stdout:Feb 18 11:54:57 smithi158 podman[72556]: 2024-02-18 11:54:57.508064167 +0000 UTC m=+0.631296988 container died cda2b3431ebaec4707e1cf0a0d7e44e8917a8da7cb4c23df2ed2e7e1f8fc6739 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-4-deactivate, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0) 2024-02-18T11:54:58.071 DEBUG:teuthology.orchestra.run.smithi158:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@osd.4.service' 2024-02-18T11:54:58.112 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-18T11:54:58.112 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-02-18T11:54:58.113 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-02-18T11:54:58.113 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@osd.5 2024-02-18T11:54:58.286 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:54:58 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:54:58.072+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:21.400001+0000 front 2024-02-18T11:54:21.400128+0000 (oldest deadline 2024-02-18T11:54:48.400621+0000) 2024-02-18T11:54:58.344 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:54:58 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:54:58.072+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.901802+0000 front 2024-02-18T11:54:28.901977+0000 (oldest deadline 2024-02-18T11:54:53.001759+0000) 2024-02-18T11:54:58.345 INFO:journalctl@ceph.osd.5.smithi188.stdout:Feb 18 11:54:57 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-5[48668]: 2024-02-18T11:54:57.973+0000 7f7791b5f700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:20.524067+0000 front 2024-02-18T11:54:20.523988+0000 (oldest deadline 2024-02-18T11:54:45.823829+0000) 2024-02-18T11:54:58.345 INFO:journalctl@ceph.osd.5.smithi188.stdout:Feb 18 11:54:57 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-5[48668]: 2024-02-18T11:54:57.973+0000 7f7791b5f700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:29.324890+0000 front 2024-02-18T11:54:29.324961+0000 (oldest deadline 2024-02-18T11:54:54.624902+0000) 2024-02-18T11:54:58.345 INFO:journalctl@ceph.osd.5.smithi188.stdout:Feb 18 11:54:58 smithi188 systemd[1]: Stopping Ceph osd.5 for 0c8bb0de-ce51-11ee-95ba-87774f69a715... 2024-02-18T11:54:58.377 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:54:58 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:54:58.184+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:22.525039+0000 front 2024-02-18T11:54:22.525199+0000 (oldest deadline 2024-02-18T11:54:46.024887+0000) 2024-02-18T11:54:58.377 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:54:58 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:54:58.184+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:31.127112+0000 front 2024-02-18T11:54:31.127151+0000 (oldest deadline 2024-02-18T11:54:54.027319+0000) 2024-02-18T11:54:58.424 INFO:journalctl@ceph.osd.6.smithi012.stdout:Feb 18 11:54:58 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-6[58474]: 2024-02-18T11:54:58.148+0000 7fcbe0990700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:19.389184+0000 front 2024-02-18T11:54:19.389074+0000 (oldest deadline 2024-02-18T11:54:49.989726+0000) 2024-02-18T11:54:58.424 INFO:journalctl@ceph.osd.6.smithi012.stdout:Feb 18 11:54:58 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-6[58474]: 2024-02-18T11:54:58.148+0000 7fcbe0990700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:29.990442+0000 front 2024-02-18T11:54:29.990459+0000 (oldest deadline 2024-02-18T11:54:55.290283+0000) 2024-02-18T11:54:58.424 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:54:58 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:54:58.052+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.279478+0000 front 2024-02-18T11:54:23.279513+0000 (oldest deadline 2024-02-18T11:54:49.179497+0000) 2024-02-18T11:54:58.424 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:54:58 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:54:58.052+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:29.180115+0000 front 2024-02-18T11:54:29.180243+0000 (oldest deadline 2024-02-18T11:54:52.080171+0000) 2024-02-18T11:54:58.603 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:54:58 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:54:58.291+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.164300+0000 front 2024-02-18T11:54:23.164278+0000 (oldest deadline 2024-02-18T11:54:46.664258+0000) 2024-02-18T11:54:58.603 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:54:58 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:54:58.291+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.365211+0000 front 2024-02-18T11:54:28.365390+0000 (oldest deadline 2024-02-18T11:54:53.665226+0000) 2024-02-18T11:54:58.604 INFO:journalctl@ceph.osd.5.smithi188.stdout:Feb 18 11:54:58 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-5[48668]: 2024-02-18T11:54:58.416+0000 7f7798d72700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-18T11:54:58.604 INFO:journalctl@ceph.osd.5.smithi188.stdout:Feb 18 11:54:58 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-5[48668]: 2024-02-18T11:54:58.416+0000 7f7798d72700 -1 osd.5 92 *** Got signal Terminated *** 2024-02-18T11:54:58.604 INFO:journalctl@ceph.osd.5.smithi188.stdout:Feb 18 11:54:58 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-5[48668]: 2024-02-18T11:54:58.416+0000 7f7798d72700 -1 osd.5 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-18T11:54:58.743 INFO:journalctl@ceph.osd.7.smithi158.stdout:Feb 18 11:54:58 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-7[54444]: 2024-02-18T11:54:58.383+0000 7f24cf624700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:31.308446+0000 front 2024-02-18T11:54:31.308475+0000 (oldest deadline 2024-02-18T11:54:54.808583+0000) 2024-02-18T11:54:59.289 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:54:59 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:54:59.071+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:21.400001+0000 front 2024-02-18T11:54:21.400128+0000 (oldest deadline 2024-02-18T11:54:48.400621+0000) 2024-02-18T11:54:59.289 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:54:59 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:54:59.071+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.901802+0000 front 2024-02-18T11:54:28.901977+0000 (oldest deadline 2024-02-18T11:54:53.001759+0000) 2024-02-18T11:54:59.289 INFO:journalctl@ceph.osd.5.smithi188.stdout:Feb 18 11:54:58 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-5[48668]: 2024-02-18T11:54:58.945+0000 7f7791b5f700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:20.524067+0000 front 2024-02-18T11:54:20.523988+0000 (oldest deadline 2024-02-18T11:54:45.823829+0000) 2024-02-18T11:54:59.289 INFO:journalctl@ceph.osd.5.smithi188.stdout:Feb 18 11:54:58 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-5[48668]: 2024-02-18T11:54:58.945+0000 7f7791b5f700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:29.324890+0000 front 2024-02-18T11:54:29.324961+0000 (oldest deadline 2024-02-18T11:54:54.624902+0000) 2024-02-18T11:54:59.395 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:54:59 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:54:59.136+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:22.525039+0000 front 2024-02-18T11:54:22.525199+0000 (oldest deadline 2024-02-18T11:54:46.024887+0000) 2024-02-18T11:54:59.396 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:54:59 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:54:59.136+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:31.127112+0000 front 2024-02-18T11:54:31.127151+0000 (oldest deadline 2024-02-18T11:54:54.027319+0000) 2024-02-18T11:54:59.422 INFO:journalctl@ceph.osd.6.smithi012.stdout:Feb 18 11:54:59 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-6[58474]: 2024-02-18T11:54:59.129+0000 7fcbe0990700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:19.389184+0000 front 2024-02-18T11:54:19.389074+0000 (oldest deadline 2024-02-18T11:54:49.989726+0000) 2024-02-18T11:54:59.422 INFO:journalctl@ceph.osd.6.smithi012.stdout:Feb 18 11:54:59 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-6[58474]: 2024-02-18T11:54:59.129+0000 7fcbe0990700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:29.990442+0000 front 2024-02-18T11:54:29.990459+0000 (oldest deadline 2024-02-18T11:54:55.290283+0000) 2024-02-18T11:54:59.422 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:54:59 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:54:59.022+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.279478+0000 front 2024-02-18T11:54:23.279513+0000 (oldest deadline 2024-02-18T11:54:49.179497+0000) 2024-02-18T11:54:59.423 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:54:59 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:54:59.022+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:29.180115+0000 front 2024-02-18T11:54:29.180243+0000 (oldest deadline 2024-02-18T11:54:52.080171+0000) 2024-02-18T11:54:59.603 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:54:59 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:54:59.296+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.164300+0000 front 2024-02-18T11:54:23.164278+0000 (oldest deadline 2024-02-18T11:54:46.664258+0000) 2024-02-18T11:54:59.603 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:54:59 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:54:59.296+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.365211+0000 front 2024-02-18T11:54:28.365390+0000 (oldest deadline 2024-02-18T11:54:53.665226+0000) 2024-02-18T11:54:59.743 INFO:journalctl@ceph.osd.7.smithi158.stdout:Feb 18 11:54:59 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-7[54444]: 2024-02-18T11:54:59.402+0000 7f24cf624700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:31.308446+0000 front 2024-02-18T11:54:31.308475+0000 (oldest deadline 2024-02-18T11:54:54.808583+0000) 2024-02-18T11:55:00.266 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:00 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:00.087+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:21.400001+0000 front 2024-02-18T11:54:21.400128+0000 (oldest deadline 2024-02-18T11:54:48.400621+0000) 2024-02-18T11:55:00.267 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:00 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:00.087+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.901802+0000 front 2024-02-18T11:54:28.901977+0000 (oldest deadline 2024-02-18T11:54:53.001759+0000) 2024-02-18T11:55:00.267 INFO:journalctl@ceph.osd.5.smithi188.stdout:Feb 18 11:54:59 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-5[48668]: 2024-02-18T11:54:59.961+0000 7f7791b5f700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:20.524067+0000 front 2024-02-18T11:54:20.523988+0000 (oldest deadline 2024-02-18T11:54:45.823829+0000) 2024-02-18T11:55:00.267 INFO:journalctl@ceph.osd.5.smithi188.stdout:Feb 18 11:54:59 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-5[48668]: 2024-02-18T11:54:59.961+0000 7f7791b5f700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:29.324890+0000 front 2024-02-18T11:54:29.324961+0000 (oldest deadline 2024-02-18T11:54:54.624902+0000) 2024-02-18T11:55:00.267 INFO:journalctl@ceph.osd.5.smithi188.stdout:Feb 18 11:54:59 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-5[48668]: 2024-02-18T11:54:59.961+0000 7f7791b5f700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:34.625172+0000 front 2024-02-18T11:54:34.625189+0000 (oldest deadline 2024-02-18T11:54:59.925437+0000) 2024-02-18T11:55:00.422 INFO:journalctl@ceph.osd.6.smithi012.stdout:Feb 18 11:55:00 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-6[58474]: 2024-02-18T11:55:00.141+0000 7fcbe0990700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:19.389184+0000 front 2024-02-18T11:54:19.389074+0000 (oldest deadline 2024-02-18T11:54:49.989726+0000) 2024-02-18T11:55:00.422 INFO:journalctl@ceph.osd.6.smithi012.stdout:Feb 18 11:55:00 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-6[58474]: 2024-02-18T11:55:00.141+0000 7fcbe0990700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:29.990442+0000 front 2024-02-18T11:54:29.990459+0000 (oldest deadline 2024-02-18T11:54:55.290283+0000) 2024-02-18T11:55:00.423 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:00 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:00.015+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.279478+0000 front 2024-02-18T11:54:23.279513+0000 (oldest deadline 2024-02-18T11:54:49.179497+0000) 2024-02-18T11:55:00.423 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:00 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:00.015+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:29.180115+0000 front 2024-02-18T11:54:29.180243+0000 (oldest deadline 2024-02-18T11:54:52.080171+0000) 2024-02-18T11:55:00.439 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:00 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:00.147+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:22.525039+0000 front 2024-02-18T11:54:22.525199+0000 (oldest deadline 2024-02-18T11:54:46.024887+0000) 2024-02-18T11:55:00.439 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:00 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:00.147+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:31.127112+0000 front 2024-02-18T11:54:31.127151+0000 (oldest deadline 2024-02-18T11:54:54.027319+0000) 2024-02-18T11:55:00.603 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:00 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:00.273+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.164300+0000 front 2024-02-18T11:54:23.164278+0000 (oldest deadline 2024-02-18T11:54:46.664258+0000) 2024-02-18T11:55:00.603 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:00 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:00.273+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.365211+0000 front 2024-02-18T11:54:28.365390+0000 (oldest deadline 2024-02-18T11:54:53.665226+0000) 2024-02-18T11:55:00.743 INFO:journalctl@ceph.osd.7.smithi158.stdout:Feb 18 11:55:00 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-7[54444]: 2024-02-18T11:55:00.446+0000 7f24cf624700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:31.308446+0000 front 2024-02-18T11:54:31.308475+0000 (oldest deadline 2024-02-18T11:54:54.808583+0000) 2024-02-18T11:55:01.234 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:01 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:01.102+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:21.400001+0000 front 2024-02-18T11:54:21.400128+0000 (oldest deadline 2024-02-18T11:54:48.400621+0000) 2024-02-18T11:55:01.234 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:01 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:01.102+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.901802+0000 front 2024-02-18T11:54:28.901977+0000 (oldest deadline 2024-02-18T11:54:53.001759+0000) 2024-02-18T11:55:01.234 INFO:journalctl@ceph.osd.5.smithi188.stdout:Feb 18 11:55:00 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-5[48668]: 2024-02-18T11:55:00.919+0000 7f7791b5f700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:20.524067+0000 front 2024-02-18T11:54:20.523988+0000 (oldest deadline 2024-02-18T11:54:45.823829+0000) 2024-02-18T11:55:01.234 INFO:journalctl@ceph.osd.5.smithi188.stdout:Feb 18 11:55:00 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-5[48668]: 2024-02-18T11:55:00.919+0000 7f7791b5f700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:29.324890+0000 front 2024-02-18T11:54:29.324961+0000 (oldest deadline 2024-02-18T11:54:54.624902+0000) 2024-02-18T11:55:01.234 INFO:journalctl@ceph.osd.5.smithi188.stdout:Feb 18 11:55:00 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-5[48668]: 2024-02-18T11:55:00.919+0000 7f7791b5f700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:34.625172+0000 front 2024-02-18T11:54:34.625189+0000 (oldest deadline 2024-02-18T11:54:59.925437+0000) 2024-02-18T11:55:01.422 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:01 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:01.006+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.279478+0000 front 2024-02-18T11:54:23.279513+0000 (oldest deadline 2024-02-18T11:54:49.179497+0000) 2024-02-18T11:55:01.422 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:01 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:01.006+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:29.180115+0000 front 2024-02-18T11:54:29.180243+0000 (oldest deadline 2024-02-18T11:54:52.080171+0000) 2024-02-18T11:55:01.423 INFO:journalctl@ceph.osd.6.smithi012.stdout:Feb 18 11:55:01 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-6[58474]: 2024-02-18T11:55:01.164+0000 7fcbe0990700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:19.389184+0000 front 2024-02-18T11:54:19.389074+0000 (oldest deadline 2024-02-18T11:54:49.989726+0000) 2024-02-18T11:55:01.423 INFO:journalctl@ceph.osd.6.smithi012.stdout:Feb 18 11:55:01 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-6[58474]: 2024-02-18T11:55:01.164+0000 7fcbe0990700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:29.990442+0000 front 2024-02-18T11:54:29.990459+0000 (oldest deadline 2024-02-18T11:54:55.290283+0000) 2024-02-18T11:55:01.423 INFO:journalctl@ceph.osd.6.smithi012.stdout:Feb 18 11:55:01 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-6[58474]: 2024-02-18T11:55:01.164+0000 7fcbe0990700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:36.391620+0000 front 2024-02-18T11:54:36.391503+0000 (oldest deadline 2024-02-18T11:55:01.091381+0000) 2024-02-18T11:55:01.441 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:01 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:01.121+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:22.525039+0000 front 2024-02-18T11:54:22.525199+0000 (oldest deadline 2024-02-18T11:54:46.024887+0000) 2024-02-18T11:55:01.441 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:01 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:01.121+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:31.127112+0000 front 2024-02-18T11:54:31.127151+0000 (oldest deadline 2024-02-18T11:54:54.027319+0000) 2024-02-18T11:55:01.603 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:01 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:01.241+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.164300+0000 front 2024-02-18T11:54:23.164278+0000 (oldest deadline 2024-02-18T11:54:46.664258+0000) 2024-02-18T11:55:01.603 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:01 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:01.241+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.365211+0000 front 2024-02-18T11:54:28.365390+0000 (oldest deadline 2024-02-18T11:54:53.665226+0000) 2024-02-18T11:55:01.743 INFO:journalctl@ceph.osd.7.smithi158.stdout:Feb 18 11:55:01 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-7[54444]: 2024-02-18T11:55:01.448+0000 7f24cf624700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:31.308446+0000 front 2024-02-18T11:54:31.308475+0000 (oldest deadline 2024-02-18T11:54:54.808583+0000) 2024-02-18T11:55:02.117 INFO:journalctl@ceph.osd.5.smithi188.stdout:Feb 18 11:55:01 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-5[48668]: 2024-02-18T11:55:01.870+0000 7f7791b5f700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:20.524067+0000 front 2024-02-18T11:54:20.523988+0000 (oldest deadline 2024-02-18T11:54:45.823829+0000) 2024-02-18T11:55:02.117 INFO:journalctl@ceph.osd.5.smithi188.stdout:Feb 18 11:55:01 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-5[48668]: 2024-02-18T11:55:01.870+0000 7f7791b5f700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:29.324890+0000 front 2024-02-18T11:54:29.324961+0000 (oldest deadline 2024-02-18T11:54:54.624902+0000) 2024-02-18T11:55:02.117 INFO:journalctl@ceph.osd.5.smithi188.stdout:Feb 18 11:55:01 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-5[48668]: 2024-02-18T11:55:01.870+0000 7f7791b5f700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:34.625172+0000 front 2024-02-18T11:54:34.625189+0000 (oldest deadline 2024-02-18T11:54:59.925437+0000) 2024-02-18T11:55:02.422 INFO:journalctl@ceph.osd.6.smithi012.stdout:Feb 18 11:55:02 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-6[58474]: 2024-02-18T11:55:02.163+0000 7fcbe0990700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:19.389184+0000 front 2024-02-18T11:54:19.389074+0000 (oldest deadline 2024-02-18T11:54:49.989726+0000) 2024-02-18T11:55:02.422 INFO:journalctl@ceph.osd.6.smithi012.stdout:Feb 18 11:55:02 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-6[58474]: 2024-02-18T11:55:02.163+0000 7fcbe0990700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:29.990442+0000 front 2024-02-18T11:54:29.990459+0000 (oldest deadline 2024-02-18T11:54:55.290283+0000) 2024-02-18T11:55:02.422 INFO:journalctl@ceph.osd.6.smithi012.stdout:Feb 18 11:55:02 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-6[58474]: 2024-02-18T11:55:02.163+0000 7fcbe0990700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:36.391620+0000 front 2024-02-18T11:54:36.391503+0000 (oldest deadline 2024-02-18T11:55:01.091381+0000) 2024-02-18T11:55:02.422 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:02 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:02.039+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.279478+0000 front 2024-02-18T11:54:23.279513+0000 (oldest deadline 2024-02-18T11:54:49.179497+0000) 2024-02-18T11:55:02.423 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:02 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:02.039+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:29.180115+0000 front 2024-02-18T11:54:29.180243+0000 (oldest deadline 2024-02-18T11:54:52.080171+0000) 2024-02-18T11:55:02.423 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:02 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:02.039+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:37.981514+0000 front 2024-02-18T11:54:37.981604+0000 (oldest deadline 2024-02-18T11:55:01.481436+0000) 2024-02-18T11:55:02.493 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:02 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:02.160+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:22.525039+0000 front 2024-02-18T11:54:22.525199+0000 (oldest deadline 2024-02-18T11:54:46.024887+0000) 2024-02-18T11:55:02.493 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:02 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:02.160+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:31.127112+0000 front 2024-02-18T11:54:31.127151+0000 (oldest deadline 2024-02-18T11:54:54.027319+0000) 2024-02-18T11:55:02.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:02 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:02.160+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.728559+0000 front 2024-02-18T11:54:38.728467+0000 (oldest deadline 2024-02-18T11:55:02.129139+0000) 2024-02-18T11:55:02.494 INFO:journalctl@ceph.osd.7.smithi158.stdout:Feb 18 11:55:02 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-7[54444]: 2024-02-18T11:55:02.409+0000 7f24cf624700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:31.308446+0000 front 2024-02-18T11:54:31.308475+0000 (oldest deadline 2024-02-18T11:54:54.808583+0000) 2024-02-18T11:55:02.494 INFO:journalctl@ceph.osd.7.smithi158.stdout:Feb 18 11:55:02 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-7[54444]: 2024-02-18T11:55:02.409+0000 7f24cf624700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.909685+0000 front 2024-02-18T11:54:38.909538+0000 (oldest deadline 2024-02-18T11:55:01.809623+0000) 2024-02-18T11:55:02.603 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:02 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:02.265+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.164300+0000 front 2024-02-18T11:54:23.164278+0000 (oldest deadline 2024-02-18T11:54:46.664258+0000) 2024-02-18T11:55:02.603 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:02 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:02.265+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.365211+0000 front 2024-02-18T11:54:28.365390+0000 (oldest deadline 2024-02-18T11:54:53.665226+0000) 2024-02-18T11:55:02.604 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:02 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:02.124+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:21.400001+0000 front 2024-02-18T11:54:21.400128+0000 (oldest deadline 2024-02-18T11:54:48.400621+0000) 2024-02-18T11:55:02.604 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:02 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:02.124+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.901802+0000 front 2024-02-18T11:54:28.901977+0000 (oldest deadline 2024-02-18T11:54:53.001759+0000) 2024-02-18T11:55:03.153 INFO:journalctl@ceph.osd.5.smithi188.stdout:Feb 18 11:55:02 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-5[48668]: 2024-02-18T11:55:02.878+0000 7f7791b5f700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:20.524067+0000 front 2024-02-18T11:54:20.523988+0000 (oldest deadline 2024-02-18T11:54:45.823829+0000) 2024-02-18T11:55:03.153 INFO:journalctl@ceph.osd.5.smithi188.stdout:Feb 18 11:55:02 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-5[48668]: 2024-02-18T11:55:02.878+0000 7f7791b5f700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:29.324890+0000 front 2024-02-18T11:54:29.324961+0000 (oldest deadline 2024-02-18T11:54:54.624902+0000) 2024-02-18T11:55:03.153 INFO:journalctl@ceph.osd.5.smithi188.stdout:Feb 18 11:55:02 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-5[48668]: 2024-02-18T11:55:02.878+0000 7f7791b5f700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:34.625172+0000 front 2024-02-18T11:54:34.625189+0000 (oldest deadline 2024-02-18T11:54:59.925437+0000) 2024-02-18T11:55:03.422 INFO:journalctl@ceph.osd.6.smithi012.stdout:Feb 18 11:55:03 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-6[58474]: 2024-02-18T11:55:03.176+0000 7fcbe0990700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:19.389184+0000 front 2024-02-18T11:54:19.389074+0000 (oldest deadline 2024-02-18T11:54:49.989726+0000) 2024-02-18T11:55:03.422 INFO:journalctl@ceph.osd.6.smithi012.stdout:Feb 18 11:55:03 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-6[58474]: 2024-02-18T11:55:03.176+0000 7fcbe0990700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:29.990442+0000 front 2024-02-18T11:54:29.990459+0000 (oldest deadline 2024-02-18T11:54:55.290283+0000) 2024-02-18T11:55:03.422 INFO:journalctl@ceph.osd.6.smithi012.stdout:Feb 18 11:55:03 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-6[58474]: 2024-02-18T11:55:03.176+0000 7fcbe0990700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:36.391620+0000 front 2024-02-18T11:54:36.391503+0000 (oldest deadline 2024-02-18T11:55:01.091381+0000) 2024-02-18T11:55:03.423 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:03 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:03.060+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.279478+0000 front 2024-02-18T11:54:23.279513+0000 (oldest deadline 2024-02-18T11:54:49.179497+0000) 2024-02-18T11:55:03.423 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:03 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:03.060+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:29.180115+0000 front 2024-02-18T11:54:29.180243+0000 (oldest deadline 2024-02-18T11:54:52.080171+0000) 2024-02-18T11:55:03.423 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:03 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:03.060+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:37.981514+0000 front 2024-02-18T11:54:37.981604+0000 (oldest deadline 2024-02-18T11:55:01.481436+0000) 2024-02-18T11:55:03.493 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:03 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:03.169+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:22.525039+0000 front 2024-02-18T11:54:22.525199+0000 (oldest deadline 2024-02-18T11:54:46.024887+0000) 2024-02-18T11:55:03.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:03 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:03.169+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:31.127112+0000 front 2024-02-18T11:54:31.127151+0000 (oldest deadline 2024-02-18T11:54:54.027319+0000) 2024-02-18T11:55:03.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:03 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:03.169+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.728559+0000 front 2024-02-18T11:54:38.728467+0000 (oldest deadline 2024-02-18T11:55:02.129139+0000) 2024-02-18T11:55:03.494 INFO:journalctl@ceph.osd.7.smithi158.stdout:Feb 18 11:55:03 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-7[54444]: 2024-02-18T11:55:03.402+0000 7f24cf624700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:31.308446+0000 front 2024-02-18T11:54:31.308475+0000 (oldest deadline 2024-02-18T11:54:54.808583+0000) 2024-02-18T11:55:03.494 INFO:journalctl@ceph.osd.7.smithi158.stdout:Feb 18 11:55:03 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-7[54444]: 2024-02-18T11:55:03.402+0000 7f24cf624700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.909685+0000 front 2024-02-18T11:54:38.909538+0000 (oldest deadline 2024-02-18T11:55:01.809623+0000) 2024-02-18T11:55:03.603 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:03 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:03.160+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:21.400001+0000 front 2024-02-18T11:54:21.400128+0000 (oldest deadline 2024-02-18T11:54:48.400621+0000) 2024-02-18T11:55:03.603 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:03 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:03.160+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.901802+0000 front 2024-02-18T11:54:28.901977+0000 (oldest deadline 2024-02-18T11:54:53.001759+0000) 2024-02-18T11:55:03.604 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:03 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:03.285+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.164300+0000 front 2024-02-18T11:54:23.164278+0000 (oldest deadline 2024-02-18T11:54:46.664258+0000) 2024-02-18T11:55:03.604 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:03 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:03.285+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.365211+0000 front 2024-02-18T11:54:28.365390+0000 (oldest deadline 2024-02-18T11:54:53.665226+0000) 2024-02-18T11:55:03.604 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:03 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:03.285+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:37.166118+0000 front 2024-02-18T11:54:37.166051+0000 (oldest deadline 2024-02-18T11:55:03.066093+0000) 2024-02-18T11:55:04.250 INFO:journalctl@ceph.osd.5.smithi188.stdout:Feb 18 11:55:03 smithi188 podman[73114]: 2024-02-18 11:55:03.930678969 +0000 UTC m=+5.629575745 container died 348268efc3b6c2ff2e23aeef3b7800fe1031d5cecd576a78b4581fc6ed88e20d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux ) 2024-02-18T11:55:04.251 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:04 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:04.166+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:21.400001+0000 front 2024-02-18T11:54:21.400128+0000 (oldest deadline 2024-02-18T11:54:48.400621+0000) 2024-02-18T11:55:04.251 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:04 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:04.166+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.901802+0000 front 2024-02-18T11:54:28.901977+0000 (oldest deadline 2024-02-18T11:54:53.001759+0000) 2024-02-18T11:55:04.251 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:04 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:04.166+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.302728+0000 front 2024-02-18T11:54:38.302766+0000 (oldest deadline 2024-02-18T11:55:03.602861+0000) 2024-02-18T11:55:04.411 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:04 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:04.150+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:22.525039+0000 front 2024-02-18T11:54:22.525199+0000 (oldest deadline 2024-02-18T11:54:46.024887+0000) 2024-02-18T11:55:04.411 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:04 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:04.150+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:31.127112+0000 front 2024-02-18T11:54:31.127151+0000 (oldest deadline 2024-02-18T11:54:54.027319+0000) 2024-02-18T11:55:04.411 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:04 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:04.150+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.728559+0000 front 2024-02-18T11:54:38.728467+0000 (oldest deadline 2024-02-18T11:55:02.129139+0000) 2024-02-18T11:55:04.422 INFO:journalctl@ceph.osd.6.smithi012.stdout:Feb 18 11:55:04 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-6[58474]: 2024-02-18T11:55:04.202+0000 7fcbe0990700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:19.389184+0000 front 2024-02-18T11:54:19.389074+0000 (oldest deadline 2024-02-18T11:54:49.989726+0000) 2024-02-18T11:55:04.422 INFO:journalctl@ceph.osd.6.smithi012.stdout:Feb 18 11:55:04 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-6[58474]: 2024-02-18T11:55:04.202+0000 7fcbe0990700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:29.990442+0000 front 2024-02-18T11:54:29.990459+0000 (oldest deadline 2024-02-18T11:54:55.290283+0000) 2024-02-18T11:55:04.422 INFO:journalctl@ceph.osd.6.smithi012.stdout:Feb 18 11:55:04 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-6[58474]: 2024-02-18T11:55:04.202+0000 7fcbe0990700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:36.391620+0000 front 2024-02-18T11:54:36.391503+0000 (oldest deadline 2024-02-18T11:55:01.091381+0000) 2024-02-18T11:55:04.423 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:04 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:04.062+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.279478+0000 front 2024-02-18T11:54:23.279513+0000 (oldest deadline 2024-02-18T11:54:49.179497+0000) 2024-02-18T11:55:04.423 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:04 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:04.062+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:29.180115+0000 front 2024-02-18T11:54:29.180243+0000 (oldest deadline 2024-02-18T11:54:52.080171+0000) 2024-02-18T11:55:04.423 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:04 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:04.062+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:37.981514+0000 front 2024-02-18T11:54:37.981604+0000 (oldest deadline 2024-02-18T11:55:01.481436+0000) 2024-02-18T11:55:04.603 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:04 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:04.288+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.164300+0000 front 2024-02-18T11:54:23.164278+0000 (oldest deadline 2024-02-18T11:54:46.664258+0000) 2024-02-18T11:55:04.603 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:04 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:04.288+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.365211+0000 front 2024-02-18T11:54:28.365390+0000 (oldest deadline 2024-02-18T11:54:53.665226+0000) 2024-02-18T11:55:04.603 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:04 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:04.288+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:37.166118+0000 front 2024-02-18T11:54:37.166051+0000 (oldest deadline 2024-02-18T11:55:03.066093+0000) 2024-02-18T11:55:04.604 INFO:journalctl@ceph.osd.5.smithi188.stdout:Feb 18 11:55:04 smithi188 podman[73114]: 2024-02-18 11:55:04.258185584 +0000 UTC m=+5.957082351 container cleanup 348268efc3b6c2ff2e23aeef3b7800fe1031d5cecd576a78b4581fc6ed88e20d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-02-18T11:55:04.604 INFO:journalctl@ceph.osd.5.smithi188.stdout:Feb 18 11:55:04 smithi188 bash[73114]: ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-5 2024-02-18T11:55:04.604 INFO:journalctl@ceph.osd.5.smithi188.stdout:Feb 18 11:55:04 smithi188 podman[73133]: 2024-02-18 11:55:04.475951733 +0000 UTC m=+0.549729826 container remove 348268efc3b6c2ff2e23aeef3b7800fe1031d5cecd576a78b4581fc6ed88e20d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-18T11:55:04.743 INFO:journalctl@ceph.osd.7.smithi158.stdout:Feb 18 11:55:04 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-7[54444]: 2024-02-18T11:55:04.417+0000 7f24cf624700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:31.308446+0000 front 2024-02-18T11:54:31.308475+0000 (oldest deadline 2024-02-18T11:54:54.808583+0000) 2024-02-18T11:55:04.743 INFO:journalctl@ceph.osd.7.smithi158.stdout:Feb 18 11:55:04 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-7[54444]: 2024-02-18T11:55:04.417+0000 7f24cf624700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.909685+0000 front 2024-02-18T11:54:38.909538+0000 (oldest deadline 2024-02-18T11:55:01.809623+0000) 2024-02-18T11:55:05.158 INFO:journalctl@ceph.osd.5.smithi188.stdout:Feb 18 11:55:04 smithi188 podman[73228]: 2024-02-18 11:55:04.909954274 +0000 UTC m=+0.109318477 container create 74469ae002543ec8a6961c1fda74ea84aa5d9a40da9d881d5f84e88efd52c8c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5) 2024-02-18T11:55:05.158 INFO:journalctl@ceph.osd.5.smithi188.stdout:Feb 18 11:55:04 smithi188 podman[73228]: 2024-02-18 11:55:04.81807786 +0000 UTC m=+0.017442077 image pull 53c33ee3bf62dc411b650f86eedbbee7698efb1001b4c063271dcd8d65b41165 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1 2024-02-18T11:55:05.158 INFO:journalctl@ceph.osd.5.smithi188.stdout:Feb 18 11:55:05 smithi188 podman[73228]: 2024-02-18 11:55:05.112659468 +0000 UTC m=+0.312023682 container init 74469ae002543ec8a6961c1fda74ea84aa5d9a40da9d881d5f84e88efd52c8c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-5-deactivate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-02-18T11:55:05.158 INFO:journalctl@ceph.osd.5.smithi188.stdout:Feb 18 11:55:05 smithi188 podman[73228]: 2024-02-18 11:55:05.115773575 +0000 UTC m=+0.315137779 container start 74469ae002543ec8a6961c1fda74ea84aa5d9a40da9d881d5f84e88efd52c8c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-5-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-02-18T11:55:05.401 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:05 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:05.137+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:22.525039+0000 front 2024-02-18T11:54:22.525199+0000 (oldest deadline 2024-02-18T11:54:46.024887+0000) 2024-02-18T11:55:05.401 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:05 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:05.137+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:31.127112+0000 front 2024-02-18T11:54:31.127151+0000 (oldest deadline 2024-02-18T11:54:54.027319+0000) 2024-02-18T11:55:05.401 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:05 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:05.137+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.728559+0000 front 2024-02-18T11:54:38.728467+0000 (oldest deadline 2024-02-18T11:55:02.129139+0000) 2024-02-18T11:55:05.422 INFO:journalctl@ceph.osd.6.smithi012.stdout:Feb 18 11:55:05 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-6[58474]: 2024-02-18T11:55:05.242+0000 7fcbe0990700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:19.389184+0000 front 2024-02-18T11:54:19.389074+0000 (oldest deadline 2024-02-18T11:54:49.989726+0000) 2024-02-18T11:55:05.422 INFO:journalctl@ceph.osd.6.smithi012.stdout:Feb 18 11:55:05 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-6[58474]: 2024-02-18T11:55:05.242+0000 7fcbe0990700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:29.990442+0000 front 2024-02-18T11:54:29.990459+0000 (oldest deadline 2024-02-18T11:54:55.290283+0000) 2024-02-18T11:55:05.422 INFO:journalctl@ceph.osd.6.smithi012.stdout:Feb 18 11:55:05 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-6[58474]: 2024-02-18T11:55:05.242+0000 7fcbe0990700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:36.391620+0000 front 2024-02-18T11:54:36.391503+0000 (oldest deadline 2024-02-18T11:55:01.091381+0000) 2024-02-18T11:55:05.423 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:05 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:05.071+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.279478+0000 front 2024-02-18T11:54:23.279513+0000 (oldest deadline 2024-02-18T11:54:49.179497+0000) 2024-02-18T11:55:05.423 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:05 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:05.071+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:29.180115+0000 front 2024-02-18T11:54:29.180243+0000 (oldest deadline 2024-02-18T11:54:52.080171+0000) 2024-02-18T11:55:05.423 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:05 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:05.071+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:37.981514+0000 front 2024-02-18T11:54:37.981604+0000 (oldest deadline 2024-02-18T11:55:01.481436+0000) 2024-02-18T11:55:05.603 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:05 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:05.165+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:21.400001+0000 front 2024-02-18T11:54:21.400128+0000 (oldest deadline 2024-02-18T11:54:48.400621+0000) 2024-02-18T11:55:05.603 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:05 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:05.165+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.901802+0000 front 2024-02-18T11:54:28.901977+0000 (oldest deadline 2024-02-18T11:54:53.001759+0000) 2024-02-18T11:55:05.603 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:05 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:05.165+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.302728+0000 front 2024-02-18T11:54:38.302766+0000 (oldest deadline 2024-02-18T11:55:03.602861+0000) 2024-02-18T11:55:05.604 INFO:journalctl@ceph.osd.5.smithi188.stdout:Feb 18 11:55:05 smithi188 podman[73228]: 2024-02-18 11:55:05.206853478 +0000 UTC m=+0.406217693 container attach 74469ae002543ec8a6961c1fda74ea84aa5d9a40da9d881d5f84e88efd52c8c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, org.label-schema.build-date=20240214) 2024-02-18T11:55:05.604 INFO:journalctl@ceph.osd.5.smithi188.stdout:Feb 18 11:55:05 smithi188 podman[73228]: 2024-02-18 11:55:05.400632024 +0000 UTC m=+0.599996234 container died 74469ae002543ec8a6961c1fda74ea84aa5d9a40da9d881d5f84e88efd52c8c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-5-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214) 2024-02-18T11:55:05.604 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:05 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:05.329+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.164300+0000 front 2024-02-18T11:54:23.164278+0000 (oldest deadline 2024-02-18T11:54:46.664258+0000) 2024-02-18T11:55:05.604 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:05 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:05.329+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.365211+0000 front 2024-02-18T11:54:28.365390+0000 (oldest deadline 2024-02-18T11:54:53.665226+0000) 2024-02-18T11:55:05.604 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:05 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:05.329+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:37.166118+0000 front 2024-02-18T11:54:37.166051+0000 (oldest deadline 2024-02-18T11:55:03.066093+0000) 2024-02-18T11:55:05.743 INFO:journalctl@ceph.osd.7.smithi158.stdout:Feb 18 11:55:05 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-7[54444]: 2024-02-18T11:55:05.407+0000 7f24cf624700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:31.308446+0000 front 2024-02-18T11:54:31.308475+0000 (oldest deadline 2024-02-18T11:54:54.808583+0000) 2024-02-18T11:55:05.743 INFO:journalctl@ceph.osd.7.smithi158.stdout:Feb 18 11:55:05 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-7[54444]: 2024-02-18T11:55:05.407+0000 7f24cf624700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.909685+0000 front 2024-02-18T11:54:38.909538+0000 (oldest deadline 2024-02-18T11:55:01.809623+0000) 2024-02-18T11:55:05.978 DEBUG:teuthology.orchestra.run.smithi188:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@osd.5.service' 2024-02-18T11:55:06.020 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-18T11:55:06.020 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-02-18T11:55:06.020 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-02-18T11:55:06.020 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@osd.6 2024-02-18T11:55:06.288 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:06 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:06.194+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:21.400001+0000 front 2024-02-18T11:54:21.400128+0000 (oldest deadline 2024-02-18T11:54:48.400621+0000) 2024-02-18T11:55:06.288 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:06 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:06.194+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.901802+0000 front 2024-02-18T11:54:28.901977+0000 (oldest deadline 2024-02-18T11:54:53.001759+0000) 2024-02-18T11:55:06.288 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:06 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:06.194+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.302728+0000 front 2024-02-18T11:54:38.302766+0000 (oldest deadline 2024-02-18T11:55:03.602861+0000) 2024-02-18T11:55:06.313 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:06 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:06.106+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.279478+0000 front 2024-02-18T11:54:23.279513+0000 (oldest deadline 2024-02-18T11:54:49.179497+0000) 2024-02-18T11:55:06.313 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:06 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:06.106+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:29.180115+0000 front 2024-02-18T11:54:29.180243+0000 (oldest deadline 2024-02-18T11:54:52.080171+0000) 2024-02-18T11:55:06.313 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:06 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:06.106+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:37.981514+0000 front 2024-02-18T11:54:37.981604+0000 (oldest deadline 2024-02-18T11:55:01.481436+0000) 2024-02-18T11:55:06.313 INFO:journalctl@ceph.osd.6.smithi012.stdout:Feb 18 11:55:06 smithi012 systemd[1]: Stopping Ceph osd.6 for 0c8bb0de-ce51-11ee-95ba-87774f69a715... 2024-02-18T11:55:06.313 INFO:journalctl@ceph.osd.6.smithi012.stdout:Feb 18 11:55:06 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-6[58474]: 2024-02-18T11:55:06.229+0000 7fcbe0990700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:19.389184+0000 front 2024-02-18T11:54:19.389074+0000 (oldest deadline 2024-02-18T11:54:49.989726+0000) 2024-02-18T11:55:06.314 INFO:journalctl@ceph.osd.6.smithi012.stdout:Feb 18 11:55:06 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-6[58474]: 2024-02-18T11:55:06.229+0000 7fcbe0990700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:29.990442+0000 front 2024-02-18T11:54:29.990459+0000 (oldest deadline 2024-02-18T11:54:55.290283+0000) 2024-02-18T11:55:06.314 INFO:journalctl@ceph.osd.6.smithi012.stdout:Feb 18 11:55:06 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-6[58474]: 2024-02-18T11:55:06.229+0000 7fcbe0990700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:36.391620+0000 front 2024-02-18T11:54:36.391503+0000 (oldest deadline 2024-02-18T11:55:01.091381+0000) 2024-02-18T11:55:06.417 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:06 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:06.114+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:22.525039+0000 front 2024-02-18T11:54:22.525199+0000 (oldest deadline 2024-02-18T11:54:46.024887+0000) 2024-02-18T11:55:06.417 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:06 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:06.114+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:31.127112+0000 front 2024-02-18T11:54:31.127151+0000 (oldest deadline 2024-02-18T11:54:54.027319+0000) 2024-02-18T11:55:06.417 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:06 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:06.114+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.728559+0000 front 2024-02-18T11:54:38.728467+0000 (oldest deadline 2024-02-18T11:55:02.129139+0000) 2024-02-18T11:55:06.603 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:06 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:06.295+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.164300+0000 front 2024-02-18T11:54:23.164278+0000 (oldest deadline 2024-02-18T11:54:46.664258+0000) 2024-02-18T11:55:06.603 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:06 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:06.295+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.365211+0000 front 2024-02-18T11:54:28.365390+0000 (oldest deadline 2024-02-18T11:54:53.665226+0000) 2024-02-18T11:55:06.603 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:06 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:06.295+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:37.166118+0000 front 2024-02-18T11:54:37.166051+0000 (oldest deadline 2024-02-18T11:55:03.066093+0000) 2024-02-18T11:55:06.673 INFO:journalctl@ceph.osd.6.smithi012.stdout:Feb 18 11:55:06 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-6[58474]: 2024-02-18T11:55:06.319+0000 7fcbe3b9b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-18T11:55:06.673 INFO:journalctl@ceph.osd.6.smithi012.stdout:Feb 18 11:55:06 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-6[58474]: 2024-02-18T11:55:06.319+0000 7fcbe3b9b700 -1 osd.6 92 *** Got signal Terminated *** 2024-02-18T11:55:06.673 INFO:journalctl@ceph.osd.6.smithi012.stdout:Feb 18 11:55:06 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-6[58474]: 2024-02-18T11:55:06.319+0000 7fcbe3b9b700 -1 osd.6 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-18T11:55:06.743 INFO:journalctl@ceph.osd.7.smithi158.stdout:Feb 18 11:55:06 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-7[54444]: 2024-02-18T11:55:06.423+0000 7f24cf624700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:31.308446+0000 front 2024-02-18T11:54:31.308475+0000 (oldest deadline 2024-02-18T11:54:54.808583+0000) 2024-02-18T11:55:06.743 INFO:journalctl@ceph.osd.7.smithi158.stdout:Feb 18 11:55:06 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-7[54444]: 2024-02-18T11:55:06.423+0000 7f24cf624700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.909685+0000 front 2024-02-18T11:54:38.909538+0000 (oldest deadline 2024-02-18T11:55:01.809623+0000) 2024-02-18T11:55:07.308 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:07 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:07.184+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:21.400001+0000 front 2024-02-18T11:54:21.400128+0000 (oldest deadline 2024-02-18T11:54:48.400621+0000) 2024-02-18T11:55:07.308 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:07 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:07.184+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.901802+0000 front 2024-02-18T11:54:28.901977+0000 (oldest deadline 2024-02-18T11:54:53.001759+0000) 2024-02-18T11:55:07.308 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:07 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:07.184+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.302728+0000 front 2024-02-18T11:54:38.302766+0000 (oldest deadline 2024-02-18T11:55:03.602861+0000) 2024-02-18T11:55:07.377 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:07 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:07.119+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:22.525039+0000 front 2024-02-18T11:54:22.525199+0000 (oldest deadline 2024-02-18T11:54:46.024887+0000) 2024-02-18T11:55:07.378 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:07 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:07.119+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:31.127112+0000 front 2024-02-18T11:54:31.127151+0000 (oldest deadline 2024-02-18T11:54:54.027319+0000) 2024-02-18T11:55:07.378 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:07 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:07.119+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.728559+0000 front 2024-02-18T11:54:38.728467+0000 (oldest deadline 2024-02-18T11:55:02.129139+0000) 2024-02-18T11:55:07.422 INFO:journalctl@ceph.osd.6.smithi012.stdout:Feb 18 11:55:07 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-6[58474]: 2024-02-18T11:55:07.261+0000 7fcbe0990700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:19.389184+0000 front 2024-02-18T11:54:19.389074+0000 (oldest deadline 2024-02-18T11:54:49.989726+0000) 2024-02-18T11:55:07.422 INFO:journalctl@ceph.osd.6.smithi012.stdout:Feb 18 11:55:07 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-6[58474]: 2024-02-18T11:55:07.261+0000 7fcbe0990700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:29.990442+0000 front 2024-02-18T11:54:29.990459+0000 (oldest deadline 2024-02-18T11:54:55.290283+0000) 2024-02-18T11:55:07.422 INFO:journalctl@ceph.osd.6.smithi012.stdout:Feb 18 11:55:07 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-6[58474]: 2024-02-18T11:55:07.261+0000 7fcbe0990700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:36.391620+0000 front 2024-02-18T11:54:36.391503+0000 (oldest deadline 2024-02-18T11:55:01.091381+0000) 2024-02-18T11:55:07.423 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:07 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:07.135+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.279478+0000 front 2024-02-18T11:54:23.279513+0000 (oldest deadline 2024-02-18T11:54:49.179497+0000) 2024-02-18T11:55:07.423 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:07 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:07.135+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:29.180115+0000 front 2024-02-18T11:54:29.180243+0000 (oldest deadline 2024-02-18T11:54:52.080171+0000) 2024-02-18T11:55:07.423 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:07 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:07.135+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:37.981514+0000 front 2024-02-18T11:54:37.981604+0000 (oldest deadline 2024-02-18T11:55:01.481436+0000) 2024-02-18T11:55:07.603 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:07 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:07.314+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.164300+0000 front 2024-02-18T11:54:23.164278+0000 (oldest deadline 2024-02-18T11:54:46.664258+0000) 2024-02-18T11:55:07.603 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:07 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:07.314+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.365211+0000 front 2024-02-18T11:54:28.365390+0000 (oldest deadline 2024-02-18T11:54:53.665226+0000) 2024-02-18T11:55:07.603 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:07 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:07.314+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:37.166118+0000 front 2024-02-18T11:54:37.166051+0000 (oldest deadline 2024-02-18T11:55:03.066093+0000) 2024-02-18T11:55:07.743 INFO:journalctl@ceph.osd.7.smithi158.stdout:Feb 18 11:55:07 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-7[54444]: 2024-02-18T11:55:07.384+0000 7f24cf624700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:31.308446+0000 front 2024-02-18T11:54:31.308475+0000 (oldest deadline 2024-02-18T11:54:54.808583+0000) 2024-02-18T11:55:07.744 INFO:journalctl@ceph.osd.7.smithi158.stdout:Feb 18 11:55:07 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-7[54444]: 2024-02-18T11:55:07.384+0000 7f24cf624700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.909685+0000 front 2024-02-18T11:54:38.909538+0000 (oldest deadline 2024-02-18T11:55:01.809623+0000) 2024-02-18T11:55:08.383 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:08 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:08.081+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:22.525039+0000 front 2024-02-18T11:54:22.525199+0000 (oldest deadline 2024-02-18T11:54:46.024887+0000) 2024-02-18T11:55:08.384 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:08 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:08.081+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:31.127112+0000 front 2024-02-18T11:54:31.127151+0000 (oldest deadline 2024-02-18T11:54:54.027319+0000) 2024-02-18T11:55:08.384 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:08 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:08.081+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.728559+0000 front 2024-02-18T11:54:38.728467+0000 (oldest deadline 2024-02-18T11:55:02.129139+0000) 2024-02-18T11:55:08.422 INFO:journalctl@ceph.osd.6.smithi012.stdout:Feb 18 11:55:08 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-6[58474]: 2024-02-18T11:55:08.239+0000 7fcbe0990700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:19.389184+0000 front 2024-02-18T11:54:19.389074+0000 (oldest deadline 2024-02-18T11:54:49.989726+0000) 2024-02-18T11:55:08.422 INFO:journalctl@ceph.osd.6.smithi012.stdout:Feb 18 11:55:08 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-6[58474]: 2024-02-18T11:55:08.239+0000 7fcbe0990700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:29.990442+0000 front 2024-02-18T11:54:29.990459+0000 (oldest deadline 2024-02-18T11:54:55.290283+0000) 2024-02-18T11:55:08.422 INFO:journalctl@ceph.osd.6.smithi012.stdout:Feb 18 11:55:08 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-6[58474]: 2024-02-18T11:55:08.239+0000 7fcbe0990700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:36.391620+0000 front 2024-02-18T11:54:36.391503+0000 (oldest deadline 2024-02-18T11:55:01.091381+0000) 2024-02-18T11:55:08.422 INFO:journalctl@ceph.osd.6.smithi012.stdout:Feb 18 11:55:08 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-6[58474]: 2024-02-18T11:55:08.239+0000 7fcbe0990700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:45.092547+0000 front 2024-02-18T11:54:45.092651+0000 (oldest deadline 2024-02-18T11:55:07.992806+0000) 2024-02-18T11:55:08.422 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:08 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:08.153+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.279478+0000 front 2024-02-18T11:54:23.279513+0000 (oldest deadline 2024-02-18T11:54:49.179497+0000) 2024-02-18T11:55:08.422 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:08 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:08.153+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:29.180115+0000 front 2024-02-18T11:54:29.180243+0000 (oldest deadline 2024-02-18T11:54:52.080171+0000) 2024-02-18T11:55:08.423 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:08 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:08.153+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:37.981514+0000 front 2024-02-18T11:54:37.981604+0000 (oldest deadline 2024-02-18T11:55:01.481436+0000) 2024-02-18T11:55:08.603 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:08 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:08.318+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.164300+0000 front 2024-02-18T11:54:23.164278+0000 (oldest deadline 2024-02-18T11:54:46.664258+0000) 2024-02-18T11:55:08.603 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:08 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:08.318+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.365211+0000 front 2024-02-18T11:54:28.365390+0000 (oldest deadline 2024-02-18T11:54:53.665226+0000) 2024-02-18T11:55:08.603 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:08 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:08.318+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:37.166118+0000 front 2024-02-18T11:54:37.166051+0000 (oldest deadline 2024-02-18T11:55:03.066093+0000) 2024-02-18T11:55:08.604 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:08 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:08.157+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:21.400001+0000 front 2024-02-18T11:54:21.400128+0000 (oldest deadline 2024-02-18T11:54:48.400621+0000) 2024-02-18T11:55:08.604 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:08 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:08.157+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.901802+0000 front 2024-02-18T11:54:28.901977+0000 (oldest deadline 2024-02-18T11:54:53.001759+0000) 2024-02-18T11:55:08.604 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:08 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:08.157+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.302728+0000 front 2024-02-18T11:54:38.302766+0000 (oldest deadline 2024-02-18T11:55:03.602861+0000) 2024-02-18T11:55:08.743 INFO:journalctl@ceph.osd.7.smithi158.stdout:Feb 18 11:55:08 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-7[54444]: 2024-02-18T11:55:08.390+0000 7f24cf624700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:31.308446+0000 front 2024-02-18T11:54:31.308475+0000 (oldest deadline 2024-02-18T11:54:54.808583+0000) 2024-02-18T11:55:08.743 INFO:journalctl@ceph.osd.7.smithi158.stdout:Feb 18 11:55:08 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-7[54444]: 2024-02-18T11:55:08.390+0000 7f24cf624700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.909685+0000 front 2024-02-18T11:54:38.909538+0000 (oldest deadline 2024-02-18T11:55:01.809623+0000) 2024-02-18T11:55:09.368 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:09 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:09.067+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:22.525039+0000 front 2024-02-18T11:54:22.525199+0000 (oldest deadline 2024-02-18T11:54:46.024887+0000) 2024-02-18T11:55:09.368 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:09 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:09.067+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:31.127112+0000 front 2024-02-18T11:54:31.127151+0000 (oldest deadline 2024-02-18T11:54:54.027319+0000) 2024-02-18T11:55:09.368 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:09 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:09.067+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.728559+0000 front 2024-02-18T11:54:38.728467+0000 (oldest deadline 2024-02-18T11:55:02.129139+0000) 2024-02-18T11:55:09.368 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:09 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:09.067+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:43.230285+0000 front 2024-02-18T11:54:43.230302+0000 (oldest deadline 2024-02-18T11:55:08.530478+0000) 2024-02-18T11:55:09.603 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:09 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:09.308+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.164300+0000 front 2024-02-18T11:54:23.164278+0000 (oldest deadline 2024-02-18T11:54:46.664258+0000) 2024-02-18T11:55:09.603 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:09 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:09.308+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.365211+0000 front 2024-02-18T11:54:28.365390+0000 (oldest deadline 2024-02-18T11:54:53.665226+0000) 2024-02-18T11:55:09.603 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:09 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:09.308+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:37.166118+0000 front 2024-02-18T11:54:37.166051+0000 (oldest deadline 2024-02-18T11:55:03.066093+0000) 2024-02-18T11:55:09.604 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:09 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:09.153+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:21.400001+0000 front 2024-02-18T11:54:21.400128+0000 (oldest deadline 2024-02-18T11:54:48.400621+0000) 2024-02-18T11:55:09.604 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:09 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:09.153+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.901802+0000 front 2024-02-18T11:54:28.901977+0000 (oldest deadline 2024-02-18T11:54:53.001759+0000) 2024-02-18T11:55:09.604 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:09 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:09.153+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.302728+0000 front 2024-02-18T11:54:38.302766+0000 (oldest deadline 2024-02-18T11:55:03.602861+0000) 2024-02-18T11:55:09.672 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:09 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:09.185+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.279478+0000 front 2024-02-18T11:54:23.279513+0000 (oldest deadline 2024-02-18T11:54:49.179497+0000) 2024-02-18T11:55:09.672 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:09 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:09.185+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:29.180115+0000 front 2024-02-18T11:54:29.180243+0000 (oldest deadline 2024-02-18T11:54:52.080171+0000) 2024-02-18T11:55:09.673 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:09 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:09.185+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:37.981514+0000 front 2024-02-18T11:54:37.981604+0000 (oldest deadline 2024-02-18T11:55:01.481436+0000) 2024-02-18T11:55:09.673 INFO:journalctl@ceph.osd.6.smithi012.stdout:Feb 18 11:55:09 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-6[58474]: 2024-02-18T11:55:09.261+0000 7fcbe0990700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:19.389184+0000 front 2024-02-18T11:54:19.389074+0000 (oldest deadline 2024-02-18T11:54:49.989726+0000) 2024-02-18T11:55:09.673 INFO:journalctl@ceph.osd.6.smithi012.stdout:Feb 18 11:55:09 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-6[58474]: 2024-02-18T11:55:09.261+0000 7fcbe0990700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:29.990442+0000 front 2024-02-18T11:54:29.990459+0000 (oldest deadline 2024-02-18T11:54:55.290283+0000) 2024-02-18T11:55:09.673 INFO:journalctl@ceph.osd.6.smithi012.stdout:Feb 18 11:55:09 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-6[58474]: 2024-02-18T11:55:09.261+0000 7fcbe0990700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:36.391620+0000 front 2024-02-18T11:54:36.391503+0000 (oldest deadline 2024-02-18T11:55:01.091381+0000) 2024-02-18T11:55:09.673 INFO:journalctl@ceph.osd.6.smithi012.stdout:Feb 18 11:55:09 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-6[58474]: 2024-02-18T11:55:09.261+0000 7fcbe0990700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:45.092547+0000 front 2024-02-18T11:54:45.092651+0000 (oldest deadline 2024-02-18T11:55:07.992806+0000) 2024-02-18T11:55:09.743 INFO:journalctl@ceph.osd.7.smithi158.stdout:Feb 18 11:55:09 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-7[54444]: 2024-02-18T11:55:09.373+0000 7f24cf624700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:31.308446+0000 front 2024-02-18T11:54:31.308475+0000 (oldest deadline 2024-02-18T11:54:54.808583+0000) 2024-02-18T11:55:09.744 INFO:journalctl@ceph.osd.7.smithi158.stdout:Feb 18 11:55:09 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-7[54444]: 2024-02-18T11:55:09.374+0000 7f24cf624700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.909685+0000 front 2024-02-18T11:54:38.909538+0000 (oldest deadline 2024-02-18T11:55:01.809623+0000) 2024-02-18T11:55:10.362 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:10 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:10.088+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:22.525039+0000 front 2024-02-18T11:54:22.525199+0000 (oldest deadline 2024-02-18T11:54:46.024887+0000) 2024-02-18T11:55:10.362 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:10 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:10.088+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:31.127112+0000 front 2024-02-18T11:54:31.127151+0000 (oldest deadline 2024-02-18T11:54:54.027319+0000) 2024-02-18T11:55:10.362 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:10 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:10.088+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.728559+0000 front 2024-02-18T11:54:38.728467+0000 (oldest deadline 2024-02-18T11:55:02.129139+0000) 2024-02-18T11:55:10.362 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:10 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:10.088+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:43.230285+0000 front 2024-02-18T11:54:43.230302+0000 (oldest deadline 2024-02-18T11:55:08.530478+0000) 2024-02-18T11:55:10.603 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:10 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:10.306+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.164300+0000 front 2024-02-18T11:54:23.164278+0000 (oldest deadline 2024-02-18T11:54:46.664258+0000) 2024-02-18T11:55:10.604 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:10 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:10.306+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.365211+0000 front 2024-02-18T11:54:28.365390+0000 (oldest deadline 2024-02-18T11:54:53.665226+0000) 2024-02-18T11:55:10.604 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:10 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:10.306+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:37.166118+0000 front 2024-02-18T11:54:37.166051+0000 (oldest deadline 2024-02-18T11:55:03.066093+0000) 2024-02-18T11:55:10.604 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:10 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:10.123+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:21.400001+0000 front 2024-02-18T11:54:21.400128+0000 (oldest deadline 2024-02-18T11:54:48.400621+0000) 2024-02-18T11:55:10.604 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:10 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:10.123+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.901802+0000 front 2024-02-18T11:54:28.901977+0000 (oldest deadline 2024-02-18T11:54:53.001759+0000) 2024-02-18T11:55:10.604 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:10 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:10.123+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.302728+0000 front 2024-02-18T11:54:38.302766+0000 (oldest deadline 2024-02-18T11:55:03.602861+0000) 2024-02-18T11:55:10.672 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:10 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:10.219+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.279478+0000 front 2024-02-18T11:54:23.279513+0000 (oldest deadline 2024-02-18T11:54:49.179497+0000) 2024-02-18T11:55:10.672 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:10 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:10.219+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:29.180115+0000 front 2024-02-18T11:54:29.180243+0000 (oldest deadline 2024-02-18T11:54:52.080171+0000) 2024-02-18T11:55:10.672 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:10 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:10.219+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:37.981514+0000 front 2024-02-18T11:54:37.981604+0000 (oldest deadline 2024-02-18T11:55:01.481436+0000) 2024-02-18T11:55:10.673 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:10 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:10.219+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:46.082646+0000 front 2024-02-18T11:54:46.082604+0000 (oldest deadline 2024-02-18T11:55:09.582818+0000) 2024-02-18T11:55:10.673 INFO:journalctl@ceph.osd.6.smithi012.stdout:Feb 18 11:55:10 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-6[58474]: 2024-02-18T11:55:10.280+0000 7fcbe0990700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:19.389184+0000 front 2024-02-18T11:54:19.389074+0000 (oldest deadline 2024-02-18T11:54:49.989726+0000) 2024-02-18T11:55:10.673 INFO:journalctl@ceph.osd.6.smithi012.stdout:Feb 18 11:55:10 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-6[58474]: 2024-02-18T11:55:10.280+0000 7fcbe0990700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:29.990442+0000 front 2024-02-18T11:54:29.990459+0000 (oldest deadline 2024-02-18T11:54:55.290283+0000) 2024-02-18T11:55:10.673 INFO:journalctl@ceph.osd.6.smithi012.stdout:Feb 18 11:55:10 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-6[58474]: 2024-02-18T11:55:10.280+0000 7fcbe0990700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:36.391620+0000 front 2024-02-18T11:54:36.391503+0000 (oldest deadline 2024-02-18T11:55:01.091381+0000) 2024-02-18T11:55:10.673 INFO:journalctl@ceph.osd.6.smithi012.stdout:Feb 18 11:55:10 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-6[58474]: 2024-02-18T11:55:10.280+0000 7fcbe0990700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:45.092547+0000 front 2024-02-18T11:54:45.092651+0000 (oldest deadline 2024-02-18T11:55:07.992806+0000) 2024-02-18T11:55:10.743 INFO:journalctl@ceph.osd.7.smithi158.stdout:Feb 18 11:55:10 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-7[54444]: 2024-02-18T11:55:10.368+0000 7f24cf624700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:31.308446+0000 front 2024-02-18T11:54:31.308475+0000 (oldest deadline 2024-02-18T11:54:54.808583+0000) 2024-02-18T11:55:10.744 INFO:journalctl@ceph.osd.7.smithi158.stdout:Feb 18 11:55:10 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-7[54444]: 2024-02-18T11:55:10.368+0000 7f24cf624700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.909685+0000 front 2024-02-18T11:54:38.909538+0000 (oldest deadline 2024-02-18T11:55:01.809623+0000) 2024-02-18T11:55:11.396 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:11 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:11.043+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:22.525039+0000 front 2024-02-18T11:54:22.525199+0000 (oldest deadline 2024-02-18T11:54:46.024887+0000) 2024-02-18T11:55:11.397 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:11 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:11.043+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:31.127112+0000 front 2024-02-18T11:54:31.127151+0000 (oldest deadline 2024-02-18T11:54:54.027319+0000) 2024-02-18T11:55:11.397 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:11 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:11.043+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.728559+0000 front 2024-02-18T11:54:38.728467+0000 (oldest deadline 2024-02-18T11:55:02.129139+0000) 2024-02-18T11:55:11.397 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:11 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:11.043+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:43.230285+0000 front 2024-02-18T11:54:43.230302+0000 (oldest deadline 2024-02-18T11:55:08.530478+0000) 2024-02-18T11:55:11.603 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:11 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:11.352+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.164300+0000 front 2024-02-18T11:54:23.164278+0000 (oldest deadline 2024-02-18T11:54:46.664258+0000) 2024-02-18T11:55:11.603 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:11 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:11.352+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.365211+0000 front 2024-02-18T11:54:28.365390+0000 (oldest deadline 2024-02-18T11:54:53.665226+0000) 2024-02-18T11:55:11.604 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:11 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:11.352+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:37.166118+0000 front 2024-02-18T11:54:37.166051+0000 (oldest deadline 2024-02-18T11:55:03.066093+0000) 2024-02-18T11:55:11.604 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:11 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:11.352+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:47.167264+0000 front 2024-02-18T11:54:47.167187+0000 (oldest deadline 2024-02-18T11:55:10.667175+0000) 2024-02-18T11:55:11.604 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:11 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:11.142+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:21.400001+0000 front 2024-02-18T11:54:21.400128+0000 (oldest deadline 2024-02-18T11:54:48.400621+0000) 2024-02-18T11:55:11.604 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:11 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:11.142+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.901802+0000 front 2024-02-18T11:54:28.901977+0000 (oldest deadline 2024-02-18T11:54:53.001759+0000) 2024-02-18T11:55:11.604 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:11 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:11.142+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.302728+0000 front 2024-02-18T11:54:38.302766+0000 (oldest deadline 2024-02-18T11:55:03.602861+0000) 2024-02-18T11:55:11.672 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:11 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:11.182+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.279478+0000 front 2024-02-18T11:54:23.279513+0000 (oldest deadline 2024-02-18T11:54:49.179497+0000) 2024-02-18T11:55:11.672 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:11 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:11.182+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:29.180115+0000 front 2024-02-18T11:54:29.180243+0000 (oldest deadline 2024-02-18T11:54:52.080171+0000) 2024-02-18T11:55:11.673 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:11 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:11.182+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:37.981514+0000 front 2024-02-18T11:54:37.981604+0000 (oldest deadline 2024-02-18T11:55:01.481436+0000) 2024-02-18T11:55:11.673 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:11 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:11.182+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:46.082646+0000 front 2024-02-18T11:54:46.082604+0000 (oldest deadline 2024-02-18T11:55:09.582818+0000) 2024-02-18T11:55:11.673 INFO:journalctl@ceph.osd.6.smithi012.stdout:Feb 18 11:55:11 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-6[58474]: 2024-02-18T11:55:11.291+0000 7fcbe0990700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:19.389184+0000 front 2024-02-18T11:54:19.389074+0000 (oldest deadline 2024-02-18T11:54:49.989726+0000) 2024-02-18T11:55:11.673 INFO:journalctl@ceph.osd.6.smithi012.stdout:Feb 18 11:55:11 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-6[58474]: 2024-02-18T11:55:11.291+0000 7fcbe0990700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:29.990442+0000 front 2024-02-18T11:54:29.990459+0000 (oldest deadline 2024-02-18T11:54:55.290283+0000) 2024-02-18T11:55:11.673 INFO:journalctl@ceph.osd.6.smithi012.stdout:Feb 18 11:55:11 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-6[58474]: 2024-02-18T11:55:11.291+0000 7fcbe0990700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:36.391620+0000 front 2024-02-18T11:54:36.391503+0000 (oldest deadline 2024-02-18T11:55:01.091381+0000) 2024-02-18T11:55:11.673 INFO:journalctl@ceph.osd.6.smithi012.stdout:Feb 18 11:55:11 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-6[58474]: 2024-02-18T11:55:11.291+0000 7fcbe0990700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:45.092547+0000 front 2024-02-18T11:54:45.092651+0000 (oldest deadline 2024-02-18T11:55:07.992806+0000) 2024-02-18T11:55:11.743 INFO:journalctl@ceph.osd.7.smithi158.stdout:Feb 18 11:55:11 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-7[54444]: 2024-02-18T11:55:11.403+0000 7f24cf624700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:31.308446+0000 front 2024-02-18T11:54:31.308475+0000 (oldest deadline 2024-02-18T11:54:54.808583+0000) 2024-02-18T11:55:11.743 INFO:journalctl@ceph.osd.7.smithi158.stdout:Feb 18 11:55:11 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-7[54444]: 2024-02-18T11:55:11.403+0000 7f24cf624700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.909685+0000 front 2024-02-18T11:54:38.909538+0000 (oldest deadline 2024-02-18T11:55:01.809623+0000) 2024-02-18T11:55:11.744 INFO:journalctl@ceph.osd.7.smithi158.stdout:Feb 18 11:55:11 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-7[54444]: 2024-02-18T11:55:11.403+0000 7f24cf624700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:47.011243+0000 front 2024-02-18T11:54:47.011172+0000 (oldest deadline 2024-02-18T11:55:10.511068+0000) 2024-02-18T11:55:12.013 INFO:journalctl@ceph.osd.6.smithi012.stdout:Feb 18 11:55:11 smithi012 podman[87756]: 2024-02-18 11:55:11.736177154 +0000 UTC m=+5.513693115 container died 23e6c62d1bea43d40331abfa74f52c3b4fb4713ee4c9cfab31d315ff3c215d5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-6, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5) 2024-02-18T11:55:12.355 INFO:journalctl@ceph.osd.6.smithi012.stdout:Feb 18 11:55:12 smithi012 podman[87756]: 2024-02-18 11:55:12.111085685 +0000 UTC m=+5.888601647 container cleanup 23e6c62d1bea43d40331abfa74f52c3b4fb4713ee4c9cfab31d315ff3c215d5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-6, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-18T11:55:12.355 INFO:journalctl@ceph.osd.6.smithi012.stdout:Feb 18 11:55:12 smithi012 bash[87756]: ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-6 2024-02-18T11:55:12.356 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:12 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:12.172+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.279478+0000 front 2024-02-18T11:54:23.279513+0000 (oldest deadline 2024-02-18T11:54:49.179497+0000) 2024-02-18T11:55:12.356 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:12 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:12.172+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:29.180115+0000 front 2024-02-18T11:54:29.180243+0000 (oldest deadline 2024-02-18T11:54:52.080171+0000) 2024-02-18T11:55:12.356 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:12 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:12.172+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:37.981514+0000 front 2024-02-18T11:54:37.981604+0000 (oldest deadline 2024-02-18T11:55:01.481436+0000) 2024-02-18T11:55:12.356 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:12 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:12.172+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:46.082646+0000 front 2024-02-18T11:54:46.082604+0000 (oldest deadline 2024-02-18T11:55:09.582818+0000) 2024-02-18T11:55:12.431 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:12 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:12.055+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:22.525039+0000 front 2024-02-18T11:54:22.525199+0000 (oldest deadline 2024-02-18T11:54:46.024887+0000) 2024-02-18T11:55:12.431 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:12 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:12.055+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:31.127112+0000 front 2024-02-18T11:54:31.127151+0000 (oldest deadline 2024-02-18T11:54:54.027319+0000) 2024-02-18T11:55:12.431 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:12 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:12.055+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.728559+0000 front 2024-02-18T11:54:38.728467+0000 (oldest deadline 2024-02-18T11:55:02.129139+0000) 2024-02-18T11:55:12.431 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:12 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:12.055+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:43.230285+0000 front 2024-02-18T11:54:43.230302+0000 (oldest deadline 2024-02-18T11:55:08.530478+0000) 2024-02-18T11:55:12.604 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:12 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:12.149+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:21.400001+0000 front 2024-02-18T11:54:21.400128+0000 (oldest deadline 2024-02-18T11:54:48.400621+0000) 2024-02-18T11:55:12.604 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:12 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:12.149+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.901802+0000 front 2024-02-18T11:54:28.901977+0000 (oldest deadline 2024-02-18T11:54:53.001759+0000) 2024-02-18T11:55:12.604 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:12 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:12.149+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.302728+0000 front 2024-02-18T11:54:38.302766+0000 (oldest deadline 2024-02-18T11:55:03.602861+0000) 2024-02-18T11:55:12.604 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:12 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:12.387+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.164300+0000 front 2024-02-18T11:54:23.164278+0000 (oldest deadline 2024-02-18T11:54:46.664258+0000) 2024-02-18T11:55:12.604 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:12 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:12.387+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.365211+0000 front 2024-02-18T11:54:28.365390+0000 (oldest deadline 2024-02-18T11:54:53.665226+0000) 2024-02-18T11:55:12.604 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:12 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:12.387+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:37.166118+0000 front 2024-02-18T11:54:37.166051+0000 (oldest deadline 2024-02-18T11:55:03.066093+0000) 2024-02-18T11:55:12.604 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:12 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:12.387+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:47.167264+0000 front 2024-02-18T11:54:47.167187+0000 (oldest deadline 2024-02-18T11:55:10.667175+0000) 2024-02-18T11:55:12.672 INFO:journalctl@ceph.osd.6.smithi012.stdout:Feb 18 11:55:12 smithi012 podman[87774]: 2024-02-18 11:55:12.362566761 +0000 UTC m=+0.621066078 container remove 23e6c62d1bea43d40331abfa74f52c3b4fb4713ee4c9cfab31d315ff3c215d5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-6, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240217.015652-1c920af) 2024-02-18T11:55:12.743 INFO:journalctl@ceph.osd.7.smithi158.stdout:Feb 18 11:55:12 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-7[54444]: 2024-02-18T11:55:12.437+0000 7f24cf624700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:31.308446+0000 front 2024-02-18T11:54:31.308475+0000 (oldest deadline 2024-02-18T11:54:54.808583+0000) 2024-02-18T11:55:12.744 INFO:journalctl@ceph.osd.7.smithi158.stdout:Feb 18 11:55:12 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-7[54444]: 2024-02-18T11:55:12.437+0000 7f24cf624700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.909685+0000 front 2024-02-18T11:54:38.909538+0000 (oldest deadline 2024-02-18T11:55:01.809623+0000) 2024-02-18T11:55:12.744 INFO:journalctl@ceph.osd.7.smithi158.stdout:Feb 18 11:55:12 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-7[54444]: 2024-02-18T11:55:12.437+0000 7f24cf624700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:47.011243+0000 front 2024-02-18T11:54:47.011172+0000 (oldest deadline 2024-02-18T11:55:10.511068+0000) 2024-02-18T11:55:12.975 INFO:journalctl@ceph.osd.6.smithi012.stdout:Feb 18 11:55:12 smithi012 podman[87866]: 2024-02-18 11:55:12.584844031 +0000 UTC m=+0.018036301 image pull 53c33ee3bf62dc411b650f86eedbbee7698efb1001b4c063271dcd8d65b41165 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1 2024-02-18T11:55:12.975 INFO:journalctl@ceph.osd.6.smithi012.stdout:Feb 18 11:55:12 smithi012 podman[87866]: 2024-02-18 11:55:12.696720374 +0000 UTC m=+0.129912682 container create 230f05710df6316f48df8e7db8b0bd3dd2162b8ce31a67f37a5d252050734a60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-6-deactivate, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-18T11:55:12.975 INFO:journalctl@ceph.osd.6.smithi012.stdout:Feb 18 11:55:12 smithi012 podman[87866]: 2024-02-18 11:55:12.98256459 +0000 UTC m=+0.415756856 container init 230f05710df6316f48df8e7db8b0bd3dd2162b8ce31a67f37a5d252050734a60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-6-deactivate, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-02-18T11:55:13.275 INFO:journalctl@ceph.osd.6.smithi012.stdout:Feb 18 11:55:12 smithi012 podman[87866]: 2024-02-18 11:55:12.985750651 +0000 UTC m=+0.418942927 container start 230f05710df6316f48df8e7db8b0bd3dd2162b8ce31a67f37a5d252050734a60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-6-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214) 2024-02-18T11:55:13.275 INFO:journalctl@ceph.osd.6.smithi012.stdout:Feb 18 11:55:13 smithi012 podman[87866]: 2024-02-18 11:55:13.082554109 +0000 UTC m=+0.515746377 container attach 230f05710df6316f48df8e7db8b0bd3dd2162b8ce31a67f37a5d252050734a60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-6-deactivate, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True) 2024-02-18T11:55:13.275 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:13 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:13.135+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.279478+0000 front 2024-02-18T11:54:23.279513+0000 (oldest deadline 2024-02-18T11:54:49.179497+0000) 2024-02-18T11:55:13.275 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:13 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:13.135+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:29.180115+0000 front 2024-02-18T11:54:29.180243+0000 (oldest deadline 2024-02-18T11:54:52.080171+0000) 2024-02-18T11:55:13.275 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:13 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:13.135+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:37.981514+0000 front 2024-02-18T11:54:37.981604+0000 (oldest deadline 2024-02-18T11:55:01.481436+0000) 2024-02-18T11:55:13.275 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:13 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:13.135+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:46.082646+0000 front 2024-02-18T11:54:46.082604+0000 (oldest deadline 2024-02-18T11:55:09.582818+0000) 2024-02-18T11:55:13.396 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:13 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:13.058+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:22.525039+0000 front 2024-02-18T11:54:22.525199+0000 (oldest deadline 2024-02-18T11:54:46.024887+0000) 2024-02-18T11:55:13.396 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:13 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:13.058+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:31.127112+0000 front 2024-02-18T11:54:31.127151+0000 (oldest deadline 2024-02-18T11:54:54.027319+0000) 2024-02-18T11:55:13.397 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:13 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:13.058+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.728559+0000 front 2024-02-18T11:54:38.728467+0000 (oldest deadline 2024-02-18T11:55:02.129139+0000) 2024-02-18T11:55:13.397 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:13 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:13.058+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:43.230285+0000 front 2024-02-18T11:54:43.230302+0000 (oldest deadline 2024-02-18T11:55:08.530478+0000) 2024-02-18T11:55:13.555 INFO:journalctl@ceph.osd.6.smithi012.stdout:Feb 18 11:55:13 smithi012 podman[87866]: 2024-02-18 11:55:13.282183032 +0000 UTC m=+0.715375301 container died 230f05710df6316f48df8e7db8b0bd3dd2162b8ce31a67f37a5d252050734a60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.build-date=20240214, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-18T11:55:13.603 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:13 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:13.189+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:21.400001+0000 front 2024-02-18T11:54:21.400128+0000 (oldest deadline 2024-02-18T11:54:48.400621+0000) 2024-02-18T11:55:13.604 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:13 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:13.189+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.901802+0000 front 2024-02-18T11:54:28.901977+0000 (oldest deadline 2024-02-18T11:54:53.001759+0000) 2024-02-18T11:55:13.604 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:13 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:13.189+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.302728+0000 front 2024-02-18T11:54:38.302766+0000 (oldest deadline 2024-02-18T11:55:03.602861+0000) 2024-02-18T11:55:13.604 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:13 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:13.383+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.164300+0000 front 2024-02-18T11:54:23.164278+0000 (oldest deadline 2024-02-18T11:54:46.664258+0000) 2024-02-18T11:55:13.604 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:13 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:13.383+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.365211+0000 front 2024-02-18T11:54:28.365390+0000 (oldest deadline 2024-02-18T11:54:53.665226+0000) 2024-02-18T11:55:13.604 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:13 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:13.383+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:37.166118+0000 front 2024-02-18T11:54:37.166051+0000 (oldest deadline 2024-02-18T11:55:03.066093+0000) 2024-02-18T11:55:13.604 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:13 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:13.383+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:47.167264+0000 front 2024-02-18T11:54:47.167187+0000 (oldest deadline 2024-02-18T11:55:10.667175+0000) 2024-02-18T11:55:13.744 INFO:journalctl@ceph.osd.7.smithi158.stdout:Feb 18 11:55:13 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-7[54444]: 2024-02-18T11:55:13.403+0000 7f24cf624700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:31.308446+0000 front 2024-02-18T11:54:31.308475+0000 (oldest deadline 2024-02-18T11:54:54.808583+0000) 2024-02-18T11:55:13.744 INFO:journalctl@ceph.osd.7.smithi158.stdout:Feb 18 11:55:13 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-7[54444]: 2024-02-18T11:55:13.403+0000 7f24cf624700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.909685+0000 front 2024-02-18T11:54:38.909538+0000 (oldest deadline 2024-02-18T11:55:01.809623+0000) 2024-02-18T11:55:13.744 INFO:journalctl@ceph.osd.7.smithi158.stdout:Feb 18 11:55:13 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-7[54444]: 2024-02-18T11:55:13.403+0000 7f24cf624700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:47.011243+0000 front 2024-02-18T11:54:47.011172+0000 (oldest deadline 2024-02-18T11:55:10.511068+0000) 2024-02-18T11:55:13.854 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@osd.6.service' 2024-02-18T11:55:13.895 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-18T11:55:13.895 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-02-18T11:55:13.895 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-02-18T11:55:13.896 DEBUG:teuthology.orchestra.run.smithi158:> sudo systemctl stop ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@osd.7 2024-02-18T11:55:14.218 INFO:journalctl@ceph.osd.7.smithi158.stdout:Feb 18 11:55:13 smithi158 systemd[1]: Stopping Ceph osd.7 for 0c8bb0de-ce51-11ee-95ba-87774f69a715... 2024-02-18T11:55:14.219 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:14 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:14.071+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:22.525039+0000 front 2024-02-18T11:54:22.525199+0000 (oldest deadline 2024-02-18T11:54:46.024887+0000) 2024-02-18T11:55:14.219 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:14 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:14.071+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:31.127112+0000 front 2024-02-18T11:54:31.127151+0000 (oldest deadline 2024-02-18T11:54:54.027319+0000) 2024-02-18T11:55:14.219 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:14 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:14.071+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.728559+0000 front 2024-02-18T11:54:38.728467+0000 (oldest deadline 2024-02-18T11:55:02.129139+0000) 2024-02-18T11:55:14.219 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:14 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:14.071+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:43.230285+0000 front 2024-02-18T11:54:43.230302+0000 (oldest deadline 2024-02-18T11:55:08.530478+0000) 2024-02-18T11:55:14.422 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:14 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:14.131+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.279478+0000 front 2024-02-18T11:54:23.279513+0000 (oldest deadline 2024-02-18T11:54:49.179497+0000) 2024-02-18T11:55:14.422 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:14 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:14.131+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:29.180115+0000 front 2024-02-18T11:54:29.180243+0000 (oldest deadline 2024-02-18T11:54:52.080171+0000) 2024-02-18T11:55:14.422 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:14 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:14.131+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:37.981514+0000 front 2024-02-18T11:54:37.981604+0000 (oldest deadline 2024-02-18T11:55:01.481436+0000) 2024-02-18T11:55:14.423 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:14 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:14.131+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:46.082646+0000 front 2024-02-18T11:54:46.082604+0000 (oldest deadline 2024-02-18T11:55:09.582818+0000) 2024-02-18T11:55:14.494 INFO:journalctl@ceph.osd.7.smithi158.stdout:Feb 18 11:55:14 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-7[54444]: 2024-02-18T11:55:14.224+0000 7f24d6837700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-18T11:55:14.494 INFO:journalctl@ceph.osd.7.smithi158.stdout:Feb 18 11:55:14 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-7[54444]: 2024-02-18T11:55:14.224+0000 7f24d6837700 -1 osd.7 92 *** Got signal Terminated *** 2024-02-18T11:55:14.494 INFO:journalctl@ceph.osd.7.smithi158.stdout:Feb 18 11:55:14 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-7[54444]: 2024-02-18T11:55:14.224+0000 7f24d6837700 -1 osd.7 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-18T11:55:14.494 INFO:journalctl@ceph.osd.7.smithi158.stdout:Feb 18 11:55:14 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-7[54444]: 2024-02-18T11:55:14.446+0000 7f24cf624700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:31.308446+0000 front 2024-02-18T11:54:31.308475+0000 (oldest deadline 2024-02-18T11:54:54.808583+0000) 2024-02-18T11:55:14.494 INFO:journalctl@ceph.osd.7.smithi158.stdout:Feb 18 11:55:14 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-7[54444]: 2024-02-18T11:55:14.446+0000 7f24cf624700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.909685+0000 front 2024-02-18T11:54:38.909538+0000 (oldest deadline 2024-02-18T11:55:01.809623+0000) 2024-02-18T11:55:14.494 INFO:journalctl@ceph.osd.7.smithi158.stdout:Feb 18 11:55:14 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-7[54444]: 2024-02-18T11:55:14.446+0000 7f24cf624700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:47.011243+0000 front 2024-02-18T11:54:47.011172+0000 (oldest deadline 2024-02-18T11:55:10.511068+0000) 2024-02-18T11:55:14.603 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:14 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:14.394+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.164300+0000 front 2024-02-18T11:54:23.164278+0000 (oldest deadline 2024-02-18T11:54:46.664258+0000) 2024-02-18T11:55:14.603 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:14 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:14.394+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.365211+0000 front 2024-02-18T11:54:28.365390+0000 (oldest deadline 2024-02-18T11:54:53.665226+0000) 2024-02-18T11:55:14.603 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:14 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:14.394+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:37.166118+0000 front 2024-02-18T11:54:37.166051+0000 (oldest deadline 2024-02-18T11:55:03.066093+0000) 2024-02-18T11:55:14.604 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:14 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:14.394+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:47.167264+0000 front 2024-02-18T11:54:47.167187+0000 (oldest deadline 2024-02-18T11:55:10.667175+0000) 2024-02-18T11:55:14.604 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:14 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:14.147+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:21.400001+0000 front 2024-02-18T11:54:21.400128+0000 (oldest deadline 2024-02-18T11:54:48.400621+0000) 2024-02-18T11:55:14.604 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:14 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:14.147+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.901802+0000 front 2024-02-18T11:54:28.901977+0000 (oldest deadline 2024-02-18T11:54:53.001759+0000) 2024-02-18T11:55:14.604 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:14 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:14.147+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.302728+0000 front 2024-02-18T11:54:38.302766+0000 (oldest deadline 2024-02-18T11:55:03.602861+0000) 2024-02-18T11:55:15.423 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:15 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:15.152+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.279478+0000 front 2024-02-18T11:54:23.279513+0000 (oldest deadline 2024-02-18T11:54:49.179497+0000) 2024-02-18T11:55:15.423 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:15 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:15.152+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:29.180115+0000 front 2024-02-18T11:54:29.180243+0000 (oldest deadline 2024-02-18T11:54:52.080171+0000) 2024-02-18T11:55:15.423 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:15 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:15.152+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:37.981514+0000 front 2024-02-18T11:54:37.981604+0000 (oldest deadline 2024-02-18T11:55:01.481436+0000) 2024-02-18T11:55:15.423 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:15 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:15.152+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:46.082646+0000 front 2024-02-18T11:54:46.082604+0000 (oldest deadline 2024-02-18T11:55:09.582818+0000) 2024-02-18T11:55:15.427 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:15 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:15.107+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:22.525039+0000 front 2024-02-18T11:54:22.525199+0000 (oldest deadline 2024-02-18T11:54:46.024887+0000) 2024-02-18T11:55:15.427 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:15 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:15.107+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:31.127112+0000 front 2024-02-18T11:54:31.127151+0000 (oldest deadline 2024-02-18T11:54:54.027319+0000) 2024-02-18T11:55:15.427 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:15 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:15.107+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.728559+0000 front 2024-02-18T11:54:38.728467+0000 (oldest deadline 2024-02-18T11:55:02.129139+0000) 2024-02-18T11:55:15.427 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:15 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:15.107+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:43.230285+0000 front 2024-02-18T11:54:43.230302+0000 (oldest deadline 2024-02-18T11:55:08.530478+0000) 2024-02-18T11:55:15.434 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:15 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:15.112+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:21.400001+0000 front 2024-02-18T11:54:21.400128+0000 (oldest deadline 2024-02-18T11:54:48.400621+0000) 2024-02-18T11:55:15.434 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:15 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:15.112+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.901802+0000 front 2024-02-18T11:54:28.901977+0000 (oldest deadline 2024-02-18T11:54:53.001759+0000) 2024-02-18T11:55:15.434 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:15 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:15.112+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.302728+0000 front 2024-02-18T11:54:38.302766+0000 (oldest deadline 2024-02-18T11:55:03.602861+0000) 2024-02-18T11:55:15.744 INFO:journalctl@ceph.osd.7.smithi158.stdout:Feb 18 11:55:15 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-7[54444]: 2024-02-18T11:55:15.433+0000 7f24cf624700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:31.308446+0000 front 2024-02-18T11:54:31.308475+0000 (oldest deadline 2024-02-18T11:54:54.808583+0000) 2024-02-18T11:55:15.744 INFO:journalctl@ceph.osd.7.smithi158.stdout:Feb 18 11:55:15 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-7[54444]: 2024-02-18T11:55:15.433+0000 7f24cf624700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.909685+0000 front 2024-02-18T11:54:38.909538+0000 (oldest deadline 2024-02-18T11:55:01.809623+0000) 2024-02-18T11:55:15.744 INFO:journalctl@ceph.osd.7.smithi158.stdout:Feb 18 11:55:15 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-7[54444]: 2024-02-18T11:55:15.433+0000 7f24cf624700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:47.011243+0000 front 2024-02-18T11:54:47.011172+0000 (oldest deadline 2024-02-18T11:55:10.511068+0000) 2024-02-18T11:55:15.854 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:15 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:15.441+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.164300+0000 front 2024-02-18T11:54:23.164278+0000 (oldest deadline 2024-02-18T11:54:46.664258+0000) 2024-02-18T11:55:15.854 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:15 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:15.441+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.365211+0000 front 2024-02-18T11:54:28.365390+0000 (oldest deadline 2024-02-18T11:54:53.665226+0000) 2024-02-18T11:55:15.854 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:15 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:15.441+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:37.166118+0000 front 2024-02-18T11:54:37.166051+0000 (oldest deadline 2024-02-18T11:55:03.066093+0000) 2024-02-18T11:55:15.854 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:15 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:15.441+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:47.167264+0000 front 2024-02-18T11:54:47.167187+0000 (oldest deadline 2024-02-18T11:55:10.667175+0000) 2024-02-18T11:55:16.353 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:16 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:16.104+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:21.400001+0000 front 2024-02-18T11:54:21.400128+0000 (oldest deadline 2024-02-18T11:54:48.400621+0000) 2024-02-18T11:55:16.353 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:16 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:16.104+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.901802+0000 front 2024-02-18T11:54:28.901977+0000 (oldest deadline 2024-02-18T11:54:53.001759+0000) 2024-02-18T11:55:16.353 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:16 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:16.104+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.302728+0000 front 2024-02-18T11:54:38.302766+0000 (oldest deadline 2024-02-18T11:55:03.602861+0000) 2024-02-18T11:55:16.422 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:16 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:16.165+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.279478+0000 front 2024-02-18T11:54:23.279513+0000 (oldest deadline 2024-02-18T11:54:49.179497+0000) 2024-02-18T11:55:16.423 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:16 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:16.165+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:29.180115+0000 front 2024-02-18T11:54:29.180243+0000 (oldest deadline 2024-02-18T11:54:52.080171+0000) 2024-02-18T11:55:16.423 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:16 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:16.165+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:37.981514+0000 front 2024-02-18T11:54:37.981604+0000 (oldest deadline 2024-02-18T11:55:01.481436+0000) 2024-02-18T11:55:16.423 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:16 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:16.165+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:46.082646+0000 front 2024-02-18T11:54:46.082604+0000 (oldest deadline 2024-02-18T11:55:09.582818+0000) 2024-02-18T11:55:16.432 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:16 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:16.075+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:22.525039+0000 front 2024-02-18T11:54:22.525199+0000 (oldest deadline 2024-02-18T11:54:46.024887+0000) 2024-02-18T11:55:16.432 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:16 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:16.075+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:31.127112+0000 front 2024-02-18T11:54:31.127151+0000 (oldest deadline 2024-02-18T11:54:54.027319+0000) 2024-02-18T11:55:16.432 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:16 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:16.075+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.728559+0000 front 2024-02-18T11:54:38.728467+0000 (oldest deadline 2024-02-18T11:55:02.129139+0000) 2024-02-18T11:55:16.433 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:16 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:16.075+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:43.230285+0000 front 2024-02-18T11:54:43.230302+0000 (oldest deadline 2024-02-18T11:55:08.530478+0000) 2024-02-18T11:55:16.743 INFO:journalctl@ceph.osd.7.smithi158.stdout:Feb 18 11:55:16 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-7[54444]: 2024-02-18T11:55:16.438+0000 7f24cf624700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:31.308446+0000 front 2024-02-18T11:54:31.308475+0000 (oldest deadline 2024-02-18T11:54:54.808583+0000) 2024-02-18T11:55:16.744 INFO:journalctl@ceph.osd.7.smithi158.stdout:Feb 18 11:55:16 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-7[54444]: 2024-02-18T11:55:16.438+0000 7f24cf624700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.909685+0000 front 2024-02-18T11:54:38.909538+0000 (oldest deadline 2024-02-18T11:55:01.809623+0000) 2024-02-18T11:55:16.744 INFO:journalctl@ceph.osd.7.smithi158.stdout:Feb 18 11:55:16 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-7[54444]: 2024-02-18T11:55:16.438+0000 7f24cf624700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:47.011243+0000 front 2024-02-18T11:54:47.011172+0000 (oldest deadline 2024-02-18T11:55:10.511068+0000) 2024-02-18T11:55:16.854 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:16 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:16.459+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.164300+0000 front 2024-02-18T11:54:23.164278+0000 (oldest deadline 2024-02-18T11:54:46.664258+0000) 2024-02-18T11:55:16.854 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:16 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:16.459+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.365211+0000 front 2024-02-18T11:54:28.365390+0000 (oldest deadline 2024-02-18T11:54:53.665226+0000) 2024-02-18T11:55:16.854 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:16 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:16.459+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:37.166118+0000 front 2024-02-18T11:54:37.166051+0000 (oldest deadline 2024-02-18T11:55:03.066093+0000) 2024-02-18T11:55:16.854 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:16 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:16.459+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:47.167264+0000 front 2024-02-18T11:54:47.167187+0000 (oldest deadline 2024-02-18T11:55:10.667175+0000) 2024-02-18T11:55:17.353 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:17 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:17.072+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:21.400001+0000 front 2024-02-18T11:54:21.400128+0000 (oldest deadline 2024-02-18T11:54:48.400621+0000) 2024-02-18T11:55:17.353 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:17 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:17.072+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.901802+0000 front 2024-02-18T11:54:28.901977+0000 (oldest deadline 2024-02-18T11:54:53.001759+0000) 2024-02-18T11:55:17.353 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:17 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:17.072+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.302728+0000 front 2024-02-18T11:54:38.302766+0000 (oldest deadline 2024-02-18T11:55:03.602861+0000) 2024-02-18T11:55:17.392 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:17 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:17.054+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:22.525039+0000 front 2024-02-18T11:54:22.525199+0000 (oldest deadline 2024-02-18T11:54:46.024887+0000) 2024-02-18T11:55:17.392 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:17 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:17.054+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:31.127112+0000 front 2024-02-18T11:54:31.127151+0000 (oldest deadline 2024-02-18T11:54:54.027319+0000) 2024-02-18T11:55:17.392 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:17 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:17.054+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.728559+0000 front 2024-02-18T11:54:38.728467+0000 (oldest deadline 2024-02-18T11:55:02.129139+0000) 2024-02-18T11:55:17.392 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:17 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:17.054+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:43.230285+0000 front 2024-02-18T11:54:43.230302+0000 (oldest deadline 2024-02-18T11:55:08.530478+0000) 2024-02-18T11:55:17.392 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:17 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:17.054+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.158:6812 osd.4 since back 2024-02-18T11:54:51.431536+0000 front 2024-02-18T11:54:51.431514+0000 (oldest deadline 2024-02-18T11:55:16.731790+0000) 2024-02-18T11:55:17.422 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:17 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:17.176+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.279478+0000 front 2024-02-18T11:54:23.279513+0000 (oldest deadline 2024-02-18T11:54:49.179497+0000) 2024-02-18T11:55:17.422 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:17 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:17.176+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:29.180115+0000 front 2024-02-18T11:54:29.180243+0000 (oldest deadline 2024-02-18T11:54:52.080171+0000) 2024-02-18T11:55:17.422 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:17 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:17.176+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:37.981514+0000 front 2024-02-18T11:54:37.981604+0000 (oldest deadline 2024-02-18T11:55:01.481436+0000) 2024-02-18T11:55:17.423 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:17 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:17.176+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:46.082646+0000 front 2024-02-18T11:54:46.082604+0000 (oldest deadline 2024-02-18T11:55:09.582818+0000) 2024-02-18T11:55:17.669 INFO:journalctl@ceph.osd.7.smithi158.stdout:Feb 18 11:55:17 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-7[54444]: 2024-02-18T11:55:17.398+0000 7f24cf624700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:31.308446+0000 front 2024-02-18T11:54:31.308475+0000 (oldest deadline 2024-02-18T11:54:54.808583+0000) 2024-02-18T11:55:17.669 INFO:journalctl@ceph.osd.7.smithi158.stdout:Feb 18 11:55:17 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-7[54444]: 2024-02-18T11:55:17.398+0000 7f24cf624700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.909685+0000 front 2024-02-18T11:54:38.909538+0000 (oldest deadline 2024-02-18T11:55:01.809623+0000) 2024-02-18T11:55:17.669 INFO:journalctl@ceph.osd.7.smithi158.stdout:Feb 18 11:55:17 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-7[54444]: 2024-02-18T11:55:17.398+0000 7f24cf624700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:47.011243+0000 front 2024-02-18T11:54:47.011172+0000 (oldest deadline 2024-02-18T11:55:10.511068+0000) 2024-02-18T11:55:17.853 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:17 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:17.492+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.164300+0000 front 2024-02-18T11:54:23.164278+0000 (oldest deadline 2024-02-18T11:54:46.664258+0000) 2024-02-18T11:55:17.854 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:17 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:17.492+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.365211+0000 front 2024-02-18T11:54:28.365390+0000 (oldest deadline 2024-02-18T11:54:53.665226+0000) 2024-02-18T11:55:17.854 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:17 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:17.492+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:37.166118+0000 front 2024-02-18T11:54:37.166051+0000 (oldest deadline 2024-02-18T11:55:03.066093+0000) 2024-02-18T11:55:17.854 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:17 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:17.492+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:47.167264+0000 front 2024-02-18T11:54:47.167187+0000 (oldest deadline 2024-02-18T11:55:10.667175+0000) 2024-02-18T11:55:18.357 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:18 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:18.035+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:21.400001+0000 front 2024-02-18T11:54:21.400128+0000 (oldest deadline 2024-02-18T11:54:48.400621+0000) 2024-02-18T11:55:18.357 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:18 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:18.035+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.901802+0000 front 2024-02-18T11:54:28.901977+0000 (oldest deadline 2024-02-18T11:54:53.001759+0000) 2024-02-18T11:55:18.357 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:18 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:18.035+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.302728+0000 front 2024-02-18T11:54:38.302766+0000 (oldest deadline 2024-02-18T11:55:03.602861+0000) 2024-02-18T11:55:18.357 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:18 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:18.035+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6812 osd.4 since back 2024-02-18T11:54:52.304990+0000 front 2024-02-18T11:54:52.304951+0000 (oldest deadline 2024-02-18T11:55:17.605033+0000) 2024-02-18T11:55:18.396 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:18 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:18.060+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:22.525039+0000 front 2024-02-18T11:54:22.525199+0000 (oldest deadline 2024-02-18T11:54:46.024887+0000) 2024-02-18T11:55:18.396 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:18 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:18.060+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:31.127112+0000 front 2024-02-18T11:54:31.127151+0000 (oldest deadline 2024-02-18T11:54:54.027319+0000) 2024-02-18T11:55:18.396 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:18 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:18.060+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.728559+0000 front 2024-02-18T11:54:38.728467+0000 (oldest deadline 2024-02-18T11:55:02.129139+0000) 2024-02-18T11:55:18.396 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:18 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:18.060+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:43.230285+0000 front 2024-02-18T11:54:43.230302+0000 (oldest deadline 2024-02-18T11:55:08.530478+0000) 2024-02-18T11:55:18.396 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:18 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:18.060+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.158:6812 osd.4 since back 2024-02-18T11:54:51.431536+0000 front 2024-02-18T11:54:51.431514+0000 (oldest deadline 2024-02-18T11:55:16.731790+0000) 2024-02-18T11:55:18.422 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:18 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:18.179+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.279478+0000 front 2024-02-18T11:54:23.279513+0000 (oldest deadline 2024-02-18T11:54:49.179497+0000) 2024-02-18T11:55:18.422 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:18 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:18.179+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:29.180115+0000 front 2024-02-18T11:54:29.180243+0000 (oldest deadline 2024-02-18T11:54:52.080171+0000) 2024-02-18T11:55:18.423 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:18 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:18.179+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:37.981514+0000 front 2024-02-18T11:54:37.981604+0000 (oldest deadline 2024-02-18T11:55:01.481436+0000) 2024-02-18T11:55:18.423 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:18 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:18.179+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:46.082646+0000 front 2024-02-18T11:54:46.082604+0000 (oldest deadline 2024-02-18T11:55:09.582818+0000) 2024-02-18T11:55:18.743 INFO:journalctl@ceph.osd.7.smithi158.stdout:Feb 18 11:55:18 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-7[54444]: 2024-02-18T11:55:18.401+0000 7f24cf624700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:31.308446+0000 front 2024-02-18T11:54:31.308475+0000 (oldest deadline 2024-02-18T11:54:54.808583+0000) 2024-02-18T11:55:18.744 INFO:journalctl@ceph.osd.7.smithi158.stdout:Feb 18 11:55:18 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-7[54444]: 2024-02-18T11:55:18.401+0000 7f24cf624700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.909685+0000 front 2024-02-18T11:54:38.909538+0000 (oldest deadline 2024-02-18T11:55:01.809623+0000) 2024-02-18T11:55:18.744 INFO:journalctl@ceph.osd.7.smithi158.stdout:Feb 18 11:55:18 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-7[54444]: 2024-02-18T11:55:18.401+0000 7f24cf624700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:47.011243+0000 front 2024-02-18T11:54:47.011172+0000 (oldest deadline 2024-02-18T11:55:10.511068+0000) 2024-02-18T11:55:18.853 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:18 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:18.445+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.164300+0000 front 2024-02-18T11:54:23.164278+0000 (oldest deadline 2024-02-18T11:54:46.664258+0000) 2024-02-18T11:55:18.854 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:18 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:18.445+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.365211+0000 front 2024-02-18T11:54:28.365390+0000 (oldest deadline 2024-02-18T11:54:53.665226+0000) 2024-02-18T11:55:18.854 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:18 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:18.445+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:37.166118+0000 front 2024-02-18T11:54:37.166051+0000 (oldest deadline 2024-02-18T11:55:03.066093+0000) 2024-02-18T11:55:18.854 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:18 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:18.445+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:47.167264+0000 front 2024-02-18T11:54:47.167187+0000 (oldest deadline 2024-02-18T11:55:10.667175+0000) 2024-02-18T11:55:19.353 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:19 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:19.082+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:21.400001+0000 front 2024-02-18T11:54:21.400128+0000 (oldest deadline 2024-02-18T11:54:48.400621+0000) 2024-02-18T11:55:19.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:19 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:19.082+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.901802+0000 front 2024-02-18T11:54:28.901977+0000 (oldest deadline 2024-02-18T11:54:53.001759+0000) 2024-02-18T11:55:19.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:19 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:19.082+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.302728+0000 front 2024-02-18T11:54:38.302766+0000 (oldest deadline 2024-02-18T11:55:03.602861+0000) 2024-02-18T11:55:19.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:19 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:19.082+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6812 osd.4 since back 2024-02-18T11:54:52.304990+0000 front 2024-02-18T11:54:52.304951+0000 (oldest deadline 2024-02-18T11:55:17.605033+0000) 2024-02-18T11:55:19.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:19 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:19.054+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:22.525039+0000 front 2024-02-18T11:54:22.525199+0000 (oldest deadline 2024-02-18T11:54:46.024887+0000) 2024-02-18T11:55:19.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:19 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:19.054+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:31.127112+0000 front 2024-02-18T11:54:31.127151+0000 (oldest deadline 2024-02-18T11:54:54.027319+0000) 2024-02-18T11:55:19.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:19 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:19.054+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.728559+0000 front 2024-02-18T11:54:38.728467+0000 (oldest deadline 2024-02-18T11:55:02.129139+0000) 2024-02-18T11:55:19.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:19 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:19.054+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:43.230285+0000 front 2024-02-18T11:54:43.230302+0000 (oldest deadline 2024-02-18T11:55:08.530478+0000) 2024-02-18T11:55:19.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:19 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:19.054+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.158:6812 osd.4 since back 2024-02-18T11:54:51.431536+0000 front 2024-02-18T11:54:51.431514+0000 (oldest deadline 2024-02-18T11:55:16.731790+0000) 2024-02-18T11:55:19.672 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:19 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:19.217+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.279478+0000 front 2024-02-18T11:54:23.279513+0000 (oldest deadline 2024-02-18T11:54:49.179497+0000) 2024-02-18T11:55:19.672 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:19 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:19.217+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:29.180115+0000 front 2024-02-18T11:54:29.180243+0000 (oldest deadline 2024-02-18T11:54:52.080171+0000) 2024-02-18T11:55:19.672 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:19 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:19.217+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:37.981514+0000 front 2024-02-18T11:54:37.981604+0000 (oldest deadline 2024-02-18T11:55:01.481436+0000) 2024-02-18T11:55:19.673 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:19 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:19.217+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:46.082646+0000 front 2024-02-18T11:54:46.082604+0000 (oldest deadline 2024-02-18T11:55:09.582818+0000) 2024-02-18T11:55:19.853 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:19 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:19.444+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.164300+0000 front 2024-02-18T11:54:23.164278+0000 (oldest deadline 2024-02-18T11:54:46.664258+0000) 2024-02-18T11:55:19.853 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:19 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:19.444+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.365211+0000 front 2024-02-18T11:54:28.365390+0000 (oldest deadline 2024-02-18T11:54:53.665226+0000) 2024-02-18T11:55:19.854 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:19 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:19.444+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:37.166118+0000 front 2024-02-18T11:54:37.166051+0000 (oldest deadline 2024-02-18T11:55:03.066093+0000) 2024-02-18T11:55:19.854 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:19 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:19.444+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:47.167264+0000 front 2024-02-18T11:54:47.167187+0000 (oldest deadline 2024-02-18T11:55:10.667175+0000) 2024-02-18T11:55:19.911 INFO:journalctl@ceph.osd.7.smithi158.stdout:Feb 18 11:55:19 smithi158 podman[72660]: 2024-02-18 11:55:19.641451307 +0000 UTC m=+5.543911374 container died a821390f486c9cef87be07aabdc004aad9bdf8063f003b2bd1a65ee9c70b56ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-02-18T11:55:20.244 INFO:journalctl@ceph.osd.7.smithi158.stdout:Feb 18 11:55:20 smithi158 podman[72660]: 2024-02-18 11:55:20.033591971 +0000 UTC m=+5.936052027 container cleanup a821390f486c9cef87be07aabdc004aad9bdf8063f003b2bd1a65ee9c70b56ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-02-18T11:55:20.244 INFO:journalctl@ceph.osd.7.smithi158.stdout:Feb 18 11:55:20 smithi158 bash[72660]: ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-7 2024-02-18T11:55:20.244 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:20 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:20.042+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:22.525039+0000 front 2024-02-18T11:54:22.525199+0000 (oldest deadline 2024-02-18T11:54:46.024887+0000) 2024-02-18T11:55:20.244 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:20 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:20.042+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:31.127112+0000 front 2024-02-18T11:54:31.127151+0000 (oldest deadline 2024-02-18T11:54:54.027319+0000) 2024-02-18T11:55:20.245 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:20 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:20.042+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.728559+0000 front 2024-02-18T11:54:38.728467+0000 (oldest deadline 2024-02-18T11:55:02.129139+0000) 2024-02-18T11:55:20.245 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:20 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:20.042+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:43.230285+0000 front 2024-02-18T11:54:43.230302+0000 (oldest deadline 2024-02-18T11:55:08.530478+0000) 2024-02-18T11:55:20.245 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:20 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:20.042+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.158:6812 osd.4 since back 2024-02-18T11:54:51.431536+0000 front 2024-02-18T11:54:51.431514+0000 (oldest deadline 2024-02-18T11:55:16.731790+0000) 2024-02-18T11:55:20.353 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:20 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:20.108+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:21.400001+0000 front 2024-02-18T11:54:21.400128+0000 (oldest deadline 2024-02-18T11:54:48.400621+0000) 2024-02-18T11:55:20.353 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:20 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:20.108+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.901802+0000 front 2024-02-18T11:54:28.901977+0000 (oldest deadline 2024-02-18T11:54:53.001759+0000) 2024-02-18T11:55:20.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:20 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:20.108+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.302728+0000 front 2024-02-18T11:54:38.302766+0000 (oldest deadline 2024-02-18T11:55:03.602861+0000) 2024-02-18T11:55:20.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:20 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:20.108+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6812 osd.4 since back 2024-02-18T11:54:52.304990+0000 front 2024-02-18T11:54:52.304951+0000 (oldest deadline 2024-02-18T11:55:17.605033+0000) 2024-02-18T11:55:20.554 INFO:journalctl@ceph.osd.7.smithi158.stdout:Feb 18 11:55:20 smithi158 podman[72690]: 2024-02-18 11:55:20.277363049 +0000 UTC m=+0.636748579 container remove a821390f486c9cef87be07aabdc004aad9bdf8063f003b2bd1a65ee9c70b56ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-7, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240217.015652-1c920af, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-18T11:55:20.672 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:20 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:20.204+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.279478+0000 front 2024-02-18T11:54:23.279513+0000 (oldest deadline 2024-02-18T11:54:49.179497+0000) 2024-02-18T11:55:20.672 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:20 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:20.204+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:29.180115+0000 front 2024-02-18T11:54:29.180243+0000 (oldest deadline 2024-02-18T11:54:52.080171+0000) 2024-02-18T11:55:20.673 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:20 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:20.204+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:37.981514+0000 front 2024-02-18T11:54:37.981604+0000 (oldest deadline 2024-02-18T11:55:01.481436+0000) 2024-02-18T11:55:20.673 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:20 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:20.204+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:46.082646+0000 front 2024-02-18T11:54:46.082604+0000 (oldest deadline 2024-02-18T11:55:09.582818+0000) 2024-02-18T11:55:20.839 INFO:journalctl@ceph.osd.7.smithi158.stdout:Feb 18 11:55:20 smithi158 podman[72784]: 2024-02-18 11:55:20.561027705 +0000 UTC m=+0.109479222 container create a4691210287d1a999336bea609296c5d8d25098863282756a564662a2e5dddec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-7-deactivate, GIT_CLEAN=True, ceph=True, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-02-18T11:55:20.840 INFO:journalctl@ceph.osd.7.smithi158.stdout:Feb 18 11:55:20 smithi158 podman[72784]: 2024-02-18 11:55:20.46914214 +0000 UTC m=+0.017593662 image pull 53c33ee3bf62dc411b650f86eedbbee7698efb1001b4c063271dcd8d65b41165 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1 2024-02-18T11:55:20.840 INFO:journalctl@ceph.osd.7.smithi158.stdout:Feb 18 11:55:20 smithi158 podman[72784]: 2024-02-18 11:55:20.763317702 +0000 UTC m=+0.311769230 container init a4691210287d1a999336bea609296c5d8d25098863282756a564662a2e5dddec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-7-deactivate, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, org.label-schema.license=GPLv2) 2024-02-18T11:55:20.840 INFO:journalctl@ceph.osd.7.smithi158.stdout:Feb 18 11:55:20 smithi158 podman[72784]: 2024-02-18 11:55:20.766463136 +0000 UTC m=+0.314914649 container start a4691210287d1a999336bea609296c5d8d25098863282756a564662a2e5dddec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, org.label-schema.schema-version=1.0) 2024-02-18T11:55:20.840 INFO:journalctl@ceph.osd.7.smithi158.stdout:Feb 18 11:55:20 smithi158 podman[72784]: 2024-02-18 11:55:20.846607873 +0000 UTC m=+0.395059396 container attach a4691210287d1a999336bea609296c5d8d25098863282756a564662a2e5dddec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-7-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, io.buildah.version=1.33.5, ceph=True) 2024-02-18T11:55:20.854 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:20 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:20.490+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.164300+0000 front 2024-02-18T11:54:23.164278+0000 (oldest deadline 2024-02-18T11:54:46.664258+0000) 2024-02-18T11:55:20.854 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:20 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:20.490+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.365211+0000 front 2024-02-18T11:54:28.365390+0000 (oldest deadline 2024-02-18T11:54:53.665226+0000) 2024-02-18T11:55:20.854 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:20 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:20.490+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:37.166118+0000 front 2024-02-18T11:54:37.166051+0000 (oldest deadline 2024-02-18T11:55:03.066093+0000) 2024-02-18T11:55:20.854 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:20 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:20.490+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:47.167264+0000 front 2024-02-18T11:54:47.167187+0000 (oldest deadline 2024-02-18T11:55:10.667175+0000) 2024-02-18T11:55:21.243 INFO:journalctl@ceph.osd.7.smithi158.stdout:Feb 18 11:55:21 smithi158 podman[72784]: 2024-02-18 11:55:21.055109038 +0000 UTC m=+0.603560560 container died a4691210287d1a999336bea609296c5d8d25098863282756a564662a2e5dddec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-7-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-18T11:55:21.244 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:21 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:21.040+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:22.525039+0000 front 2024-02-18T11:54:22.525199+0000 (oldest deadline 2024-02-18T11:54:46.024887+0000) 2024-02-18T11:55:21.244 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:21 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:21.040+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:31.127112+0000 front 2024-02-18T11:54:31.127151+0000 (oldest deadline 2024-02-18T11:54:54.027319+0000) 2024-02-18T11:55:21.244 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:21 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:21.040+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.728559+0000 front 2024-02-18T11:54:38.728467+0000 (oldest deadline 2024-02-18T11:55:02.129139+0000) 2024-02-18T11:55:21.244 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:21 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:21.040+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:43.230285+0000 front 2024-02-18T11:54:43.230302+0000 (oldest deadline 2024-02-18T11:55:08.530478+0000) 2024-02-18T11:55:21.244 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:21 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:21.040+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.158:6812 osd.4 since back 2024-02-18T11:54:51.431536+0000 front 2024-02-18T11:54:51.431514+0000 (oldest deadline 2024-02-18T11:55:16.731790+0000) 2024-02-18T11:55:21.353 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:21 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:21.090+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:21.400001+0000 front 2024-02-18T11:54:21.400128+0000 (oldest deadline 2024-02-18T11:54:48.400621+0000) 2024-02-18T11:55:21.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:21 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:21.090+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.901802+0000 front 2024-02-18T11:54:28.901977+0000 (oldest deadline 2024-02-18T11:54:53.001759+0000) 2024-02-18T11:55:21.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:21 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:21.090+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.302728+0000 front 2024-02-18T11:54:38.302766+0000 (oldest deadline 2024-02-18T11:55:03.602861+0000) 2024-02-18T11:55:21.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:21 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:21.090+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6812 osd.4 since back 2024-02-18T11:54:52.304990+0000 front 2024-02-18T11:54:52.304951+0000 (oldest deadline 2024-02-18T11:55:17.605033+0000) 2024-02-18T11:55:21.644 DEBUG:teuthology.orchestra.run.smithi158:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@osd.7.service' 2024-02-18T11:55:21.672 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:21 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:21.209+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.279478+0000 front 2024-02-18T11:54:23.279513+0000 (oldest deadline 2024-02-18T11:54:49.179497+0000) 2024-02-18T11:55:21.673 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:21 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:21.209+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:29.180115+0000 front 2024-02-18T11:54:29.180243+0000 (oldest deadline 2024-02-18T11:54:52.080171+0000) 2024-02-18T11:55:21.673 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:21 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:21.209+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:37.981514+0000 front 2024-02-18T11:54:37.981604+0000 (oldest deadline 2024-02-18T11:55:01.481436+0000) 2024-02-18T11:55:21.673 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:21 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:21.209+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:46.082646+0000 front 2024-02-18T11:54:46.082604+0000 (oldest deadline 2024-02-18T11:55:09.582818+0000) 2024-02-18T11:55:21.687 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-18T11:55:21.687 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-02-18T11:55:21.687 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-02-18T11:55:21.687 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@osd.8 2024-02-18T11:55:21.737 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:21 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:21.479+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.164300+0000 front 2024-02-18T11:54:23.164278+0000 (oldest deadline 2024-02-18T11:54:46.664258+0000) 2024-02-18T11:55:21.737 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:21 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:21.479+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.365211+0000 front 2024-02-18T11:54:28.365390+0000 (oldest deadline 2024-02-18T11:54:53.665226+0000) 2024-02-18T11:55:21.737 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:21 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:21.479+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:37.166118+0000 front 2024-02-18T11:54:37.166051+0000 (oldest deadline 2024-02-18T11:55:03.066093+0000) 2024-02-18T11:55:21.737 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:21 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:21.479+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:47.167264+0000 front 2024-02-18T11:54:47.167187+0000 (oldest deadline 2024-02-18T11:55:10.667175+0000) 2024-02-18T11:55:22.075 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:21 smithi188 systemd[1]: Stopping Ceph osd.8 for 0c8bb0de-ce51-11ee-95ba-87774f69a715... 2024-02-18T11:55:22.075 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:21 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:21.993+0000 7f9718bf5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-18T11:55:22.075 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:21 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:21.993+0000 7f9718bf5700 -1 osd.8 92 *** Got signal Terminated *** 2024-02-18T11:55:22.076 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:21 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:21.993+0000 7f9718bf5700 -1 osd.8 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-18T11:55:22.353 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:22 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:22.082+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:21.400001+0000 front 2024-02-18T11:54:21.400128+0000 (oldest deadline 2024-02-18T11:54:48.400621+0000) 2024-02-18T11:55:22.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:22 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:22.082+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.901802+0000 front 2024-02-18T11:54:28.901977+0000 (oldest deadline 2024-02-18T11:54:53.001759+0000) 2024-02-18T11:55:22.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:22 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:22.082+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.302728+0000 front 2024-02-18T11:54:38.302766+0000 (oldest deadline 2024-02-18T11:55:03.602861+0000) 2024-02-18T11:55:22.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:22 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:22.082+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6812 osd.4 since back 2024-02-18T11:54:52.304990+0000 front 2024-02-18T11:54:52.304951+0000 (oldest deadline 2024-02-18T11:55:17.605033+0000) 2024-02-18T11:55:22.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:22 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:22.024+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:22.525039+0000 front 2024-02-18T11:54:22.525199+0000 (oldest deadline 2024-02-18T11:54:46.024887+0000) 2024-02-18T11:55:22.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:22 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:22.024+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:31.127112+0000 front 2024-02-18T11:54:31.127151+0000 (oldest deadline 2024-02-18T11:54:54.027319+0000) 2024-02-18T11:55:22.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:22 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:22.024+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.728559+0000 front 2024-02-18T11:54:38.728467+0000 (oldest deadline 2024-02-18T11:55:02.129139+0000) 2024-02-18T11:55:22.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:22 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:22.024+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:43.230285+0000 front 2024-02-18T11:54:43.230302+0000 (oldest deadline 2024-02-18T11:55:08.530478+0000) 2024-02-18T11:55:22.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:22 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:22.024+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.158:6812 osd.4 since back 2024-02-18T11:54:51.431536+0000 front 2024-02-18T11:54:51.431514+0000 (oldest deadline 2024-02-18T11:55:16.731790+0000) 2024-02-18T11:55:22.672 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:22 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:22.188+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.279478+0000 front 2024-02-18T11:54:23.279513+0000 (oldest deadline 2024-02-18T11:54:49.179497+0000) 2024-02-18T11:55:22.672 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:22 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:22.188+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:29.180115+0000 front 2024-02-18T11:54:29.180243+0000 (oldest deadline 2024-02-18T11:54:52.080171+0000) 2024-02-18T11:55:22.673 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:22 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:22.188+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:37.981514+0000 front 2024-02-18T11:54:37.981604+0000 (oldest deadline 2024-02-18T11:55:01.481436+0000) 2024-02-18T11:55:22.673 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:22 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:22.188+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:46.082646+0000 front 2024-02-18T11:54:46.082604+0000 (oldest deadline 2024-02-18T11:55:09.582818+0000) 2024-02-18T11:55:22.853 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:22 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:22.525+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.164300+0000 front 2024-02-18T11:54:23.164278+0000 (oldest deadline 2024-02-18T11:54:46.664258+0000) 2024-02-18T11:55:22.854 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:22 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:22.525+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.365211+0000 front 2024-02-18T11:54:28.365390+0000 (oldest deadline 2024-02-18T11:54:53.665226+0000) 2024-02-18T11:55:22.854 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:22 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:22.525+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:37.166118+0000 front 2024-02-18T11:54:37.166051+0000 (oldest deadline 2024-02-18T11:55:03.066093+0000) 2024-02-18T11:55:22.854 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:22 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:22.525+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:47.167264+0000 front 2024-02-18T11:54:47.167187+0000 (oldest deadline 2024-02-18T11:55:10.667175+0000) 2024-02-18T11:55:23.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:23 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:23.009+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:22.525039+0000 front 2024-02-18T11:54:22.525199+0000 (oldest deadline 2024-02-18T11:54:46.024887+0000) 2024-02-18T11:55:23.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:23 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:23.009+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:31.127112+0000 front 2024-02-18T11:54:31.127151+0000 (oldest deadline 2024-02-18T11:54:54.027319+0000) 2024-02-18T11:55:23.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:23 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:23.009+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.728559+0000 front 2024-02-18T11:54:38.728467+0000 (oldest deadline 2024-02-18T11:55:02.129139+0000) 2024-02-18T11:55:23.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:23 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:23.009+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:43.230285+0000 front 2024-02-18T11:54:43.230302+0000 (oldest deadline 2024-02-18T11:55:08.530478+0000) 2024-02-18T11:55:23.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:23 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:23.009+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.158:6812 osd.4 since back 2024-02-18T11:54:51.431536+0000 front 2024-02-18T11:54:51.431514+0000 (oldest deadline 2024-02-18T11:55:16.731790+0000) 2024-02-18T11:55:23.552 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:23 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:23.126+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:21.400001+0000 front 2024-02-18T11:54:21.400128+0000 (oldest deadline 2024-02-18T11:54:48.400621+0000) 2024-02-18T11:55:23.552 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:23 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:23.126+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.901802+0000 front 2024-02-18T11:54:28.901977+0000 (oldest deadline 2024-02-18T11:54:53.001759+0000) 2024-02-18T11:55:23.553 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:23 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:23.126+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.302728+0000 front 2024-02-18T11:54:38.302766+0000 (oldest deadline 2024-02-18T11:55:03.602861+0000) 2024-02-18T11:55:23.553 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:23 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:23.126+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6812 osd.4 since back 2024-02-18T11:54:52.304990+0000 front 2024-02-18T11:54:52.304951+0000 (oldest deadline 2024-02-18T11:55:17.605033+0000) 2024-02-18T11:55:23.672 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:23 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:23.213+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.279478+0000 front 2024-02-18T11:54:23.279513+0000 (oldest deadline 2024-02-18T11:54:49.179497+0000) 2024-02-18T11:55:23.672 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:23 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:23.213+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:29.180115+0000 front 2024-02-18T11:54:29.180243+0000 (oldest deadline 2024-02-18T11:54:52.080171+0000) 2024-02-18T11:55:23.673 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:23 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:23.213+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:37.981514+0000 front 2024-02-18T11:54:37.981604+0000 (oldest deadline 2024-02-18T11:55:01.481436+0000) 2024-02-18T11:55:23.673 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:23 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:23.213+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:46.082646+0000 front 2024-02-18T11:54:46.082604+0000 (oldest deadline 2024-02-18T11:55:09.582818+0000) 2024-02-18T11:55:23.854 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:23 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:23.557+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.164300+0000 front 2024-02-18T11:54:23.164278+0000 (oldest deadline 2024-02-18T11:54:46.664258+0000) 2024-02-18T11:55:23.854 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:23 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:23.558+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.365211+0000 front 2024-02-18T11:54:28.365390+0000 (oldest deadline 2024-02-18T11:54:53.665226+0000) 2024-02-18T11:55:23.854 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:23 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:23.558+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:37.166118+0000 front 2024-02-18T11:54:37.166051+0000 (oldest deadline 2024-02-18T11:55:03.066093+0000) 2024-02-18T11:55:23.854 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:23 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:23.558+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:47.167264+0000 front 2024-02-18T11:54:47.167187+0000 (oldest deadline 2024-02-18T11:55:10.667175+0000) 2024-02-18T11:55:24.423 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:24 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:24.231+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.279478+0000 front 2024-02-18T11:54:23.279513+0000 (oldest deadline 2024-02-18T11:54:49.179497+0000) 2024-02-18T11:55:24.423 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:24 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:24.231+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:29.180115+0000 front 2024-02-18T11:54:29.180243+0000 (oldest deadline 2024-02-18T11:54:52.080171+0000) 2024-02-18T11:55:24.423 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:24 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:24.231+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:37.981514+0000 front 2024-02-18T11:54:37.981604+0000 (oldest deadline 2024-02-18T11:55:01.481436+0000) 2024-02-18T11:55:24.423 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:24 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:24.231+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:46.082646+0000 front 2024-02-18T11:54:46.082604+0000 (oldest deadline 2024-02-18T11:55:09.582818+0000) 2024-02-18T11:55:24.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:24 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:24.014+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:22.525039+0000 front 2024-02-18T11:54:22.525199+0000 (oldest deadline 2024-02-18T11:54:46.024887+0000) 2024-02-18T11:55:24.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:24 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:24.014+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:31.127112+0000 front 2024-02-18T11:54:31.127151+0000 (oldest deadline 2024-02-18T11:54:54.027319+0000) 2024-02-18T11:55:24.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:24 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:24.014+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.728559+0000 front 2024-02-18T11:54:38.728467+0000 (oldest deadline 2024-02-18T11:55:02.129139+0000) 2024-02-18T11:55:24.495 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:24 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:24.014+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:43.230285+0000 front 2024-02-18T11:54:43.230302+0000 (oldest deadline 2024-02-18T11:55:08.530478+0000) 2024-02-18T11:55:24.495 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:24 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:24.014+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.158:6812 osd.4 since back 2024-02-18T11:54:51.431536+0000 front 2024-02-18T11:54:51.431514+0000 (oldest deadline 2024-02-18T11:55:16.731790+0000) 2024-02-18T11:55:24.557 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:24 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:24.121+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:21.400001+0000 front 2024-02-18T11:54:21.400128+0000 (oldest deadline 2024-02-18T11:54:48.400621+0000) 2024-02-18T11:55:24.557 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:24 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:24.121+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.901802+0000 front 2024-02-18T11:54:28.901977+0000 (oldest deadline 2024-02-18T11:54:53.001759+0000) 2024-02-18T11:55:24.557 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:24 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:24.121+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.302728+0000 front 2024-02-18T11:54:38.302766+0000 (oldest deadline 2024-02-18T11:55:03.602861+0000) 2024-02-18T11:55:24.558 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:24 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:24.121+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6812 osd.4 since back 2024-02-18T11:54:52.304990+0000 front 2024-02-18T11:54:52.304951+0000 (oldest deadline 2024-02-18T11:55:17.605033+0000) 2024-02-18T11:55:24.853 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:24 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:24.563+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.164300+0000 front 2024-02-18T11:54:23.164278+0000 (oldest deadline 2024-02-18T11:54:46.664258+0000) 2024-02-18T11:55:24.854 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:24 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:24.563+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.365211+0000 front 2024-02-18T11:54:28.365390+0000 (oldest deadline 2024-02-18T11:54:53.665226+0000) 2024-02-18T11:55:24.854 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:24 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:24.563+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:37.166118+0000 front 2024-02-18T11:54:37.166051+0000 (oldest deadline 2024-02-18T11:55:03.066093+0000) 2024-02-18T11:55:24.854 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:24 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:24.563+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:47.167264+0000 front 2024-02-18T11:54:47.167187+0000 (oldest deadline 2024-02-18T11:55:10.667175+0000) 2024-02-18T11:55:25.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:25 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:25.097+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:21.400001+0000 front 2024-02-18T11:54:21.400128+0000 (oldest deadline 2024-02-18T11:54:48.400621+0000) 2024-02-18T11:55:25.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:25 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:25.097+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.901802+0000 front 2024-02-18T11:54:28.901977+0000 (oldest deadline 2024-02-18T11:54:53.001759+0000) 2024-02-18T11:55:25.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:25 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:25.097+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.302728+0000 front 2024-02-18T11:54:38.302766+0000 (oldest deadline 2024-02-18T11:55:03.602861+0000) 2024-02-18T11:55:25.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:25 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:25.097+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6812 osd.4 since back 2024-02-18T11:54:52.304990+0000 front 2024-02-18T11:54:52.304951+0000 (oldest deadline 2024-02-18T11:55:17.605033+0000) 2024-02-18T11:55:25.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:25 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:25.054+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:22.525039+0000 front 2024-02-18T11:54:22.525199+0000 (oldest deadline 2024-02-18T11:54:46.024887+0000) 2024-02-18T11:55:25.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:25 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:25.054+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:31.127112+0000 front 2024-02-18T11:54:31.127151+0000 (oldest deadline 2024-02-18T11:54:54.027319+0000) 2024-02-18T11:55:25.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:25 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:25.054+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.728559+0000 front 2024-02-18T11:54:38.728467+0000 (oldest deadline 2024-02-18T11:55:02.129139+0000) 2024-02-18T11:55:25.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:25 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:25.054+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:43.230285+0000 front 2024-02-18T11:54:43.230302+0000 (oldest deadline 2024-02-18T11:55:08.530478+0000) 2024-02-18T11:55:25.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:25 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:25.054+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.158:6812 osd.4 since back 2024-02-18T11:54:51.431536+0000 front 2024-02-18T11:54:51.431514+0000 (oldest deadline 2024-02-18T11:55:16.731790+0000) 2024-02-18T11:55:25.631 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:25 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:25.585+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.164300+0000 front 2024-02-18T11:54:23.164278+0000 (oldest deadline 2024-02-18T11:54:46.664258+0000) 2024-02-18T11:55:25.631 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:25 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:25.585+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.365211+0000 front 2024-02-18T11:54:28.365390+0000 (oldest deadline 2024-02-18T11:54:53.665226+0000) 2024-02-18T11:55:25.631 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:25 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:25.585+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:37.166118+0000 front 2024-02-18T11:54:37.166051+0000 (oldest deadline 2024-02-18T11:55:03.066093+0000) 2024-02-18T11:55:25.631 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:25 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:25.585+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:47.167264+0000 front 2024-02-18T11:54:47.167187+0000 (oldest deadline 2024-02-18T11:55:10.667175+0000) 2024-02-18T11:55:25.672 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:25 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:25.236+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.279478+0000 front 2024-02-18T11:54:23.279513+0000 (oldest deadline 2024-02-18T11:54:49.179497+0000) 2024-02-18T11:55:25.672 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:25 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:25.236+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:29.180115+0000 front 2024-02-18T11:54:29.180243+0000 (oldest deadline 2024-02-18T11:54:52.080171+0000) 2024-02-18T11:55:25.673 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:25 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:25.236+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:37.981514+0000 front 2024-02-18T11:54:37.981604+0000 (oldest deadline 2024-02-18T11:55:01.481436+0000) 2024-02-18T11:55:25.673 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:25 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:25.236+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:46.082646+0000 front 2024-02-18T11:54:46.082604+0000 (oldest deadline 2024-02-18T11:55:09.582818+0000) 2024-02-18T11:55:26.353 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:26 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:26.085+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:21.400001+0000 front 2024-02-18T11:54:21.400128+0000 (oldest deadline 2024-02-18T11:54:48.400621+0000) 2024-02-18T11:55:26.353 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:26 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:26.085+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.901802+0000 front 2024-02-18T11:54:28.901977+0000 (oldest deadline 2024-02-18T11:54:53.001759+0000) 2024-02-18T11:55:26.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:26 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:26.085+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.302728+0000 front 2024-02-18T11:54:38.302766+0000 (oldest deadline 2024-02-18T11:55:03.602861+0000) 2024-02-18T11:55:26.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:26 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:26.085+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6812 osd.4 since back 2024-02-18T11:54:52.304990+0000 front 2024-02-18T11:54:52.304951+0000 (oldest deadline 2024-02-18T11:55:17.605033+0000) 2024-02-18T11:55:26.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:26 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:26.068+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:22.525039+0000 front 2024-02-18T11:54:22.525199+0000 (oldest deadline 2024-02-18T11:54:46.024887+0000) 2024-02-18T11:55:26.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:26 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:26.068+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:31.127112+0000 front 2024-02-18T11:54:31.127151+0000 (oldest deadline 2024-02-18T11:54:54.027319+0000) 2024-02-18T11:55:26.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:26 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:26.068+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.728559+0000 front 2024-02-18T11:54:38.728467+0000 (oldest deadline 2024-02-18T11:55:02.129139+0000) 2024-02-18T11:55:26.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:26 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:26.068+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:43.230285+0000 front 2024-02-18T11:54:43.230302+0000 (oldest deadline 2024-02-18T11:55:08.530478+0000) 2024-02-18T11:55:26.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:26 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:26.068+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.158:6812 osd.4 since back 2024-02-18T11:54:51.431536+0000 front 2024-02-18T11:54:51.431514+0000 (oldest deadline 2024-02-18T11:55:16.731790+0000) 2024-02-18T11:55:26.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:26 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:26.068+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-02-18T11:55:00.133598+0000 front 2024-02-18T11:55:00.133537+0000 (oldest deadline 2024-02-18T11:55:25.433398+0000) 2024-02-18T11:55:26.672 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:26 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:26.266+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.279478+0000 front 2024-02-18T11:54:23.279513+0000 (oldest deadline 2024-02-18T11:54:49.179497+0000) 2024-02-18T11:55:26.672 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:26 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:26.266+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:29.180115+0000 front 2024-02-18T11:54:29.180243+0000 (oldest deadline 2024-02-18T11:54:52.080171+0000) 2024-02-18T11:55:26.673 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:26 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:26.266+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:37.981514+0000 front 2024-02-18T11:54:37.981604+0000 (oldest deadline 2024-02-18T11:55:01.481436+0000) 2024-02-18T11:55:26.673 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:26 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:26.266+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:46.082646+0000 front 2024-02-18T11:54:46.082604+0000 (oldest deadline 2024-02-18T11:55:09.582818+0000) 2024-02-18T11:55:26.854 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:26 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:26.583+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.164300+0000 front 2024-02-18T11:54:23.164278+0000 (oldest deadline 2024-02-18T11:54:46.664258+0000) 2024-02-18T11:55:26.854 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:26 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:26.583+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.365211+0000 front 2024-02-18T11:54:28.365390+0000 (oldest deadline 2024-02-18T11:54:53.665226+0000) 2024-02-18T11:55:26.854 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:26 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:26.583+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:37.166118+0000 front 2024-02-18T11:54:37.166051+0000 (oldest deadline 2024-02-18T11:55:03.066093+0000) 2024-02-18T11:55:26.854 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:26 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:26.583+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:47.167264+0000 front 2024-02-18T11:54:47.167187+0000 (oldest deadline 2024-02-18T11:55:10.667175+0000) 2024-02-18T11:55:26.854 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:26 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8[55273]: 2024-02-18T11:55:26.583+0000 7f97119e2700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-02-18T11:55:02.169777+0000 front 2024-02-18T11:55:02.169843+0000 (oldest deadline 2024-02-18T11:55:26.269953+0000) 2024-02-18T11:55:27.353 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:27 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:27.043+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:21.400001+0000 front 2024-02-18T11:54:21.400128+0000 (oldest deadline 2024-02-18T11:54:48.400621+0000) 2024-02-18T11:55:27.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:27 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:27.043+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.901802+0000 front 2024-02-18T11:54:28.901977+0000 (oldest deadline 2024-02-18T11:54:53.001759+0000) 2024-02-18T11:55:27.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:27 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:27.043+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.302728+0000 front 2024-02-18T11:54:38.302766+0000 (oldest deadline 2024-02-18T11:55:03.602861+0000) 2024-02-18T11:55:27.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:27 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:27.043+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6812 osd.4 since back 2024-02-18T11:54:52.304990+0000 front 2024-02-18T11:54:52.304951+0000 (oldest deadline 2024-02-18T11:55:17.605033+0000) 2024-02-18T11:55:27.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:27 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:27.046+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:22.525039+0000 front 2024-02-18T11:54:22.525199+0000 (oldest deadline 2024-02-18T11:54:46.024887+0000) 2024-02-18T11:55:27.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:27 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:27.046+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:31.127112+0000 front 2024-02-18T11:54:31.127151+0000 (oldest deadline 2024-02-18T11:54:54.027319+0000) 2024-02-18T11:55:27.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:27 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:27.047+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.728559+0000 front 2024-02-18T11:54:38.728467+0000 (oldest deadline 2024-02-18T11:55:02.129139+0000) 2024-02-18T11:55:27.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:27 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:27.047+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:43.230285+0000 front 2024-02-18T11:54:43.230302+0000 (oldest deadline 2024-02-18T11:55:08.530478+0000) 2024-02-18T11:55:27.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:27 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:27.047+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.158:6812 osd.4 since back 2024-02-18T11:54:51.431536+0000 front 2024-02-18T11:54:51.431514+0000 (oldest deadline 2024-02-18T11:55:16.731790+0000) 2024-02-18T11:55:27.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:27 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:27.047+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-02-18T11:55:00.133598+0000 front 2024-02-18T11:55:00.133537+0000 (oldest deadline 2024-02-18T11:55:25.433398+0000) 2024-02-18T11:55:27.673 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:27 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:27.223+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.279478+0000 front 2024-02-18T11:54:23.279513+0000 (oldest deadline 2024-02-18T11:54:49.179497+0000) 2024-02-18T11:55:27.673 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:27 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:27.223+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:29.180115+0000 front 2024-02-18T11:54:29.180243+0000 (oldest deadline 2024-02-18T11:54:52.080171+0000) 2024-02-18T11:55:27.673 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:27 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:27.223+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:37.981514+0000 front 2024-02-18T11:54:37.981604+0000 (oldest deadline 2024-02-18T11:55:01.481436+0000) 2024-02-18T11:55:27.673 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:27 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:27.223+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:46.082646+0000 front 2024-02-18T11:54:46.082604+0000 (oldest deadline 2024-02-18T11:55:09.582818+0000) 2024-02-18T11:55:27.838 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:27 smithi188 podman[73334]: 2024-02-18 11:55:27.562584412 +0000 UTC m=+5.675464484 container died 0af07b138499a221796e166b9e2b43074a732db69c1aea655cfdfd94b22b68e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-02-18T11:55:28.104 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:27 smithi188 podman[73334]: 2024-02-18 11:55:27.93573619 +0000 UTC m=+6.048616254 container cleanup 0af07b138499a221796e166b9e2b43074a732db69c1aea655cfdfd94b22b68e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-02-18T11:55:28.104 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:27 smithi188 bash[73334]: ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8 2024-02-18T11:55:28.104 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:28 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:28.011+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:21.400001+0000 front 2024-02-18T11:54:21.400128+0000 (oldest deadline 2024-02-18T11:54:48.400621+0000) 2024-02-18T11:55:28.104 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:28 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:28.011+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.901802+0000 front 2024-02-18T11:54:28.901977+0000 (oldest deadline 2024-02-18T11:54:53.001759+0000) 2024-02-18T11:55:28.104 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:28 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:28.011+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.302728+0000 front 2024-02-18T11:54:38.302766+0000 (oldest deadline 2024-02-18T11:55:03.602861+0000) 2024-02-18T11:55:28.104 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:28 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:28.011+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6812 osd.4 since back 2024-02-18T11:54:52.304990+0000 front 2024-02-18T11:54:52.304951+0000 (oldest deadline 2024-02-18T11:55:17.605033+0000) 2024-02-18T11:55:28.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:28 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:28.008+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:22.525039+0000 front 2024-02-18T11:54:22.525199+0000 (oldest deadline 2024-02-18T11:54:46.024887+0000) 2024-02-18T11:55:28.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:28 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:28.008+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:31.127112+0000 front 2024-02-18T11:54:31.127151+0000 (oldest deadline 2024-02-18T11:54:54.027319+0000) 2024-02-18T11:55:28.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:28 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:28.008+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.728559+0000 front 2024-02-18T11:54:38.728467+0000 (oldest deadline 2024-02-18T11:55:02.129139+0000) 2024-02-18T11:55:28.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:28 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:28.008+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:43.230285+0000 front 2024-02-18T11:54:43.230302+0000 (oldest deadline 2024-02-18T11:55:08.530478+0000) 2024-02-18T11:55:28.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:28 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:28.008+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.158:6812 osd.4 since back 2024-02-18T11:54:51.431536+0000 front 2024-02-18T11:54:51.431514+0000 (oldest deadline 2024-02-18T11:55:16.731790+0000) 2024-02-18T11:55:28.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:28 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:28.008+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-02-18T11:55:00.133598+0000 front 2024-02-18T11:55:00.133537+0000 (oldest deadline 2024-02-18T11:55:25.433398+0000) 2024-02-18T11:55:28.532 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:28 smithi188 podman[73360]: 2024-02-18 11:55:28.220404519 +0000 UTC m=+0.661821355 container remove 0af07b138499a221796e166b9e2b43074a732db69c1aea655cfdfd94b22b68e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2) 2024-02-18T11:55:28.672 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:28 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:28.210+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.279478+0000 front 2024-02-18T11:54:23.279513+0000 (oldest deadline 2024-02-18T11:54:49.179497+0000) 2024-02-18T11:55:28.673 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:28 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:28.210+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:29.180115+0000 front 2024-02-18T11:54:29.180243+0000 (oldest deadline 2024-02-18T11:54:52.080171+0000) 2024-02-18T11:55:28.673 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:28 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:28.210+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:37.981514+0000 front 2024-02-18T11:54:37.981604+0000 (oldest deadline 2024-02-18T11:55:01.481436+0000) 2024-02-18T11:55:28.673 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:28 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:28.210+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:46.082646+0000 front 2024-02-18T11:54:46.082604+0000 (oldest deadline 2024-02-18T11:55:09.582818+0000) 2024-02-18T11:55:28.833 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:28 smithi188 podman[73451]: 2024-02-18 11:55:28.441957635 +0000 UTC m=+0.017852183 image pull 53c33ee3bf62dc411b650f86eedbbee7698efb1001b4c063271dcd8d65b41165 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1 2024-02-18T11:55:28.833 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:28 smithi188 podman[73451]: 2024-02-18 11:55:28.62951088 +0000 UTC m=+0.205405417 container create 861dfd278dfbc31af8a0642e059f5863a2e6ac1a8d4b52d2626519f086578098 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-18T11:55:28.833 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:28 smithi188 podman[73451]: 2024-02-18 11:55:28.840094247 +0000 UTC m=+0.415988799 container init 861dfd278dfbc31af8a0642e059f5863a2e6ac1a8d4b52d2626519f086578098 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8-deactivate, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-18T11:55:29.104 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:28 smithi188 podman[73451]: 2024-02-18 11:55:28.843233804 +0000 UTC m=+0.419128341 container start 861dfd278dfbc31af8a0642e059f5863a2e6ac1a8d4b52d2626519f086578098 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True) 2024-02-18T11:55:29.104 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:28 smithi188 podman[73451]: 2024-02-18 11:55:28.931723382 +0000 UTC m=+0.507617921 container attach 861dfd278dfbc31af8a0642e059f5863a2e6ac1a8d4b52d2626519f086578098 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-02-18T11:55:29.105 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:29 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:29.012+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:21.400001+0000 front 2024-02-18T11:54:21.400128+0000 (oldest deadline 2024-02-18T11:54:48.400621+0000) 2024-02-18T11:55:29.105 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:29 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:29.013+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.901802+0000 front 2024-02-18T11:54:28.901977+0000 (oldest deadline 2024-02-18T11:54:53.001759+0000) 2024-02-18T11:55:29.105 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:29 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:29.013+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.302728+0000 front 2024-02-18T11:54:38.302766+0000 (oldest deadline 2024-02-18T11:55:03.602861+0000) 2024-02-18T11:55:29.105 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:29 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:29.013+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6812 osd.4 since back 2024-02-18T11:54:52.304990+0000 front 2024-02-18T11:54:52.304951+0000 (oldest deadline 2024-02-18T11:55:17.605033+0000) 2024-02-18T11:55:29.419 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:29 smithi188 podman[73451]: 2024-02-18 11:55:29.133297279 +0000 UTC m=+0.709191817 container died 861dfd278dfbc31af8a0642e059f5863a2e6ac1a8d4b52d2626519f086578098 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-vshankar-testing-20240217.015652-1c920af) 2024-02-18T11:55:29.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:29 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:29.017+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:22.525039+0000 front 2024-02-18T11:54:22.525199+0000 (oldest deadline 2024-02-18T11:54:46.024887+0000) 2024-02-18T11:55:29.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:29 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:29.017+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:31.127112+0000 front 2024-02-18T11:54:31.127151+0000 (oldest deadline 2024-02-18T11:54:54.027319+0000) 2024-02-18T11:55:29.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:29 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:29.017+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.728559+0000 front 2024-02-18T11:54:38.728467+0000 (oldest deadline 2024-02-18T11:55:02.129139+0000) 2024-02-18T11:55:29.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:29 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:29.017+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:43.230285+0000 front 2024-02-18T11:54:43.230302+0000 (oldest deadline 2024-02-18T11:55:08.530478+0000) 2024-02-18T11:55:29.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:29 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:29.017+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.158:6812 osd.4 since back 2024-02-18T11:54:51.431536+0000 front 2024-02-18T11:54:51.431514+0000 (oldest deadline 2024-02-18T11:55:16.731790+0000) 2024-02-18T11:55:29.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:29 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:29.017+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-02-18T11:55:00.133598+0000 front 2024-02-18T11:55:00.133537+0000 (oldest deadline 2024-02-18T11:55:25.433398+0000) 2024-02-18T11:55:29.670 INFO:journalctl@ceph.osd.8.smithi188.stdout:Feb 18 11:55:29 smithi188 podman[73478]: 2024-02-18 11:55:29.659982192 +0000 UTC m=+0.521398396 container remove 861dfd278dfbc31af8a0642e059f5863a2e6ac1a8d4b52d2626519f086578098 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-8-deactivate, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS) 2024-02-18T11:55:29.672 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:29 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:29.241+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.279478+0000 front 2024-02-18T11:54:23.279513+0000 (oldest deadline 2024-02-18T11:54:49.179497+0000) 2024-02-18T11:55:29.672 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:29 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:29.241+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:29.180115+0000 front 2024-02-18T11:54:29.180243+0000 (oldest deadline 2024-02-18T11:54:52.080171+0000) 2024-02-18T11:55:29.673 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:29 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:29.241+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:37.981514+0000 front 2024-02-18T11:54:37.981604+0000 (oldest deadline 2024-02-18T11:55:01.481436+0000) 2024-02-18T11:55:29.673 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:29 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:29.241+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:46.082646+0000 front 2024-02-18T11:54:46.082604+0000 (oldest deadline 2024-02-18T11:55:09.582818+0000) 2024-02-18T11:55:29.673 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:29 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:29.241+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-02-18T11:55:02.985329+0000 front 2024-02-18T11:55:02.985279+0000 (oldest deadline 2024-02-18T11:55:28.885146+0000) 2024-02-18T11:55:29.674 DEBUG:teuthology.orchestra.run.smithi188:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@osd.8.service' 2024-02-18T11:55:29.716 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-18T11:55:29.716 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-02-18T11:55:29.716 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-02-18T11:55:29.716 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@osd.9 2024-02-18T11:55:30.005 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:29 smithi012 systemd[1]: Stopping Ceph osd.9 for 0c8bb0de-ce51-11ee-95ba-87774f69a715... 2024-02-18T11:55:30.244 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:29 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:29.974+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:22.525039+0000 front 2024-02-18T11:54:22.525199+0000 (oldest deadline 2024-02-18T11:54:46.024887+0000) 2024-02-18T11:55:30.244 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:29 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:29.974+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:31.127112+0000 front 2024-02-18T11:54:31.127151+0000 (oldest deadline 2024-02-18T11:54:54.027319+0000) 2024-02-18T11:55:30.244 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:29 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:29.974+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.728559+0000 front 2024-02-18T11:54:38.728467+0000 (oldest deadline 2024-02-18T11:55:02.129139+0000) 2024-02-18T11:55:30.244 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:29 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:29.974+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:43.230285+0000 front 2024-02-18T11:54:43.230302+0000 (oldest deadline 2024-02-18T11:55:08.530478+0000) 2024-02-18T11:55:30.244 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:29 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:29.974+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.158:6812 osd.4 since back 2024-02-18T11:54:51.431536+0000 front 2024-02-18T11:54:51.431514+0000 (oldest deadline 2024-02-18T11:55:16.731790+0000) 2024-02-18T11:55:30.244 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:29 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:29.974+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-02-18T11:55:00.133598+0000 front 2024-02-18T11:55:00.133537+0000 (oldest deadline 2024-02-18T11:55:25.433398+0000) 2024-02-18T11:55:30.353 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:29 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:29.987+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:21.400001+0000 front 2024-02-18T11:54:21.400128+0000 (oldest deadline 2024-02-18T11:54:48.400621+0000) 2024-02-18T11:55:30.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:29 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:29.987+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.901802+0000 front 2024-02-18T11:54:28.901977+0000 (oldest deadline 2024-02-18T11:54:53.001759+0000) 2024-02-18T11:55:30.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:29 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:29.987+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.302728+0000 front 2024-02-18T11:54:38.302766+0000 (oldest deadline 2024-02-18T11:55:03.602861+0000) 2024-02-18T11:55:30.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:29 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:29.987+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6812 osd.4 since back 2024-02-18T11:54:52.304990+0000 front 2024-02-18T11:54:52.304951+0000 (oldest deadline 2024-02-18T11:55:17.605033+0000) 2024-02-18T11:55:30.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:29 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:29.987+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-02-18T11:55:03.406514+0000 front 2024-02-18T11:55:03.406461+0000 (oldest deadline 2024-02-18T11:55:29.207182+0000) 2024-02-18T11:55:30.422 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:30 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:30.011+0000 7f7a74bec700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-18T11:55:30.422 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:30 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:30.011+0000 7f7a74bec700 -1 osd.9 92 *** Got signal Terminated *** 2024-02-18T11:55:30.423 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:30 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:30.011+0000 7f7a74bec700 -1 osd.9 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-18T11:55:30.423 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:30 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:30.211+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.279478+0000 front 2024-02-18T11:54:23.279513+0000 (oldest deadline 2024-02-18T11:54:49.179497+0000) 2024-02-18T11:55:30.423 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:30 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:30.211+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:29.180115+0000 front 2024-02-18T11:54:29.180243+0000 (oldest deadline 2024-02-18T11:54:52.080171+0000) 2024-02-18T11:55:30.423 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:30 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:30.211+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:37.981514+0000 front 2024-02-18T11:54:37.981604+0000 (oldest deadline 2024-02-18T11:55:01.481436+0000) 2024-02-18T11:55:30.423 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:30 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:30.211+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:46.082646+0000 front 2024-02-18T11:54:46.082604+0000 (oldest deadline 2024-02-18T11:55:09.582818+0000) 2024-02-18T11:55:30.423 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:30 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:30.211+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-02-18T11:55:02.985329+0000 front 2024-02-18T11:55:02.985279+0000 (oldest deadline 2024-02-18T11:55:28.885146+0000) 2024-02-18T11:55:31.244 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:30 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:30.932+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:22.525039+0000 front 2024-02-18T11:54:22.525199+0000 (oldest deadline 2024-02-18T11:54:46.024887+0000) 2024-02-18T11:55:31.244 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:30 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:30.932+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:31.127112+0000 front 2024-02-18T11:54:31.127151+0000 (oldest deadline 2024-02-18T11:54:54.027319+0000) 2024-02-18T11:55:31.244 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:30 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:30.932+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.728559+0000 front 2024-02-18T11:54:38.728467+0000 (oldest deadline 2024-02-18T11:55:02.129139+0000) 2024-02-18T11:55:31.244 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:30 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:30.932+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:43.230285+0000 front 2024-02-18T11:54:43.230302+0000 (oldest deadline 2024-02-18T11:55:08.530478+0000) 2024-02-18T11:55:31.244 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:30 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:30.932+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.158:6812 osd.4 since back 2024-02-18T11:54:51.431536+0000 front 2024-02-18T11:54:51.431514+0000 (oldest deadline 2024-02-18T11:55:16.731790+0000) 2024-02-18T11:55:31.244 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:30 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:30.932+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-02-18T11:55:00.133598+0000 front 2024-02-18T11:55:00.133537+0000 (oldest deadline 2024-02-18T11:55:25.433398+0000) 2024-02-18T11:55:31.353 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:30 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:30.992+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:21.400001+0000 front 2024-02-18T11:54:21.400128+0000 (oldest deadline 2024-02-18T11:54:48.400621+0000) 2024-02-18T11:55:31.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:30 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:30.992+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.901802+0000 front 2024-02-18T11:54:28.901977+0000 (oldest deadline 2024-02-18T11:54:53.001759+0000) 2024-02-18T11:55:31.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:30 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:30.992+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.302728+0000 front 2024-02-18T11:54:38.302766+0000 (oldest deadline 2024-02-18T11:55:03.602861+0000) 2024-02-18T11:55:31.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:30 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:30.992+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6812 osd.4 since back 2024-02-18T11:54:52.304990+0000 front 2024-02-18T11:54:52.304951+0000 (oldest deadline 2024-02-18T11:55:17.605033+0000) 2024-02-18T11:55:31.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:30 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:30.992+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-02-18T11:55:03.406514+0000 front 2024-02-18T11:55:03.406461+0000 (oldest deadline 2024-02-18T11:55:29.207182+0000) 2024-02-18T11:55:31.422 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:31 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:31.178+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.279478+0000 front 2024-02-18T11:54:23.279513+0000 (oldest deadline 2024-02-18T11:54:49.179497+0000) 2024-02-18T11:55:31.423 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:31 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:31.178+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:29.180115+0000 front 2024-02-18T11:54:29.180243+0000 (oldest deadline 2024-02-18T11:54:52.080171+0000) 2024-02-18T11:55:31.423 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:31 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:31.178+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:37.981514+0000 front 2024-02-18T11:54:37.981604+0000 (oldest deadline 2024-02-18T11:55:01.481436+0000) 2024-02-18T11:55:31.423 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:31 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:31.178+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:46.082646+0000 front 2024-02-18T11:54:46.082604+0000 (oldest deadline 2024-02-18T11:55:09.582818+0000) 2024-02-18T11:55:31.423 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:31 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:31.178+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-02-18T11:55:02.985329+0000 front 2024-02-18T11:55:02.985279+0000 (oldest deadline 2024-02-18T11:55:28.885146+0000) 2024-02-18T11:55:32.244 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:31 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:31.924+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:22.525039+0000 front 2024-02-18T11:54:22.525199+0000 (oldest deadline 2024-02-18T11:54:46.024887+0000) 2024-02-18T11:55:32.244 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:31 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:31.924+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:31.127112+0000 front 2024-02-18T11:54:31.127151+0000 (oldest deadline 2024-02-18T11:54:54.027319+0000) 2024-02-18T11:55:32.244 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:31 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:31.924+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.728559+0000 front 2024-02-18T11:54:38.728467+0000 (oldest deadline 2024-02-18T11:55:02.129139+0000) 2024-02-18T11:55:32.244 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:31 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:31.924+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:43.230285+0000 front 2024-02-18T11:54:43.230302+0000 (oldest deadline 2024-02-18T11:55:08.530478+0000) 2024-02-18T11:55:32.244 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:31 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:31.924+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.158:6812 osd.4 since back 2024-02-18T11:54:51.431536+0000 front 2024-02-18T11:54:51.431514+0000 (oldest deadline 2024-02-18T11:55:16.731790+0000) 2024-02-18T11:55:32.244 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:31 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:31.924+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-02-18T11:55:00.133598+0000 front 2024-02-18T11:55:00.133537+0000 (oldest deadline 2024-02-18T11:55:25.433398+0000) 2024-02-18T11:55:32.244 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:31 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:31.924+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-02-18T11:55:10.134482+0000 front 2024-02-18T11:55:10.134499+0000 (oldest deadline 2024-02-18T11:55:31.834352+0000) 2024-02-18T11:55:32.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:31 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:31.960+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:21.400001+0000 front 2024-02-18T11:54:21.400128+0000 (oldest deadline 2024-02-18T11:54:48.400621+0000) 2024-02-18T11:55:32.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:31 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:31.960+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.901802+0000 front 2024-02-18T11:54:28.901977+0000 (oldest deadline 2024-02-18T11:54:53.001759+0000) 2024-02-18T11:55:32.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:31 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:31.960+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.302728+0000 front 2024-02-18T11:54:38.302766+0000 (oldest deadline 2024-02-18T11:55:03.602861+0000) 2024-02-18T11:55:32.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:31 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:31.960+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6812 osd.4 since back 2024-02-18T11:54:52.304990+0000 front 2024-02-18T11:54:52.304951+0000 (oldest deadline 2024-02-18T11:55:17.605033+0000) 2024-02-18T11:55:32.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:31 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:31.960+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-02-18T11:55:03.406514+0000 front 2024-02-18T11:55:03.406461+0000 (oldest deadline 2024-02-18T11:55:29.207182+0000) 2024-02-18T11:55:32.672 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:32 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:32.209+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.279478+0000 front 2024-02-18T11:54:23.279513+0000 (oldest deadline 2024-02-18T11:54:49.179497+0000) 2024-02-18T11:55:32.673 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:32 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:32.209+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:29.180115+0000 front 2024-02-18T11:54:29.180243+0000 (oldest deadline 2024-02-18T11:54:52.080171+0000) 2024-02-18T11:55:32.673 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:32 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:32.209+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:37.981514+0000 front 2024-02-18T11:54:37.981604+0000 (oldest deadline 2024-02-18T11:55:01.481436+0000) 2024-02-18T11:55:32.673 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:32 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:32.209+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:46.082646+0000 front 2024-02-18T11:54:46.082604+0000 (oldest deadline 2024-02-18T11:55:09.582818+0000) 2024-02-18T11:55:32.673 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:32 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:32.209+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-02-18T11:55:02.985329+0000 front 2024-02-18T11:55:02.985279+0000 (oldest deadline 2024-02-18T11:55:28.885146+0000) 2024-02-18T11:55:33.244 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:32 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:32.972+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:22.525039+0000 front 2024-02-18T11:54:22.525199+0000 (oldest deadline 2024-02-18T11:54:46.024887+0000) 2024-02-18T11:55:33.244 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:32 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:32.972+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:31.127112+0000 front 2024-02-18T11:54:31.127151+0000 (oldest deadline 2024-02-18T11:54:54.027319+0000) 2024-02-18T11:55:33.244 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:32 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:32.972+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.728559+0000 front 2024-02-18T11:54:38.728467+0000 (oldest deadline 2024-02-18T11:55:02.129139+0000) 2024-02-18T11:55:33.244 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:32 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:32.972+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:43.230285+0000 front 2024-02-18T11:54:43.230302+0000 (oldest deadline 2024-02-18T11:55:08.530478+0000) 2024-02-18T11:55:33.244 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:32 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:32.972+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.158:6812 osd.4 since back 2024-02-18T11:54:51.431536+0000 front 2024-02-18T11:54:51.431514+0000 (oldest deadline 2024-02-18T11:55:16.731790+0000) 2024-02-18T11:55:33.244 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:32 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:32.972+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-02-18T11:55:00.133598+0000 front 2024-02-18T11:55:00.133537+0000 (oldest deadline 2024-02-18T11:55:25.433398+0000) 2024-02-18T11:55:33.244 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:32 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:32.972+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-02-18T11:55:10.134482+0000 front 2024-02-18T11:55:10.134499+0000 (oldest deadline 2024-02-18T11:55:31.834352+0000) 2024-02-18T11:55:33.353 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:32 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:32.976+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:21.400001+0000 front 2024-02-18T11:54:21.400128+0000 (oldest deadline 2024-02-18T11:54:48.400621+0000) 2024-02-18T11:55:33.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:32 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:32.976+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.901802+0000 front 2024-02-18T11:54:28.901977+0000 (oldest deadline 2024-02-18T11:54:53.001759+0000) 2024-02-18T11:55:33.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:32 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:32.976+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.302728+0000 front 2024-02-18T11:54:38.302766+0000 (oldest deadline 2024-02-18T11:55:03.602861+0000) 2024-02-18T11:55:33.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:32 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:32.976+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6812 osd.4 since back 2024-02-18T11:54:52.304990+0000 front 2024-02-18T11:54:52.304951+0000 (oldest deadline 2024-02-18T11:55:17.605033+0000) 2024-02-18T11:55:33.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:32 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:32.976+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-02-18T11:55:03.406514+0000 front 2024-02-18T11:55:03.406461+0000 (oldest deadline 2024-02-18T11:55:29.207182+0000) 2024-02-18T11:55:33.673 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:33 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:33.251+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.279478+0000 front 2024-02-18T11:54:23.279513+0000 (oldest deadline 2024-02-18T11:54:49.179497+0000) 2024-02-18T11:55:33.673 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:33 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:33.251+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:29.180115+0000 front 2024-02-18T11:54:29.180243+0000 (oldest deadline 2024-02-18T11:54:52.080171+0000) 2024-02-18T11:55:33.673 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:33 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:33.251+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:37.981514+0000 front 2024-02-18T11:54:37.981604+0000 (oldest deadline 2024-02-18T11:55:01.481436+0000) 2024-02-18T11:55:33.673 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:33 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:33.251+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:46.082646+0000 front 2024-02-18T11:54:46.082604+0000 (oldest deadline 2024-02-18T11:55:09.582818+0000) 2024-02-18T11:55:33.673 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:33 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:33.251+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-02-18T11:55:02.985329+0000 front 2024-02-18T11:55:02.985279+0000 (oldest deadline 2024-02-18T11:55:28.885146+0000) 2024-02-18T11:55:34.244 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:33 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:33.988+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:22.525039+0000 front 2024-02-18T11:54:22.525199+0000 (oldest deadline 2024-02-18T11:54:46.024887+0000) 2024-02-18T11:55:34.244 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:33 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:33.988+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:31.127112+0000 front 2024-02-18T11:54:31.127151+0000 (oldest deadline 2024-02-18T11:54:54.027319+0000) 2024-02-18T11:55:34.244 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:33 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:33.988+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.728559+0000 front 2024-02-18T11:54:38.728467+0000 (oldest deadline 2024-02-18T11:55:02.129139+0000) 2024-02-18T11:55:34.244 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:33 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:33.988+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:43.230285+0000 front 2024-02-18T11:54:43.230302+0000 (oldest deadline 2024-02-18T11:55:08.530478+0000) 2024-02-18T11:55:34.245 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:33 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:33.988+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.158:6812 osd.4 since back 2024-02-18T11:54:51.431536+0000 front 2024-02-18T11:54:51.431514+0000 (oldest deadline 2024-02-18T11:55:16.731790+0000) 2024-02-18T11:55:34.245 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:33 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:33.988+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-02-18T11:55:00.133598+0000 front 2024-02-18T11:55:00.133537+0000 (oldest deadline 2024-02-18T11:55:25.433398+0000) 2024-02-18T11:55:34.245 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:33 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:33.988+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-02-18T11:55:10.134482+0000 front 2024-02-18T11:55:10.134499+0000 (oldest deadline 2024-02-18T11:55:31.834352+0000) 2024-02-18T11:55:34.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:34 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:34.000+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:21.400001+0000 front 2024-02-18T11:54:21.400128+0000 (oldest deadline 2024-02-18T11:54:48.400621+0000) 2024-02-18T11:55:34.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:34 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:34.000+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.901802+0000 front 2024-02-18T11:54:28.901977+0000 (oldest deadline 2024-02-18T11:54:53.001759+0000) 2024-02-18T11:55:34.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:34 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:34.000+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.302728+0000 front 2024-02-18T11:54:38.302766+0000 (oldest deadline 2024-02-18T11:55:03.602861+0000) 2024-02-18T11:55:34.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:34 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:34.000+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6812 osd.4 since back 2024-02-18T11:54:52.304990+0000 front 2024-02-18T11:54:52.304951+0000 (oldest deadline 2024-02-18T11:55:17.605033+0000) 2024-02-18T11:55:34.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:34 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:34.000+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-02-18T11:55:03.406514+0000 front 2024-02-18T11:55:03.406461+0000 (oldest deadline 2024-02-18T11:55:29.207182+0000) 2024-02-18T11:55:34.673 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:34 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:34.260+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:23.279478+0000 front 2024-02-18T11:54:23.279513+0000 (oldest deadline 2024-02-18T11:54:49.179497+0000) 2024-02-18T11:55:34.673 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:34 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:34.260+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:29.180115+0000 front 2024-02-18T11:54:29.180243+0000 (oldest deadline 2024-02-18T11:54:52.080171+0000) 2024-02-18T11:55:34.673 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:34 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:34.260+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:37.981514+0000 front 2024-02-18T11:54:37.981604+0000 (oldest deadline 2024-02-18T11:55:01.481436+0000) 2024-02-18T11:55:34.673 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:34 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:34.260+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:46.082646+0000 front 2024-02-18T11:54:46.082604+0000 (oldest deadline 2024-02-18T11:55:09.582818+0000) 2024-02-18T11:55:34.673 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:34 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:34.260+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-02-18T11:55:02.985329+0000 front 2024-02-18T11:55:02.985279+0000 (oldest deadline 2024-02-18T11:55:28.885146+0000) 2024-02-18T11:55:34.673 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:34 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9[65537]: 2024-02-18T11:55:34.260+0000 7f7a719e1700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-02-18T11:55:08.885589+0000 front 2024-02-18T11:55:08.885527+0000 (oldest deadline 2024-02-18T11:55:34.185512+0000) 2024-02-18T11:55:35.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:34 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:34.972+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:21.400001+0000 front 2024-02-18T11:54:21.400128+0000 (oldest deadline 2024-02-18T11:54:48.400621+0000) 2024-02-18T11:55:35.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:34 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:34.972+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.901802+0000 front 2024-02-18T11:54:28.901977+0000 (oldest deadline 2024-02-18T11:54:53.001759+0000) 2024-02-18T11:55:35.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:34 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:34.972+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.302728+0000 front 2024-02-18T11:54:38.302766+0000 (oldest deadline 2024-02-18T11:55:03.602861+0000) 2024-02-18T11:55:35.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:34 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:34.972+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6812 osd.4 since back 2024-02-18T11:54:52.304990+0000 front 2024-02-18T11:54:52.304951+0000 (oldest deadline 2024-02-18T11:55:17.605033+0000) 2024-02-18T11:55:35.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:34 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:34.972+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-02-18T11:55:03.406514+0000 front 2024-02-18T11:55:03.406461+0000 (oldest deadline 2024-02-18T11:55:29.207182+0000) 2024-02-18T11:55:35.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:35 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:35.014+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:22.525039+0000 front 2024-02-18T11:54:22.525199+0000 (oldest deadline 2024-02-18T11:54:46.024887+0000) 2024-02-18T11:55:35.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:35 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:35.014+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:31.127112+0000 front 2024-02-18T11:54:31.127151+0000 (oldest deadline 2024-02-18T11:54:54.027319+0000) 2024-02-18T11:55:35.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:35 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:35.014+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.728559+0000 front 2024-02-18T11:54:38.728467+0000 (oldest deadline 2024-02-18T11:55:02.129139+0000) 2024-02-18T11:55:35.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:35 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:35.014+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:43.230285+0000 front 2024-02-18T11:54:43.230302+0000 (oldest deadline 2024-02-18T11:55:08.530478+0000) 2024-02-18T11:55:35.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:35 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:35.014+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.158:6812 osd.4 since back 2024-02-18T11:54:51.431536+0000 front 2024-02-18T11:54:51.431514+0000 (oldest deadline 2024-02-18T11:55:16.731790+0000) 2024-02-18T11:55:35.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:35 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:35.014+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-02-18T11:55:00.133598+0000 front 2024-02-18T11:55:00.133537+0000 (oldest deadline 2024-02-18T11:55:25.433398+0000) 2024-02-18T11:55:35.495 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:35 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:35.014+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-02-18T11:55:10.134482+0000 front 2024-02-18T11:55:10.134499+0000 (oldest deadline 2024-02-18T11:55:31.834352+0000) 2024-02-18T11:55:35.673 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:35 smithi012 podman[87985]: 2024-02-18 11:55:35.385098735 +0000 UTC m=+5.465456819 container died 85fecb26ec7f4330879bcf7667e08eca10741fb030e898ad470eea7e6cd417b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-02-18T11:55:36.172 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:35 smithi012 podman[87985]: 2024-02-18 11:55:35.706271744 +0000 UTC m=+5.786629828 container cleanup 85fecb26ec7f4330879bcf7667e08eca10741fb030e898ad470eea7e6cd417b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-18T11:55:36.173 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:35 smithi012 bash[87985]: ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9 2024-02-18T11:55:36.173 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:35 smithi012 podman[88002]: 2024-02-18 11:55:35.937510075 +0000 UTC m=+0.546977881 container remove 85fecb26ec7f4330879bcf7667e08eca10741fb030e898ad470eea7e6cd417b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240214) 2024-02-18T11:55:36.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:36 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:35.998+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:21.400001+0000 front 2024-02-18T11:54:21.400128+0000 (oldest deadline 2024-02-18T11:54:48.400621+0000) 2024-02-18T11:55:36.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:36 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:35.998+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.901802+0000 front 2024-02-18T11:54:28.901977+0000 (oldest deadline 2024-02-18T11:54:53.001759+0000) 2024-02-18T11:55:36.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:36 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:35.998+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.302728+0000 front 2024-02-18T11:54:38.302766+0000 (oldest deadline 2024-02-18T11:55:03.602861+0000) 2024-02-18T11:55:36.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:36 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:35.998+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6812 osd.4 since back 2024-02-18T11:54:52.304990+0000 front 2024-02-18T11:54:52.304951+0000 (oldest deadline 2024-02-18T11:55:17.605033+0000) 2024-02-18T11:55:36.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:36 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:35.998+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-02-18T11:55:03.406514+0000 front 2024-02-18T11:55:03.406461+0000 (oldest deadline 2024-02-18T11:55:29.207182+0000) 2024-02-18T11:55:36.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:36 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:36.058+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:22.525039+0000 front 2024-02-18T11:54:22.525199+0000 (oldest deadline 2024-02-18T11:54:46.024887+0000) 2024-02-18T11:55:36.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:36 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:36.058+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:31.127112+0000 front 2024-02-18T11:54:31.127151+0000 (oldest deadline 2024-02-18T11:54:54.027319+0000) 2024-02-18T11:55:36.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:36 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:36.058+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.728559+0000 front 2024-02-18T11:54:38.728467+0000 (oldest deadline 2024-02-18T11:55:02.129139+0000) 2024-02-18T11:55:36.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:36 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:36.058+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:43.230285+0000 front 2024-02-18T11:54:43.230302+0000 (oldest deadline 2024-02-18T11:55:08.530478+0000) 2024-02-18T11:55:36.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:36 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:36.058+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.158:6812 osd.4 since back 2024-02-18T11:54:51.431536+0000 front 2024-02-18T11:54:51.431514+0000 (oldest deadline 2024-02-18T11:55:16.731790+0000) 2024-02-18T11:55:36.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:36 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:36.058+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-02-18T11:55:00.133598+0000 front 2024-02-18T11:55:00.133537+0000 (oldest deadline 2024-02-18T11:55:25.433398+0000) 2024-02-18T11:55:36.495 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:36 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:36.058+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-02-18T11:55:10.134482+0000 front 2024-02-18T11:55:10.134499+0000 (oldest deadline 2024-02-18T11:55:31.834352+0000) 2024-02-18T11:55:36.673 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:36 smithi012 podman[88091]: 2024-02-18 11:55:36.282611372 +0000 UTC m=+0.017461036 image pull 53c33ee3bf62dc411b650f86eedbbee7698efb1001b4c063271dcd8d65b41165 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1 2024-02-18T11:55:36.673 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:36 smithi012 podman[88091]: 2024-02-18 11:55:36.406425136 +0000 UTC m=+0.141274837 container create 6416e8d62779a2e06b41e7ce41502131cfd3df998f5876f168f553787241238f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-02-18T11:55:36.673 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:36 smithi012 podman[88091]: 2024-02-18 11:55:36.607650131 +0000 UTC m=+0.342499802 container init 6416e8d62779a2e06b41e7ce41502131cfd3df998f5876f168f553787241238f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-02-18T11:55:36.673 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:36 smithi012 podman[88091]: 2024-02-18 11:55:36.610925256 +0000 UTC m=+0.345774924 container start 6416e8d62779a2e06b41e7ce41502131cfd3df998f5876f168f553787241238f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9-deactivate, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-02-18T11:55:37.173 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:36 smithi012 podman[88091]: 2024-02-18 11:55:36.682346634 +0000 UTC m=+0.417196301 container attach 6416e8d62779a2e06b41e7ce41502131cfd3df998f5876f168f553787241238f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-18T11:55:37.173 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:36 smithi012 podman[88091]: 2024-02-18 11:55:36.908874176 +0000 UTC m=+0.643723836 container died 6416e8d62779a2e06b41e7ce41502131cfd3df998f5876f168f553787241238f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9-deactivate, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-02-18T11:55:37.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:37 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:37.002+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:21.400001+0000 front 2024-02-18T11:54:21.400128+0000 (oldest deadline 2024-02-18T11:54:48.400621+0000) 2024-02-18T11:55:37.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:37 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:37.002+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.901802+0000 front 2024-02-18T11:54:28.901977+0000 (oldest deadline 2024-02-18T11:54:53.001759+0000) 2024-02-18T11:55:37.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:37 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:37.002+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.302728+0000 front 2024-02-18T11:54:38.302766+0000 (oldest deadline 2024-02-18T11:55:03.602861+0000) 2024-02-18T11:55:37.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:37 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:37.002+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6812 osd.4 since back 2024-02-18T11:54:52.304990+0000 front 2024-02-18T11:54:52.304951+0000 (oldest deadline 2024-02-18T11:55:17.605033+0000) 2024-02-18T11:55:37.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:37 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:37.002+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-02-18T11:55:03.406514+0000 front 2024-02-18T11:55:03.406461+0000 (oldest deadline 2024-02-18T11:55:29.207182+0000) 2024-02-18T11:55:37.434 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@osd.9.service' 2024-02-18T11:55:37.460 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:37 smithi012 podman[88117]: 2024-02-18 11:55:37.418841287 +0000 UTC m=+0.505215440 container remove 6416e8d62779a2e06b41e7ce41502131cfd3df998f5876f168f553787241238f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-9-deactivate, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214) 2024-02-18T11:55:37.460 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:37 smithi012 systemd[1]: ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@osd.9.service: Deactivated successfully. 2024-02-18T11:55:37.460 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:37 smithi012 systemd[1]: Stopped Ceph osd.9 for 0c8bb0de-ce51-11ee-95ba-87774f69a715. 2024-02-18T11:55:37.460 INFO:journalctl@ceph.osd.9.smithi012.stdout:Feb 18 11:55:37 smithi012 systemd[1]: ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@osd.9.service: Consumed 6min 11.094s CPU time. 2024-02-18T11:55:37.476 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-18T11:55:37.476 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-02-18T11:55:37.476 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-02-18T11:55:37.477 DEBUG:teuthology.orchestra.run.smithi158:> sudo systemctl stop ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@osd.10 2024-02-18T11:55:37.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:37 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:37.048+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:22.525039+0000 front 2024-02-18T11:54:22.525199+0000 (oldest deadline 2024-02-18T11:54:46.024887+0000) 2024-02-18T11:55:37.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:37 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:37.048+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:31.127112+0000 front 2024-02-18T11:54:31.127151+0000 (oldest deadline 2024-02-18T11:54:54.027319+0000) 2024-02-18T11:55:37.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:37 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:37.048+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.728559+0000 front 2024-02-18T11:54:38.728467+0000 (oldest deadline 2024-02-18T11:55:02.129139+0000) 2024-02-18T11:55:37.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:37 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:37.048+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:43.230285+0000 front 2024-02-18T11:54:43.230302+0000 (oldest deadline 2024-02-18T11:55:08.530478+0000) 2024-02-18T11:55:37.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:37 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:37.048+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.158:6812 osd.4 since back 2024-02-18T11:54:51.431536+0000 front 2024-02-18T11:54:51.431514+0000 (oldest deadline 2024-02-18T11:55:16.731790+0000) 2024-02-18T11:55:37.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:37 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:37.048+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-02-18T11:55:00.133598+0000 front 2024-02-18T11:55:00.133537+0000 (oldest deadline 2024-02-18T11:55:25.433398+0000) 2024-02-18T11:55:37.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:37 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:37.048+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-02-18T11:55:10.134482+0000 front 2024-02-18T11:55:10.134499+0000 (oldest deadline 2024-02-18T11:55:31.834352+0000) 2024-02-18T11:55:37.813 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:37 smithi158 systemd[1]: Stopping Ceph osd.10 for 0c8bb0de-ce51-11ee-95ba-87774f69a715... 2024-02-18T11:55:38.087 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:37 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:37.818+0000 7f56e3f35700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-18T11:55:38.087 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:37 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:37.818+0000 7f56e3f35700 -1 osd.10 92 *** Got signal Terminated *** 2024-02-18T11:55:38.087 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:37 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:37.818+0000 7f56e3f35700 -1 osd.10 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-18T11:55:38.088 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:38 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:38.093+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:22.525039+0000 front 2024-02-18T11:54:22.525199+0000 (oldest deadline 2024-02-18T11:54:46.024887+0000) 2024-02-18T11:55:38.088 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:38 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:38.093+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:31.127112+0000 front 2024-02-18T11:54:31.127151+0000 (oldest deadline 2024-02-18T11:54:54.027319+0000) 2024-02-18T11:55:38.088 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:38 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:38.093+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.728559+0000 front 2024-02-18T11:54:38.728467+0000 (oldest deadline 2024-02-18T11:55:02.129139+0000) 2024-02-18T11:55:38.088 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:38 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:38.093+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:43.230285+0000 front 2024-02-18T11:54:43.230302+0000 (oldest deadline 2024-02-18T11:55:08.530478+0000) 2024-02-18T11:55:38.088 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:38 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:38.093+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.158:6812 osd.4 since back 2024-02-18T11:54:51.431536+0000 front 2024-02-18T11:54:51.431514+0000 (oldest deadline 2024-02-18T11:55:16.731790+0000) 2024-02-18T11:55:38.088 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:38 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:38.093+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-02-18T11:55:00.133598+0000 front 2024-02-18T11:55:00.133537+0000 (oldest deadline 2024-02-18T11:55:25.433398+0000) 2024-02-18T11:55:38.088 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:38 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:38.093+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-02-18T11:55:10.134482+0000 front 2024-02-18T11:55:10.134499+0000 (oldest deadline 2024-02-18T11:55:31.834352+0000) 2024-02-18T11:55:38.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:37 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:37.980+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:21.400001+0000 front 2024-02-18T11:54:21.400128+0000 (oldest deadline 2024-02-18T11:54:48.400621+0000) 2024-02-18T11:55:38.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:37 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:37.980+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.901802+0000 front 2024-02-18T11:54:28.901977+0000 (oldest deadline 2024-02-18T11:54:53.001759+0000) 2024-02-18T11:55:38.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:37 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:37.980+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.302728+0000 front 2024-02-18T11:54:38.302766+0000 (oldest deadline 2024-02-18T11:55:03.602861+0000) 2024-02-18T11:55:38.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:37 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:37.980+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6812 osd.4 since back 2024-02-18T11:54:52.304990+0000 front 2024-02-18T11:54:52.304951+0000 (oldest deadline 2024-02-18T11:55:17.605033+0000) 2024-02-18T11:55:38.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:37 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:37.980+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-02-18T11:55:03.406514+0000 front 2024-02-18T11:55:03.406461+0000 (oldest deadline 2024-02-18T11:55:29.207182+0000) 2024-02-18T11:55:38.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:37 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:37.980+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-02-18T11:55:09.208076+0000 front 2024-02-18T11:55:09.207926+0000 (oldest deadline 2024-02-18T11:55:37.408172+0000) 2024-02-18T11:55:39.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:39 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:39.024+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:21.400001+0000 front 2024-02-18T11:54:21.400128+0000 (oldest deadline 2024-02-18T11:54:48.400621+0000) 2024-02-18T11:55:39.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:39 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:39.024+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.901802+0000 front 2024-02-18T11:54:28.901977+0000 (oldest deadline 2024-02-18T11:54:53.001759+0000) 2024-02-18T11:55:39.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:39 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:39.024+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.302728+0000 front 2024-02-18T11:54:38.302766+0000 (oldest deadline 2024-02-18T11:55:03.602861+0000) 2024-02-18T11:55:39.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:39 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:39.024+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6812 osd.4 since back 2024-02-18T11:54:52.304990+0000 front 2024-02-18T11:54:52.304951+0000 (oldest deadline 2024-02-18T11:55:17.605033+0000) 2024-02-18T11:55:39.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:39 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:39.024+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-02-18T11:55:03.406514+0000 front 2024-02-18T11:55:03.406461+0000 (oldest deadline 2024-02-18T11:55:29.207182+0000) 2024-02-18T11:55:39.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:39 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:39.024+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-02-18T11:55:09.208076+0000 front 2024-02-18T11:55:09.207926+0000 (oldest deadline 2024-02-18T11:55:37.408172+0000) 2024-02-18T11:55:39.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:39 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:39.096+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:22.525039+0000 front 2024-02-18T11:54:22.525199+0000 (oldest deadline 2024-02-18T11:54:46.024887+0000) 2024-02-18T11:55:39.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:39 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:39.096+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:31.127112+0000 front 2024-02-18T11:54:31.127151+0000 (oldest deadline 2024-02-18T11:54:54.027319+0000) 2024-02-18T11:55:39.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:39 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:39.096+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.728559+0000 front 2024-02-18T11:54:38.728467+0000 (oldest deadline 2024-02-18T11:55:02.129139+0000) 2024-02-18T11:55:39.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:39 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:39.096+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:43.230285+0000 front 2024-02-18T11:54:43.230302+0000 (oldest deadline 2024-02-18T11:55:08.530478+0000) 2024-02-18T11:55:39.495 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:39 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:39.096+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.158:6812 osd.4 since back 2024-02-18T11:54:51.431536+0000 front 2024-02-18T11:54:51.431514+0000 (oldest deadline 2024-02-18T11:55:16.731790+0000) 2024-02-18T11:55:39.495 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:39 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:39.096+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-02-18T11:55:00.133598+0000 front 2024-02-18T11:55:00.133537+0000 (oldest deadline 2024-02-18T11:55:25.433398+0000) 2024-02-18T11:55:39.495 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:39 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:39.096+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-02-18T11:55:10.134482+0000 front 2024-02-18T11:55:10.134499+0000 (oldest deadline 2024-02-18T11:55:31.834352+0000) 2024-02-18T11:55:40.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:40 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:40.069+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:21.400001+0000 front 2024-02-18T11:54:21.400128+0000 (oldest deadline 2024-02-18T11:54:48.400621+0000) 2024-02-18T11:55:40.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:40 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:40.069+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.901802+0000 front 2024-02-18T11:54:28.901977+0000 (oldest deadline 2024-02-18T11:54:53.001759+0000) 2024-02-18T11:55:40.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:40 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:40.069+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.302728+0000 front 2024-02-18T11:54:38.302766+0000 (oldest deadline 2024-02-18T11:55:03.602861+0000) 2024-02-18T11:55:40.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:40 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:40.069+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6812 osd.4 since back 2024-02-18T11:54:52.304990+0000 front 2024-02-18T11:54:52.304951+0000 (oldest deadline 2024-02-18T11:55:17.605033+0000) 2024-02-18T11:55:40.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:40 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:40.069+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-02-18T11:55:03.406514+0000 front 2024-02-18T11:55:03.406461+0000 (oldest deadline 2024-02-18T11:55:29.207182+0000) 2024-02-18T11:55:40.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:40 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:40.069+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-02-18T11:55:09.208076+0000 front 2024-02-18T11:55:09.207926+0000 (oldest deadline 2024-02-18T11:55:37.408172+0000) 2024-02-18T11:55:40.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:40 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:40.083+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:22.525039+0000 front 2024-02-18T11:54:22.525199+0000 (oldest deadline 2024-02-18T11:54:46.024887+0000) 2024-02-18T11:55:40.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:40 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:40.083+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:31.127112+0000 front 2024-02-18T11:54:31.127151+0000 (oldest deadline 2024-02-18T11:54:54.027319+0000) 2024-02-18T11:55:40.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:40 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:40.083+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.728559+0000 front 2024-02-18T11:54:38.728467+0000 (oldest deadline 2024-02-18T11:55:02.129139+0000) 2024-02-18T11:55:40.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:40 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:40.083+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:43.230285+0000 front 2024-02-18T11:54:43.230302+0000 (oldest deadline 2024-02-18T11:55:08.530478+0000) 2024-02-18T11:55:40.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:40 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:40.083+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.158:6812 osd.4 since back 2024-02-18T11:54:51.431536+0000 front 2024-02-18T11:54:51.431514+0000 (oldest deadline 2024-02-18T11:55:16.731790+0000) 2024-02-18T11:55:40.495 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:40 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:40.083+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-02-18T11:55:00.133598+0000 front 2024-02-18T11:55:00.133537+0000 (oldest deadline 2024-02-18T11:55:25.433398+0000) 2024-02-18T11:55:40.495 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:40 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:40.083+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-02-18T11:55:10.134482+0000 front 2024-02-18T11:55:10.134499+0000 (oldest deadline 2024-02-18T11:55:31.834352+0000) 2024-02-18T11:55:41.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:41 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:41.059+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:21.400001+0000 front 2024-02-18T11:54:21.400128+0000 (oldest deadline 2024-02-18T11:54:48.400621+0000) 2024-02-18T11:55:41.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:41 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:41.059+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.901802+0000 front 2024-02-18T11:54:28.901977+0000 (oldest deadline 2024-02-18T11:54:53.001759+0000) 2024-02-18T11:55:41.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:41 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:41.059+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.302728+0000 front 2024-02-18T11:54:38.302766+0000 (oldest deadline 2024-02-18T11:55:03.602861+0000) 2024-02-18T11:55:41.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:41 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:41.059+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6812 osd.4 since back 2024-02-18T11:54:52.304990+0000 front 2024-02-18T11:54:52.304951+0000 (oldest deadline 2024-02-18T11:55:17.605033+0000) 2024-02-18T11:55:41.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:41 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:41.059+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-02-18T11:55:03.406514+0000 front 2024-02-18T11:55:03.406461+0000 (oldest deadline 2024-02-18T11:55:29.207182+0000) 2024-02-18T11:55:41.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:41 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:41.059+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-02-18T11:55:09.208076+0000 front 2024-02-18T11:55:09.207926+0000 (oldest deadline 2024-02-18T11:55:37.408172+0000) 2024-02-18T11:55:41.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:41 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:41.042+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:22.525039+0000 front 2024-02-18T11:54:22.525199+0000 (oldest deadline 2024-02-18T11:54:46.024887+0000) 2024-02-18T11:55:41.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:41 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:41.042+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:31.127112+0000 front 2024-02-18T11:54:31.127151+0000 (oldest deadline 2024-02-18T11:54:54.027319+0000) 2024-02-18T11:55:41.495 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:41 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:41.042+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.728559+0000 front 2024-02-18T11:54:38.728467+0000 (oldest deadline 2024-02-18T11:55:02.129139+0000) 2024-02-18T11:55:41.495 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:41 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:41.042+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:43.230285+0000 front 2024-02-18T11:54:43.230302+0000 (oldest deadline 2024-02-18T11:55:08.530478+0000) 2024-02-18T11:55:41.495 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:41 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:41.042+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.158:6812 osd.4 since back 2024-02-18T11:54:51.431536+0000 front 2024-02-18T11:54:51.431514+0000 (oldest deadline 2024-02-18T11:55:16.731790+0000) 2024-02-18T11:55:41.495 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:41 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:41.042+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-02-18T11:55:00.133598+0000 front 2024-02-18T11:55:00.133537+0000 (oldest deadline 2024-02-18T11:55:25.433398+0000) 2024-02-18T11:55:41.495 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:41 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:41.042+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-02-18T11:55:10.134482+0000 front 2024-02-18T11:55:10.134499+0000 (oldest deadline 2024-02-18T11:55:31.834352+0000) 2024-02-18T11:55:42.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:42 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:42.099+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:21.400001+0000 front 2024-02-18T11:54:21.400128+0000 (oldest deadline 2024-02-18T11:54:48.400621+0000) 2024-02-18T11:55:42.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:42 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:42.099+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.901802+0000 front 2024-02-18T11:54:28.901977+0000 (oldest deadline 2024-02-18T11:54:53.001759+0000) 2024-02-18T11:55:42.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:42 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:42.099+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.302728+0000 front 2024-02-18T11:54:38.302766+0000 (oldest deadline 2024-02-18T11:55:03.602861+0000) 2024-02-18T11:55:42.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:42 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:42.099+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6812 osd.4 since back 2024-02-18T11:54:52.304990+0000 front 2024-02-18T11:54:52.304951+0000 (oldest deadline 2024-02-18T11:55:17.605033+0000) 2024-02-18T11:55:42.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:42 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:42.099+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-02-18T11:55:03.406514+0000 front 2024-02-18T11:55:03.406461+0000 (oldest deadline 2024-02-18T11:55:29.207182+0000) 2024-02-18T11:55:42.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:42 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:42.099+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-02-18T11:55:09.208076+0000 front 2024-02-18T11:55:09.207926+0000 (oldest deadline 2024-02-18T11:55:37.408172+0000) 2024-02-18T11:55:42.494 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:42 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:42.058+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:22.525039+0000 front 2024-02-18T11:54:22.525199+0000 (oldest deadline 2024-02-18T11:54:46.024887+0000) 2024-02-18T11:55:42.495 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:42 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:42.058+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:31.127112+0000 front 2024-02-18T11:54:31.127151+0000 (oldest deadline 2024-02-18T11:54:54.027319+0000) 2024-02-18T11:55:42.495 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:42 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:42.058+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.728559+0000 front 2024-02-18T11:54:38.728467+0000 (oldest deadline 2024-02-18T11:55:02.129139+0000) 2024-02-18T11:55:42.495 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:42 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:42.058+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-02-18T11:54:43.230285+0000 front 2024-02-18T11:54:43.230302+0000 (oldest deadline 2024-02-18T11:55:08.530478+0000) 2024-02-18T11:55:42.495 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:42 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:42.058+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.158:6812 osd.4 since back 2024-02-18T11:54:51.431536+0000 front 2024-02-18T11:54:51.431514+0000 (oldest deadline 2024-02-18T11:55:16.731790+0000) 2024-02-18T11:55:42.495 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:42 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:42.058+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-02-18T11:55:00.133598+0000 front 2024-02-18T11:55:00.133537+0000 (oldest deadline 2024-02-18T11:55:25.433398+0000) 2024-02-18T11:55:42.496 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:42 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10[61242]: 2024-02-18T11:55:42.058+0000 7f56dcd22700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-02-18T11:55:10.134482+0000 front 2024-02-18T11:55:10.134499+0000 (oldest deadline 2024-02-18T11:55:31.834352+0000) 2024-02-18T11:55:43.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:43 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:43.109+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:21.400001+0000 front 2024-02-18T11:54:21.400128+0000 (oldest deadline 2024-02-18T11:54:48.400621+0000) 2024-02-18T11:55:43.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:43 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:43.109+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.901802+0000 front 2024-02-18T11:54:28.901977+0000 (oldest deadline 2024-02-18T11:54:53.001759+0000) 2024-02-18T11:55:43.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:43 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:43.109+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.302728+0000 front 2024-02-18T11:54:38.302766+0000 (oldest deadline 2024-02-18T11:55:03.602861+0000) 2024-02-18T11:55:43.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:43 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:43.109+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6812 osd.4 since back 2024-02-18T11:54:52.304990+0000 front 2024-02-18T11:54:52.304951+0000 (oldest deadline 2024-02-18T11:55:17.605033+0000) 2024-02-18T11:55:43.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:43 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:43.109+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-02-18T11:55:03.406514+0000 front 2024-02-18T11:55:03.406461+0000 (oldest deadline 2024-02-18T11:55:29.207182+0000) 2024-02-18T11:55:43.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:43 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:43.109+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-02-18T11:55:09.208076+0000 front 2024-02-18T11:55:09.207926+0000 (oldest deadline 2024-02-18T11:55:37.408172+0000) 2024-02-18T11:55:43.672 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:43 smithi158 podman[72889]: 2024-02-18 11:55:43.405664771 +0000 UTC m=+5.730541010 container died ad4c95a3f804173a0847b9e75cb9c47209fa551b1531c0000747bc8d63f7c8c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, org.label-schema.build-date=20240214, ceph=True, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-02-18T11:55:43.980 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:43 smithi158 podman[72889]: 2024-02-18 11:55:43.776991582 +0000 UTC m=+6.101867812 container cleanup ad4c95a3f804173a0847b9e75cb9c47209fa551b1531c0000747bc8d63f7c8c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214) 2024-02-18T11:55:43.980 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:43 smithi158 bash[72889]: ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10 2024-02-18T11:55:44.244 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:43 smithi158 podman[72906]: 2024-02-18 11:55:43.986900581 +0000 UTC m=+0.577621761 container remove ad4c95a3f804173a0847b9e75cb9c47209fa551b1531c0000747bc8d63f7c8c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0) 2024-02-18T11:55:44.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:44 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:44.089+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:21.400001+0000 front 2024-02-18T11:54:21.400128+0000 (oldest deadline 2024-02-18T11:54:48.400621+0000) 2024-02-18T11:55:44.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:44 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:44.089+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.901802+0000 front 2024-02-18T11:54:28.901977+0000 (oldest deadline 2024-02-18T11:54:53.001759+0000) 2024-02-18T11:55:44.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:44 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:44.089+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.302728+0000 front 2024-02-18T11:54:38.302766+0000 (oldest deadline 2024-02-18T11:55:03.602861+0000) 2024-02-18T11:55:44.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:44 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:44.089+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6812 osd.4 since back 2024-02-18T11:54:52.304990+0000 front 2024-02-18T11:54:52.304951+0000 (oldest deadline 2024-02-18T11:55:17.605033+0000) 2024-02-18T11:55:44.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:44 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:44.089+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-02-18T11:55:03.406514+0000 front 2024-02-18T11:55:03.406461+0000 (oldest deadline 2024-02-18T11:55:29.207182+0000) 2024-02-18T11:55:44.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:44 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:44.089+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-02-18T11:55:09.208076+0000 front 2024-02-18T11:55:09.207926+0000 (oldest deadline 2024-02-18T11:55:37.408172+0000) 2024-02-18T11:55:44.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:44 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:44.089+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6820 osd.7 since back 2024-02-18T11:55:17.408819+0000 front 2024-02-18T11:55:17.408719+0000 (oldest deadline 2024-02-18T11:55:43.308620+0000) 2024-02-18T11:55:44.708 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:44 smithi158 podman[72998]: 2024-02-18 11:55:44.379321716 +0000 UTC m=+0.111299343 container create 98c6b371426264fa36aa8b929e2087c5f9cc30e543f017b2f6913de5024c7199 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10-deactivate, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-02-18T11:55:44.709 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:44 smithi158 podman[72998]: 2024-02-18 11:55:44.28562149 +0000 UTC m=+0.017599141 image pull 53c33ee3bf62dc411b650f86eedbbee7698efb1001b4c063271dcd8d65b41165 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1 2024-02-18T11:55:44.709 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:44 smithi158 podman[72998]: 2024-02-18 11:55:44.623536307 +0000 UTC m=+0.355513940 container init 98c6b371426264fa36aa8b929e2087c5f9cc30e543f017b2f6913de5024c7199 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-02-18T11:55:44.709 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:44 smithi158 podman[72998]: 2024-02-18 11:55:44.626595549 +0000 UTC m=+0.358573185 container start 98c6b371426264fa36aa8b929e2087c5f9cc30e543f017b2f6913de5024c7199 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10-deactivate, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240217.015652-1c920af, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=) 2024-02-18T11:55:44.709 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:44 smithi158 podman[72998]: 2024-02-18 11:55:44.715205102 +0000 UTC m=+0.447182745 container attach 98c6b371426264fa36aa8b929e2087c5f9cc30e543f017b2f6913de5024c7199 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10-deactivate, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-02-18T11:55:44.994 INFO:journalctl@ceph.osd.10.smithi158.stdout:Feb 18 11:55:44 smithi158 podman[72998]: 2024-02-18 11:55:44.911301293 +0000 UTC m=+0.643278935 container died 98c6b371426264fa36aa8b929e2087c5f9cc30e543f017b2f6913de5024c7199 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-10-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-18T11:55:45.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:45 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:45.071+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:21.400001+0000 front 2024-02-18T11:54:21.400128+0000 (oldest deadline 2024-02-18T11:54:48.400621+0000) 2024-02-18T11:55:45.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:45 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:45.071+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.901802+0000 front 2024-02-18T11:54:28.901977+0000 (oldest deadline 2024-02-18T11:54:53.001759+0000) 2024-02-18T11:55:45.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:45 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:45.071+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.302728+0000 front 2024-02-18T11:54:38.302766+0000 (oldest deadline 2024-02-18T11:55:03.602861+0000) 2024-02-18T11:55:45.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:45 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:45.071+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6812 osd.4 since back 2024-02-18T11:54:52.304990+0000 front 2024-02-18T11:54:52.304951+0000 (oldest deadline 2024-02-18T11:55:17.605033+0000) 2024-02-18T11:55:45.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:45 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:45.071+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-02-18T11:55:03.406514+0000 front 2024-02-18T11:55:03.406461+0000 (oldest deadline 2024-02-18T11:55:29.207182+0000) 2024-02-18T11:55:45.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:45 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:45.071+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-02-18T11:55:09.208076+0000 front 2024-02-18T11:55:09.207926+0000 (oldest deadline 2024-02-18T11:55:37.408172+0000) 2024-02-18T11:55:45.355 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:45 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:45.071+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6820 osd.7 since back 2024-02-18T11:55:17.408819+0000 front 2024-02-18T11:55:17.408719+0000 (oldest deadline 2024-02-18T11:55:43.308620+0000) 2024-02-18T11:55:45.465 DEBUG:teuthology.orchestra.run.smithi158:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@osd.10.service' 2024-02-18T11:55:45.549 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-18T11:55:45.549 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-02-18T11:55:45.549 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-02-18T11:55:45.549 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@osd.11 2024-02-18T11:55:45.854 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:45 smithi188 systemd[1]: Stopping Ceph osd.11 for 0c8bb0de-ce51-11ee-95ba-87774f69a715... 2024-02-18T11:55:46.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:45 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:45.868+0000 7f1f4c8e0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-18T11:55:46.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:45 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:45.868+0000 7f1f4c8e0700 -1 osd.11 92 *** Got signal Terminated *** 2024-02-18T11:55:46.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:45 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:45.868+0000 7f1f4c8e0700 -1 osd.11 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-18T11:55:46.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:46 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:46.049+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:21.400001+0000 front 2024-02-18T11:54:21.400128+0000 (oldest deadline 2024-02-18T11:54:48.400621+0000) 2024-02-18T11:55:46.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:46 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:46.049+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.901802+0000 front 2024-02-18T11:54:28.901977+0000 (oldest deadline 2024-02-18T11:54:53.001759+0000) 2024-02-18T11:55:46.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:46 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:46.049+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.302728+0000 front 2024-02-18T11:54:38.302766+0000 (oldest deadline 2024-02-18T11:55:03.602861+0000) 2024-02-18T11:55:46.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:46 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:46.049+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6812 osd.4 since back 2024-02-18T11:54:52.304990+0000 front 2024-02-18T11:54:52.304951+0000 (oldest deadline 2024-02-18T11:55:17.605033+0000) 2024-02-18T11:55:46.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:46 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:46.049+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-02-18T11:55:03.406514+0000 front 2024-02-18T11:55:03.406461+0000 (oldest deadline 2024-02-18T11:55:29.207182+0000) 2024-02-18T11:55:46.355 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:46 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:46.049+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-02-18T11:55:09.208076+0000 front 2024-02-18T11:55:09.207926+0000 (oldest deadline 2024-02-18T11:55:37.408172+0000) 2024-02-18T11:55:46.355 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:46 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:46.049+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6820 osd.7 since back 2024-02-18T11:55:17.408819+0000 front 2024-02-18T11:55:17.408719+0000 (oldest deadline 2024-02-18T11:55:43.308620+0000) 2024-02-18T11:55:47.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:47 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:47.084+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:21.400001+0000 front 2024-02-18T11:54:21.400128+0000 (oldest deadline 2024-02-18T11:54:48.400621+0000) 2024-02-18T11:55:47.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:47 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:47.084+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.901802+0000 front 2024-02-18T11:54:28.901977+0000 (oldest deadline 2024-02-18T11:54:53.001759+0000) 2024-02-18T11:55:47.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:47 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:47.084+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.302728+0000 front 2024-02-18T11:54:38.302766+0000 (oldest deadline 2024-02-18T11:55:03.602861+0000) 2024-02-18T11:55:47.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:47 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:47.084+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6812 osd.4 since back 2024-02-18T11:54:52.304990+0000 front 2024-02-18T11:54:52.304951+0000 (oldest deadline 2024-02-18T11:55:17.605033+0000) 2024-02-18T11:55:47.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:47 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:47.084+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-02-18T11:55:03.406514+0000 front 2024-02-18T11:55:03.406461+0000 (oldest deadline 2024-02-18T11:55:29.207182+0000) 2024-02-18T11:55:47.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:47 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:47.084+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-02-18T11:55:09.208076+0000 front 2024-02-18T11:55:09.207926+0000 (oldest deadline 2024-02-18T11:55:37.408172+0000) 2024-02-18T11:55:47.355 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:47 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:47.084+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6820 osd.7 since back 2024-02-18T11:55:17.408819+0000 front 2024-02-18T11:55:17.408719+0000 (oldest deadline 2024-02-18T11:55:43.308620+0000) 2024-02-18T11:55:48.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:48 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:48.037+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:21.400001+0000 front 2024-02-18T11:54:21.400128+0000 (oldest deadline 2024-02-18T11:54:48.400621+0000) 2024-02-18T11:55:48.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:48 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:48.037+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.901802+0000 front 2024-02-18T11:54:28.901977+0000 (oldest deadline 2024-02-18T11:54:53.001759+0000) 2024-02-18T11:55:48.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:48 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:48.037+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.302728+0000 front 2024-02-18T11:54:38.302766+0000 (oldest deadline 2024-02-18T11:55:03.602861+0000) 2024-02-18T11:55:48.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:48 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:48.037+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6812 osd.4 since back 2024-02-18T11:54:52.304990+0000 front 2024-02-18T11:54:52.304951+0000 (oldest deadline 2024-02-18T11:55:17.605033+0000) 2024-02-18T11:55:48.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:48 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:48.037+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-02-18T11:55:03.406514+0000 front 2024-02-18T11:55:03.406461+0000 (oldest deadline 2024-02-18T11:55:29.207182+0000) 2024-02-18T11:55:48.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:48 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:48.037+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-02-18T11:55:09.208076+0000 front 2024-02-18T11:55:09.207926+0000 (oldest deadline 2024-02-18T11:55:37.408172+0000) 2024-02-18T11:55:48.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:48 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:48.037+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6820 osd.7 since back 2024-02-18T11:55:17.408819+0000 front 2024-02-18T11:55:17.408719+0000 (oldest deadline 2024-02-18T11:55:43.308620+0000) 2024-02-18T11:55:49.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:48 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:48.995+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:21.400001+0000 front 2024-02-18T11:54:21.400128+0000 (oldest deadline 2024-02-18T11:54:48.400621+0000) 2024-02-18T11:55:49.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:48 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:48.995+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.901802+0000 front 2024-02-18T11:54:28.901977+0000 (oldest deadline 2024-02-18T11:54:53.001759+0000) 2024-02-18T11:55:49.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:48 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:48.995+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.302728+0000 front 2024-02-18T11:54:38.302766+0000 (oldest deadline 2024-02-18T11:55:03.602861+0000) 2024-02-18T11:55:49.355 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:48 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:48.995+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6812 osd.4 since back 2024-02-18T11:54:52.304990+0000 front 2024-02-18T11:54:52.304951+0000 (oldest deadline 2024-02-18T11:55:17.605033+0000) 2024-02-18T11:55:49.355 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:48 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:48.995+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-02-18T11:55:03.406514+0000 front 2024-02-18T11:55:03.406461+0000 (oldest deadline 2024-02-18T11:55:29.207182+0000) 2024-02-18T11:55:49.355 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:48 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:48.995+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-02-18T11:55:09.208076+0000 front 2024-02-18T11:55:09.207926+0000 (oldest deadline 2024-02-18T11:55:37.408172+0000) 2024-02-18T11:55:49.355 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:48 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:48.995+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6820 osd.7 since back 2024-02-18T11:55:17.408819+0000 front 2024-02-18T11:55:17.408719+0000 (oldest deadline 2024-02-18T11:55:43.308620+0000) 2024-02-18T11:55:49.355 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:48 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:48.995+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.188:6820 osd.8 since back 2024-02-18T11:55:26.209412+0000 front 2024-02-18T11:55:26.209383+0000 (oldest deadline 2024-02-18T11:55:48.509484+0000) 2024-02-18T11:55:50.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:49 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:49.974+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-02-18T11:54:21.400001+0000 front 2024-02-18T11:54:21.400128+0000 (oldest deadline 2024-02-18T11:54:48.400621+0000) 2024-02-18T11:55:50.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:49 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:49.974+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6804 osd.1 since back 2024-02-18T11:54:28.901802+0000 front 2024-02-18T11:54:28.901977+0000 (oldest deadline 2024-02-18T11:54:53.001759+0000) 2024-02-18T11:55:50.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:49 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:49.974+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-02-18T11:54:38.302728+0000 front 2024-02-18T11:54:38.302766+0000 (oldest deadline 2024-02-18T11:55:03.602861+0000) 2024-02-18T11:55:50.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:49 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:49.974+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6812 osd.4 since back 2024-02-18T11:54:52.304990+0000 front 2024-02-18T11:54:52.304951+0000 (oldest deadline 2024-02-18T11:55:17.605033+0000) 2024-02-18T11:55:50.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:49 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:49.974+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-02-18T11:55:03.406514+0000 front 2024-02-18T11:55:03.406461+0000 (oldest deadline 2024-02-18T11:55:29.207182+0000) 2024-02-18T11:55:50.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:49 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:49.974+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-02-18T11:55:09.208076+0000 front 2024-02-18T11:55:09.207926+0000 (oldest deadline 2024-02-18T11:55:37.408172+0000) 2024-02-18T11:55:50.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:49 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:49.974+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.158:6820 osd.7 since back 2024-02-18T11:55:17.408819+0000 front 2024-02-18T11:55:17.408719+0000 (oldest deadline 2024-02-18T11:55:43.308620+0000) 2024-02-18T11:55:50.354 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:49 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11[62364]: 2024-02-18T11:55:49.974+0000 7f1f456cd700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.188:6820 osd.8 since back 2024-02-18T11:55:26.209412+0000 front 2024-02-18T11:55:26.209383+0000 (oldest deadline 2024-02-18T11:55:48.509484+0000) 2024-02-18T11:55:51.521 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:51 smithi188 podman[73557]: 2024-02-18 11:55:51.265455117 +0000 UTC m=+5.516829662 container died dce8e37b6a874f0a6ec25cca6c695f39f9254a7683308ba41408857b24659c9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-02-18T11:55:51.854 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:51 smithi188 podman[73557]: 2024-02-18 11:55:51.618566642 +0000 UTC m=+5.869941187 container cleanup dce8e37b6a874f0a6ec25cca6c695f39f9254a7683308ba41408857b24659c9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, org.label-schema.license=GPLv2) 2024-02-18T11:55:51.854 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:51 smithi188 bash[73557]: ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11 2024-02-18T11:55:52.215 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:51 smithi188 podman[73574]: 2024-02-18 11:55:51.878989469 +0000 UTC m=+0.608887639 container remove dce8e37b6a874f0a6ec25cca6c695f39f9254a7683308ba41408857b24659c9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-18T11:55:52.517 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:52 smithi188 podman[73668]: 2024-02-18 11:55:52.124319298 +0000 UTC m=+0.017524660 image pull 53c33ee3bf62dc411b650f86eedbbee7698efb1001b4c063271dcd8d65b41165 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1 2024-02-18T11:55:52.517 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:52 smithi188 podman[73668]: 2024-02-18 11:55:52.238118184 +0000 UTC m=+0.131323540 container create 7bfa04c4d2ad0a7882206d1319b5d894c5d437efef68a709dab280ad95291331 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11-deactivate, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-02-18T11:55:52.517 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:52 smithi188 podman[73668]: 2024-02-18 11:55:52.449280504 +0000 UTC m=+0.342485878 container init 7bfa04c4d2ad0a7882206d1319b5d894c5d437efef68a709dab280ad95291331 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-18T11:55:52.517 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:52 smithi188 podman[73668]: 2024-02-18 11:55:52.452683142 +0000 UTC m=+0.345888491 container start 7bfa04c4d2ad0a7882206d1319b5d894c5d437efef68a709dab280ad95291331 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True) 2024-02-18T11:55:52.517 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:52 smithi188 podman[73668]: 2024-02-18 11:55:52.523598109 +0000 UTC m=+0.416803491 container attach 7bfa04c4d2ad0a7882206d1319b5d894c5d437efef68a709dab280ad95291331 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11-deactivate, org.label-schema.build-date=20240214, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240217.015652-1c920af) 2024-02-18T11:55:52.854 INFO:journalctl@ceph.osd.11.smithi188.stdout:Feb 18 11:55:52 smithi188 podman[73668]: 2024-02-18 11:55:52.735689203 +0000 UTC m=+0.628894563 container died 7bfa04c4d2ad0a7882206d1319b5d894c5d437efef68a709dab280ad95291331 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240217.015652-1c920af, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2) 2024-02-18T11:55:53.347 DEBUG:teuthology.orchestra.run.smithi188:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@osd.11.service' 2024-02-18T11:55:53.386 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-18T11:55:53.386 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-02-18T11:55:53.387 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-02-18T11:55:53.387 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@mds.a 2024-02-18T11:55:53.673 INFO:journalctl@ceph.mds.a.smithi012.stdout:Feb 18 11:55:53 smithi012 systemd[1]: Stopping Ceph mds.a for 0c8bb0de-ce51-11ee-95ba-87774f69a715... 2024-02-18T11:55:54.173 INFO:journalctl@ceph.mds.a.smithi012.stdout:Feb 18 11:55:53 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-a[72553]: 2024-02-18T11:55:53.702+0000 7f7ed73ee700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-18T11:55:54.173 INFO:journalctl@ceph.mds.a.smithi012.stdout:Feb 18 11:55:53 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-a[72553]: 2024-02-18T11:55:53.702+0000 7f7ed73ee700 -1 mds.a *** got signal Terminated *** 2024-02-18T11:55:55.863 INFO:journalctl@ceph.mds.a.smithi012.stdout:Feb 18 11:55:55 smithi012 podman[88207]: 2024-02-18 11:55:55.60974088 +0000 UTC m=+2.012840192 container died 651883f871d93bee10b8a16dfb7ec4103b5a802fb93b9069e0d7377dae87ad27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-a, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-02-18T11:55:56.173 INFO:journalctl@ceph.mds.a.smithi012.stdout:Feb 18 11:55:55 smithi012 podman[88207]: 2024-02-18 11:55:55.960881277 +0000 UTC m=+2.363980582 container cleanup 651883f871d93bee10b8a16dfb7ec4103b5a802fb93b9069e0d7377dae87ad27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-a, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=) 2024-02-18T11:55:56.173 INFO:journalctl@ceph.mds.a.smithi012.stdout:Feb 18 11:55:55 smithi012 bash[88207]: ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-a 2024-02-18T11:55:56.332 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@mds.a.service' 2024-02-18T11:55:56.369 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-18T11:55:56.369 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-02-18T11:55:56.369 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-02-18T11:55:56.369 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@mds.d 2024-02-18T11:55:56.923 INFO:journalctl@ceph.mds.d.smithi012.stdout:Feb 18 11:55:56 smithi012 systemd[1]: Stopping Ceph mds.d for 0c8bb0de-ce51-11ee-95ba-87774f69a715... 2024-02-18T11:55:56.923 INFO:journalctl@ceph.mds.d.smithi012.stdout:Feb 18 11:55:56 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-d[71589]: 2024-02-18T11:55:56.680+0000 7f01cd8a1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-18T11:55:56.923 INFO:journalctl@ceph.mds.d.smithi012.stdout:Feb 18 11:55:56 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-d[71589]: 2024-02-18T11:55:56.680+0000 7f01cd8a1700 -1 mds.d *** got signal Terminated *** 2024-02-18T11:56:00.016 INFO:journalctl@ceph.mds.d.smithi012.stdout:Feb 18 11:55:59 smithi012 podman[88326]: 2024-02-18 11:55:59.767466185 +0000 UTC m=+3.171978589 container died bb1d1e660e10b85641096fd76db7438805f424e2e9f306662311ee2655839cee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-d, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, RELEASE=wip-vshankar-testing-20240217.015652-1c920af) 2024-02-18T11:56:00.423 INFO:journalctl@ceph.mds.d.smithi012.stdout:Feb 18 11:56:00 smithi012 podman[88326]: 2024-02-18 11:56:00.113975973 +0000 UTC m=+3.518488388 container cleanup bb1d1e660e10b85641096fd76db7438805f424e2e9f306662311ee2655839cee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True) 2024-02-18T11:56:00.423 INFO:journalctl@ceph.mds.d.smithi012.stdout:Feb 18 11:56:00 smithi012 bash[88326]: ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-d 2024-02-18T11:56:00.423 INFO:journalctl@ceph.mds.d.smithi012.stdout:Feb 18 11:56:00 smithi012 podman[88341]: 2024-02-18 11:56:00.265001325 +0000 UTC m=+0.491872147 container remove bb1d1e660e10b85641096fd76db7438805f424e2e9f306662311ee2655839cee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-d, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-02-18T11:56:00.490 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@mds.d.service' 2024-02-18T11:56:00.524 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-18T11:56:00.525 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-02-18T11:56:00.525 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-02-18T11:56:00.525 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@mds.g 2024-02-18T11:56:00.852 INFO:journalctl@ceph.mds.g.smithi012.stdout:Feb 18 11:56:00 smithi012 systemd[1]: Stopping Ceph mds.g for 0c8bb0de-ce51-11ee-95ba-87774f69a715... 2024-02-18T11:56:01.173 INFO:journalctl@ceph.mds.g.smithi012.stdout:Feb 18 11:56:00 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-g[72195]: 2024-02-18T11:56:00.858+0000 7f5a61a79700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-18T11:56:01.173 INFO:journalctl@ceph.mds.g.smithi012.stdout:Feb 18 11:56:00 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-g[72195]: 2024-02-18T11:56:00.858+0000 7f5a61a79700 -1 mds.g *** got signal Terminated *** 2024-02-18T11:56:04.862 INFO:journalctl@ceph.mds.g.smithi012.stdout:Feb 18 11:56:04 smithi012 podman[88447]: 2024-02-18 11:56:04.59178679 +0000 UTC m=+3.840287805 container died 2d26a2be1b71391324e8748deec8596b86578bf837ffc8236e894069a4af03b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-g, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-02-18T11:56:05.173 INFO:journalctl@ceph.mds.g.smithi012.stdout:Feb 18 11:56:04 smithi012 podman[88447]: 2024-02-18 11:56:04.950642943 +0000 UTC m=+4.199143957 container cleanup 2d26a2be1b71391324e8748deec8596b86578bf837ffc8236e894069a4af03b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-g, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, io.buildah.version=1.33.5) 2024-02-18T11:56:05.173 INFO:journalctl@ceph.mds.g.smithi012.stdout:Feb 18 11:56:04 smithi012 bash[88447]: ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-g 2024-02-18T11:56:05.424 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@mds.g.service' 2024-02-18T11:56:05.459 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-18T11:56:05.459 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-02-18T11:56:05.459 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-02-18T11:56:05.459 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@mds.j 2024-02-18T11:56:05.773 INFO:journalctl@ceph.mds.j.smithi012.stdout:Feb 18 11:56:05 smithi012 systemd[1]: Stopping Ceph mds.j for 0c8bb0de-ce51-11ee-95ba-87774f69a715... 2024-02-18T11:56:06.173 INFO:journalctl@ceph.mds.j.smithi012.stdout:Feb 18 11:56:05 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-j[71198]: 2024-02-18T11:56:05.778+0000 7ff310151700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-18T11:56:06.173 INFO:journalctl@ceph.mds.j.smithi012.stdout:Feb 18 11:56:05 smithi012 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-j[71198]: 2024-02-18T11:56:05.778+0000 7ff310151700 -1 mds.j *** got signal Terminated *** 2024-02-18T11:56:09.273 INFO:journalctl@ceph.mds.j.smithi012.stdout:Feb 18 11:56:09 smithi012 podman[88568]: 2024-02-18 11:56:09.022269185 +0000 UTC m=+3.329733058 container died b73432e950b98e782de80edc57cda36c7a2abf37e1ec54f05409689099cf5e45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-j, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD) 2024-02-18T11:56:09.673 INFO:journalctl@ceph.mds.j.smithi012.stdout:Feb 18 11:56:09 smithi012 podman[88568]: 2024-02-18 11:56:09.361958577 +0000 UTC m=+3.669422449 container cleanup b73432e950b98e782de80edc57cda36c7a2abf37e1ec54f05409689099cf5e45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-j, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240217.015652-1c920af, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, ceph=True) 2024-02-18T11:56:09.674 INFO:journalctl@ceph.mds.j.smithi012.stdout:Feb 18 11:56:09 smithi012 bash[88568]: ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-j 2024-02-18T11:56:09.674 INFO:journalctl@ceph.mds.j.smithi012.stdout:Feb 18 11:56:09 smithi012 podman[88582]: 2024-02-18 11:56:09.512764879 +0000 UTC m=+0.489213541 container remove b73432e950b98e782de80edc57cda36c7a2abf37e1ec54f05409689099cf5e45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-j, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240214, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-02-18T11:56:09.742 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@mds.j.service' 2024-02-18T11:56:09.775 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-18T11:56:09.776 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-02-18T11:56:09.776 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-02-18T11:56:09.776 DEBUG:teuthology.orchestra.run.smithi158:> sudo systemctl stop ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@mds.b 2024-02-18T11:56:10.071 INFO:journalctl@ceph.mds.b.smithi158.stdout:Feb 18 11:56:09 smithi158 systemd[1]: Stopping Ceph mds.b for 0c8bb0de-ce51-11ee-95ba-87774f69a715... 2024-02-18T11:56:10.494 INFO:journalctl@ceph.mds.b.smithi158.stdout:Feb 18 11:56:10 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-b[66835]: 2024-02-18T11:56:10.102+0000 7f3f42e5a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-18T11:56:10.494 INFO:journalctl@ceph.mds.b.smithi158.stdout:Feb 18 11:56:10 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-b[66835]: 2024-02-18T11:56:10.102+0000 7f3f42e5a700 -1 mds.b *** got signal Terminated *** 2024-02-18T11:56:11.705 INFO:journalctl@ceph.mds.b.smithi158.stdout:Feb 18 11:56:11 smithi158 podman[73103]: 2024-02-18 11:56:11.452114471 +0000 UTC m=+1.474983719 container died d083ddecd08ab8952b58583c7d93de4bb0c4e8089ef4874cf9502feca89761cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-b, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, ceph=True) 2024-02-18T11:56:11.994 INFO:journalctl@ceph.mds.b.smithi158.stdout:Feb 18 11:56:11 smithi158 podman[73103]: 2024-02-18 11:56:11.819160768 +0000 UTC m=+1.842030015 container cleanup d083ddecd08ab8952b58583c7d93de4bb0c4e8089ef4874cf9502feca89761cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-02-18T11:56:11.994 INFO:journalctl@ceph.mds.b.smithi158.stdout:Feb 18 11:56:11 smithi158 bash[73103]: ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-b 2024-02-18T11:56:12.236 DEBUG:teuthology.orchestra.run.smithi158:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@mds.b.service' 2024-02-18T11:56:12.276 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-18T11:56:12.276 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-02-18T11:56:12.276 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-02-18T11:56:12.276 DEBUG:teuthology.orchestra.run.smithi158:> sudo systemctl stop ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@mds.e 2024-02-18T11:56:12.651 INFO:journalctl@ceph.mds.e.smithi158.stdout:Feb 18 11:56:12 smithi158 systemd[1]: Stopping Ceph mds.e for 0c8bb0de-ce51-11ee-95ba-87774f69a715... 2024-02-18T11:56:12.994 INFO:journalctl@ceph.mds.e.smithi158.stdout:Feb 18 11:56:12 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-e[66203]: 2024-02-18T11:56:12.656+0000 7f98c18bc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-18T11:56:12.994 INFO:journalctl@ceph.mds.e.smithi158.stdout:Feb 18 11:56:12 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-e[66203]: 2024-02-18T11:56:12.656+0000 7f98c18bc700 -1 mds.e *** got signal Terminated *** 2024-02-18T11:56:15.735 INFO:journalctl@ceph.mds.e.smithi158.stdout:Feb 18 11:56:15 smithi158 podman[73233]: 2024-02-18 11:56:15.444322295 +0000 UTC m=+2.937049337 container died b0f4c55b33cd7cee45e3933153f3253031e7d02a5f357e8105efccbf8c21225f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-e, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-02-18T11:56:15.995 INFO:journalctl@ceph.mds.e.smithi158.stdout:Feb 18 11:56:15 smithi158 podman[73233]: 2024-02-18 11:56:15.823290251 +0000 UTC m=+3.316017284 container cleanup b0f4c55b33cd7cee45e3933153f3253031e7d02a5f357e8105efccbf8c21225f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-e, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-02-18T11:56:15.995 INFO:journalctl@ceph.mds.e.smithi158.stdout:Feb 18 11:56:15 smithi158 bash[73233]: ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-e 2024-02-18T11:56:16.235 DEBUG:teuthology.orchestra.run.smithi158:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@mds.e.service' 2024-02-18T11:56:16.271 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-18T11:56:16.271 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-02-18T11:56:16.271 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-02-18T11:56:16.272 DEBUG:teuthology.orchestra.run.smithi158:> sudo systemctl stop ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@mds.h 2024-02-18T11:56:16.744 INFO:journalctl@ceph.mds.h.smithi158.stdout:Feb 18 11:56:16 smithi158 systemd[1]: Stopping Ceph mds.h for 0c8bb0de-ce51-11ee-95ba-87774f69a715... 2024-02-18T11:56:16.744 INFO:journalctl@ceph.mds.h.smithi158.stdout:Feb 18 11:56:16 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-h[66515]: 2024-02-18T11:56:16.601+0000 7fb6e5d59700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-18T11:56:16.745 INFO:journalctl@ceph.mds.h.smithi158.stdout:Feb 18 11:56:16 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-h[66515]: 2024-02-18T11:56:16.601+0000 7fb6e5d59700 -1 mds.h *** got signal Terminated *** 2024-02-18T11:56:19.722 INFO:journalctl@ceph.mds.h.smithi158.stdout:Feb 18 11:56:19 smithi158 podman[73353]: 2024-02-18 11:56:19.448957137 +0000 UTC m=+2.953747532 container died 93295aab312378f4cd4e1fd17aab5d39b956922813f83a0f306a5cd5e69fff7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-h, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-02-18T11:56:19.994 INFO:journalctl@ceph.mds.h.smithi158.stdout:Feb 18 11:56:19 smithi158 podman[73353]: 2024-02-18 11:56:19.794096516 +0000 UTC m=+3.298886928 container cleanup 93295aab312378f4cd4e1fd17aab5d39b956922813f83a0f306a5cd5e69fff7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-h, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True) 2024-02-18T11:56:19.995 INFO:journalctl@ceph.mds.h.smithi158.stdout:Feb 18 11:56:19 smithi158 bash[73353]: ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-h 2024-02-18T11:56:20.256 DEBUG:teuthology.orchestra.run.smithi158:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@mds.h.service' 2024-02-18T11:56:20.280 INFO:journalctl@ceph.mds.h.smithi158.stdout:Feb 18 11:56:20 smithi158 podman[73367]: 2024-02-18 11:56:20.023617786 +0000 UTC m=+0.570971253 container remove 93295aab312378f4cd4e1fd17aab5d39b956922813f83a0f306a5cd5e69fff7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-h, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-18T11:56:20.280 INFO:journalctl@ceph.mds.h.smithi158.stdout:Feb 18 11:56:20 smithi158 systemd[1]: ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@mds.h.service: Deactivated successfully. 2024-02-18T11:56:20.280 INFO:journalctl@ceph.mds.h.smithi158.stdout:Feb 18 11:56:20 smithi158 systemd[1]: Stopped Ceph mds.h for 0c8bb0de-ce51-11ee-95ba-87774f69a715. 2024-02-18T11:56:20.280 INFO:journalctl@ceph.mds.h.smithi158.stdout:Feb 18 11:56:20 smithi158 systemd[1]: ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@mds.h.service: Consumed 1min 41.003s CPU time. 2024-02-18T11:56:20.291 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-18T11:56:20.291 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-02-18T11:56:20.292 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-02-18T11:56:20.292 DEBUG:teuthology.orchestra.run.smithi158:> sudo systemctl stop ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@mds.k 2024-02-18T11:56:20.617 INFO:journalctl@ceph.mds.k.smithi158.stdout:Feb 18 11:56:20 smithi158 systemd[1]: Stopping Ceph mds.k for 0c8bb0de-ce51-11ee-95ba-87774f69a715... 2024-02-18T11:56:20.995 INFO:journalctl@ceph.mds.k.smithi158.stdout:Feb 18 11:56:20 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-k[65881]: 2024-02-18T11:56:20.622+0000 7fd901a89700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-18T11:56:20.995 INFO:journalctl@ceph.mds.k.smithi158.stdout:Feb 18 11:56:20 smithi158 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-k[65881]: 2024-02-18T11:56:20.622+0000 7fd901a89700 -1 mds.k *** got signal Terminated *** 2024-02-18T11:56:22.668 INFO:journalctl@ceph.mds.k.smithi158.stdout:Feb 18 11:56:22 smithi158 podman[73475]: 2024-02-18 11:56:22.354410778 +0000 UTC m=+1.837434119 container died fa7b6fd22c88c458cbf3435f779212be76c1baffae13ca3f7cc6db5438896bf2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-k, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-18T11:56:22.992 INFO:journalctl@ceph.mds.k.smithi158.stdout:Feb 18 11:56:22 smithi158 podman[73475]: 2024-02-18 11:56:22.673199429 +0000 UTC m=+2.156222837 container cleanup fa7b6fd22c88c458cbf3435f779212be76c1baffae13ca3f7cc6db5438896bf2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-k, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-18T11:56:22.993 INFO:journalctl@ceph.mds.k.smithi158.stdout:Feb 18 11:56:22 smithi158 bash[73475]: ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-k 2024-02-18T11:56:22.993 INFO:journalctl@ceph.mds.k.smithi158.stdout:Feb 18 11:56:22 smithi158 podman[73489]: 2024-02-18 11:56:22.88263828 +0000 UTC m=+0.521923852 container remove fa7b6fd22c88c458cbf3435f779212be76c1baffae13ca3f7cc6db5438896bf2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-02-18T11:56:23.007 DEBUG:teuthology.orchestra.run.smithi158:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@mds.k.service' 2024-02-18T11:56:23.042 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-18T11:56:23.043 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-02-18T11:56:23.043 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-02-18T11:56:23.043 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@mds.c 2024-02-18T11:56:23.337 INFO:journalctl@ceph.mds.c.smithi188.stdout:Feb 18 11:56:23 smithi188 systemd[1]: Stopping Ceph mds.c for 0c8bb0de-ce51-11ee-95ba-87774f69a715... 2024-02-18T11:56:23.596 INFO:journalctl@ceph.mds.c.smithi188.stdout:Feb 18 11:56:23 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-c[67595]: 2024-02-18T11:56:23.433+0000 7f905ee4d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-18T11:56:23.596 INFO:journalctl@ceph.mds.c.smithi188.stdout:Feb 18 11:56:23 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-c[67595]: 2024-02-18T11:56:23.433+0000 7f905ee4d700 -1 mds.c *** got signal Terminated *** 2024-02-18T11:56:27.685 INFO:journalctl@ceph.mds.c.smithi188.stdout:Feb 18 11:56:27 smithi188 podman[73772]: 2024-02-18 11:56:27.432645717 +0000 UTC m=+4.194581011 container died b51fc279ee8ac6bfc4b836b67a96b29832d13e549ebda5a6a3cad5a17269bb1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-18T11:56:27.936 INFO:journalctl@ceph.mds.c.smithi188.stdout:Feb 18 11:56:27 smithi188 podman[73772]: 2024-02-18 11:56:27.765926682 +0000 UTC m=+4.527861977 container cleanup b51fc279ee8ac6bfc4b836b67a96b29832d13e549ebda5a6a3cad5a17269bb1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-c, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-02-18T11:56:27.936 INFO:journalctl@ceph.mds.c.smithi188.stdout:Feb 18 11:56:27 smithi188 bash[73772]: ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-c 2024-02-18T11:56:28.171 DEBUG:teuthology.orchestra.run.smithi188:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@mds.c.service' 2024-02-18T11:56:28.196 INFO:journalctl@ceph.mds.c.smithi188.stdout:Feb 18 11:56:27 smithi188 podman[73798]: 2024-02-18 11:56:27.941930996 +0000 UTC m=+0.508938445 container remove b51fc279ee8ac6bfc4b836b67a96b29832d13e549ebda5a6a3cad5a17269bb1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-c, ceph=True, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-02-18T11:56:28.196 INFO:journalctl@ceph.mds.c.smithi188.stdout:Feb 18 11:56:28 smithi188 systemd[1]: ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@mds.c.service: Deactivated successfully. 2024-02-18T11:56:28.196 INFO:journalctl@ceph.mds.c.smithi188.stdout:Feb 18 11:56:28 smithi188 systemd[1]: Stopped Ceph mds.c for 0c8bb0de-ce51-11ee-95ba-87774f69a715. 2024-02-18T11:56:28.196 INFO:journalctl@ceph.mds.c.smithi188.stdout:Feb 18 11:56:28 smithi188 systemd[1]: ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@mds.c.service: Consumed 1.925s CPU time. 2024-02-18T11:56:28.209 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-18T11:56:28.209 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-02-18T11:56:28.210 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-02-18T11:56:28.210 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@mds.f 2024-02-18T11:56:28.547 INFO:journalctl@ceph.mds.f.smithi188.stdout:Feb 18 11:56:28 smithi188 systemd[1]: Stopping Ceph mds.f for 0c8bb0de-ce51-11ee-95ba-87774f69a715... 2024-02-18T11:56:28.854 INFO:journalctl@ceph.mds.f.smithi188.stdout:Feb 18 11:56:28 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-f[67292]: 2024-02-18T11:56:28.551+0000 7f8db2bf5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-18T11:56:28.854 INFO:journalctl@ceph.mds.f.smithi188.stdout:Feb 18 11:56:28 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-f[67292]: 2024-02-18T11:56:28.551+0000 7f8db2bf5700 -1 mds.f *** got signal Terminated *** 2024-02-18T11:56:31.689 INFO:journalctl@ceph.mds.f.smithi188.stdout:Feb 18 11:56:31 smithi188 podman[73904]: 2024-02-18 11:56:31.444698754 +0000 UTC m=+3.009614640 container died 38349ecccb970532609eb26ed89e33c7e22d4f17cd87ee6a73fb798e5b913ed4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-f, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240217.015652-1c920af) 2024-02-18T11:56:32.104 INFO:journalctl@ceph.mds.f.smithi188.stdout:Feb 18 11:56:31 smithi188 podman[73904]: 2024-02-18 11:56:31.769201184 +0000 UTC m=+3.334117061 container cleanup 38349ecccb970532609eb26ed89e33c7e22d4f17cd87ee6a73fb798e5b913ed4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-f, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-18T11:56:32.104 INFO:journalctl@ceph.mds.f.smithi188.stdout:Feb 18 11:56:31 smithi188 bash[73904]: ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-f 2024-02-18T11:56:32.104 INFO:journalctl@ceph.mds.f.smithi188.stdout:Feb 18 11:56:31 smithi188 podman[73918]: 2024-02-18 11:56:31.937032457 +0000 UTC m=+0.487698704 container remove 38349ecccb970532609eb26ed89e33c7e22d4f17cd87ee6a73fb798e5b913ed4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-f, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-02-18T11:56:32.184 DEBUG:teuthology.orchestra.run.smithi188:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@mds.f.service' 2024-02-18T11:56:32.220 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-18T11:56:32.220 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-02-18T11:56:32.220 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-02-18T11:56:32.220 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@mds.i 2024-02-18T11:56:32.542 INFO:journalctl@ceph.mds.i.smithi188.stdout:Feb 18 11:56:32 smithi188 systemd[1]: Stopping Ceph mds.i for 0c8bb0de-ce51-11ee-95ba-87774f69a715... 2024-02-18T11:56:32.854 INFO:journalctl@ceph.mds.i.smithi188.stdout:Feb 18 11:56:32 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-i[67902]: 2024-02-18T11:56:32.547+0000 7f3cf8854700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-18T11:56:32.854 INFO:journalctl@ceph.mds.i.smithi188.stdout:Feb 18 11:56:32 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-i[67902]: 2024-02-18T11:56:32.547+0000 7f3cf8854700 -1 mds.i *** got signal Terminated *** 2024-02-18T11:56:35.825 INFO:journalctl@ceph.mds.i.smithi188.stdout:Feb 18 11:56:35 smithi188 podman[74025]: 2024-02-18 11:56:35.445134015 +0000 UTC m=+3.004134869 container died 7a8b1bd187ae09241dfe967ccff872543710cee7044ef24d2f84292ce2c8f654 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-i, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-02-18T11:56:36.104 INFO:journalctl@ceph.mds.i.smithi188.stdout:Feb 18 11:56:35 smithi188 podman[74025]: 2024-02-18 11:56:35.905721142 +0000 UTC m=+3.464722044 container cleanup 7a8b1bd187ae09241dfe967ccff872543710cee7044ef24d2f84292ce2c8f654 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-i, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-18T11:56:36.105 INFO:journalctl@ceph.mds.i.smithi188.stdout:Feb 18 11:56:35 smithi188 bash[74025]: ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-i 2024-02-18T11:56:36.371 DEBUG:teuthology.orchestra.run.smithi188:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@mds.i.service' 2024-02-18T11:56:36.406 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-18T11:56:36.406 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-02-18T11:56:36.406 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-02-18T11:56:36.406 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@mds.l 2024-02-18T11:56:36.745 INFO:journalctl@ceph.mds.l.smithi188.stdout:Feb 18 11:56:36 smithi188 systemd[1]: Stopping Ceph mds.l for 0c8bb0de-ce51-11ee-95ba-87774f69a715... 2024-02-18T11:56:37.105 INFO:journalctl@ceph.mds.l.smithi188.stdout:Feb 18 11:56:36 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-l[66987]: 2024-02-18T11:56:36.750+0000 7f4c366bb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-18T11:56:37.105 INFO:journalctl@ceph.mds.l.smithi188.stdout:Feb 18 11:56:36 smithi188 ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-l[66987]: 2024-02-18T11:56:36.750+0000 7f4c366bb700 -1 mds.l *** got signal Terminated *** 2024-02-18T11:56:39.670 INFO:journalctl@ceph.mds.l.smithi188.stdout:Feb 18 11:56:39 smithi188 podman[74146]: 2024-02-18 11:56:39.3941065 +0000 UTC m=+2.769488010 container died c3ce165c7789c08dc5b5574447e32f69e2f2e65fe24760a6ba65a38bbbe2b7b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-l, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-02-18T11:56:40.021 INFO:journalctl@ceph.mds.l.smithi188.stdout:Feb 18 11:56:39 smithi188 podman[74146]: 2024-02-18 11:56:39.767272505 +0000 UTC m=+3.142654004 container cleanup c3ce165c7789c08dc5b5574447e32f69e2f2e65fe24760a6ba65a38bbbe2b7b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:992baf60cd2f0a9ce62859e53f072d242a6594e3ef42d0bd252f0f962f6382b1, name=ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-l, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240217.015652-1c920af, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-18T11:56:40.021 INFO:journalctl@ceph.mds.l.smithi188.stdout:Feb 18 11:56:39 smithi188 bash[74146]: ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715-mds-l 2024-02-18T11:56:40.197 DEBUG:teuthology.orchestra.run.smithi188:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0c8bb0de-ce51-11ee-95ba-87774f69a715@mds.l.service' 2024-02-18T11:56:40.230 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-18T11:56:40.230 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-02-18T11:56:40.230 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 --force --keep-logs 2024-02-18T11:56:40.427 INFO:teuthology.orchestra.run.smithi012.stdout:Deleting cluster with fsid: 0c8bb0de-ce51-11ee-95ba-87774f69a715 2024-02-18T11:56:42.837 DEBUG:teuthology.orchestra.run.smithi158:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 --force --keep-logs 2024-02-18T11:56:43.030 INFO:teuthology.orchestra.run.smithi158.stdout:Deleting cluster with fsid: 0c8bb0de-ce51-11ee-95ba-87774f69a715 2024-02-18T11:56:45.636 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 --force --keep-logs 2024-02-18T11:56:45.826 INFO:teuthology.orchestra.run.smithi188.stdout:Deleting cluster with fsid: 0c8bb0de-ce51-11ee-95ba-87774f69a715 2024-02-18T11:56:48.269 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-18T11:56:48.305 DEBUG:teuthology.orchestra.run.smithi158:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-18T11:56:48.339 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-18T11:56:48.369 INFO:tasks.cephadm:Archiving crash dumps... 2024-02-18T11:56:48.371 DEBUG:teuthology.misc:Transferring archived files from smithi012:/var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/crash to /home/teuthworker/archive/vshankar-2024-02-17_05:57:56-fs-wip-vshankar-testing-20240217.015652-testing-default-smithi/7564362/remote/smithi012/crash 2024-02-18T11:56:48.371 DEBUG:teuthology.orchestra.run.smithi012:> sudo tar c -f - -C /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/crash -- . 2024-02-18T11:56:48.401 INFO:teuthology.orchestra.run.smithi012.stderr:tar: /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/crash: Cannot open: No such file or directory 2024-02-18T11:56:48.402 INFO:teuthology.orchestra.run.smithi012.stderr:tar: Error is not recoverable: exiting now 2024-02-18T11:56:48.403 DEBUG:teuthology.misc:Transferring archived files from smithi158:/var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/crash to /home/teuthworker/archive/vshankar-2024-02-17_05:57:56-fs-wip-vshankar-testing-20240217.015652-testing-default-smithi/7564362/remote/smithi158/crash 2024-02-18T11:56:48.404 DEBUG:teuthology.orchestra.run.smithi158:> sudo tar c -f - -C /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/crash -- . 2024-02-18T11:56:48.444 INFO:teuthology.orchestra.run.smithi158.stderr:tar: /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/crash: Cannot open: No such file or directory 2024-02-18T11:56:48.444 INFO:teuthology.orchestra.run.smithi158.stderr:tar: Error is not recoverable: exiting now 2024-02-18T11:56:48.446 DEBUG:teuthology.misc:Transferring archived files from smithi188:/var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/crash to /home/teuthworker/archive/vshankar-2024-02-17_05:57:56-fs-wip-vshankar-testing-20240217.015652-testing-default-smithi/7564362/remote/smithi188/crash 2024-02-18T11:56:48.447 DEBUG:teuthology.orchestra.run.smithi188:> sudo tar c -f - -C /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/crash -- . 2024-02-18T11:56:48.489 INFO:teuthology.orchestra.run.smithi188.stderr:tar: /var/lib/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/crash: Cannot open: No such file or directory 2024-02-18T11:56:48.489 INFO:teuthology.orchestra.run.smithi188.stderr:tar: Error is not recoverable: exiting now 2024-02-18T11:56:48.491 INFO:tasks.cephadm:Checking cluster log for badness... 2024-02-18T11:56:48.491 DEBUG:teuthology.orchestra.run.smithi012:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-02-18T11:56:48.529 INFO:teuthology.orchestra.run.smithi012.stdout:2024-02-18T11:32:48.888403+0000 mon.a (mon.0) 190 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-02-18T11:56:48.530 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-02-18T11:56:48.530 DEBUG:teuthology.orchestra.run.smithi012:> sudo egrep '\[SEC\]' /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-02-18T11:56:48.605 DEBUG:teuthology.orchestra.run.smithi012:> sudo egrep '\[ERR\]' /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-02-18T11:56:48.678 DEBUG:teuthology.orchestra.run.smithi012:> sudo egrep '\[WRN\]' /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-02-18T11:56:48.751 INFO:teuthology.orchestra.run.smithi012.stdout:2024-02-18T11:32:48.888403+0000 mon.a (mon.0) 190 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-02-18T11:56:48.751 INFO:tasks.cephadm:Compressing logs... 2024-02-18T11:56:48.751 DEBUG:teuthology.orchestra.run.smithi012:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-18T11:56:48.794 DEBUG:teuthology.orchestra.run.smithi158:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-18T11:56:48.796 DEBUG:teuthology.orchestra.run.smithi188:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-18T11:56:48.821 INFO:teuthology.orchestra.run.smithi012.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-02-18T11:56:48.821 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-18T11:56:48.821 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-osd.6.log 2024-02-18T11:56:48.821 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-mon.a.log 2024-02-18T11:56:48.821 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph.audit.log 2024-02-18T11:56:48.824 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-mds.g.log 2024-02-18T11:56:48.825 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-mds.j.log 2024-02-18T11:56:48.825 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-osd.3.log 2024-02-18T11:56:48.825 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-mgr.x.log 2024-02-18T11:56:48.825 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-osd.0.log 2024-02-18T11:56:48.825 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-mgr.x.log: 90.1% -- replaced with /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph.audit.log.gz 2024-02-18T11:56:48.826 INFO:teuthology.orchestra.run.smithi188.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-18T11:56:48.826 INFO:teuthology.orchestra.run.smithi188.stderr:: No such file or directory 2024-02-18T11:56:48.826 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-mds.l.log 2024-02-18T11:56:48.827 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph.audit.log 2024-02-18T11:56:48.827 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-mds.i.log 2024-02-18T11:56:48.827 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-osd.11.log 2024-02-18T11:56:48.827 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-mgr.z.log 2024-02-18T11:56:48.831 INFO:teuthology.orchestra.run.smithi012.stderr: 94.4% -- replaced with /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-mds.j.log.gz 2024-02-18T11:56:48.831 INFO:teuthology.orchestra.run.smithi012.stderr: 94.5% -- replaced with /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-mds.g.log.gz 2024-02-18T11:56:48.831 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph.cephadm.log 2024-02-18T11:56:48.831 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph.log 2024-02-18T11:56:48.831 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph.cephadm.log: 93.3%gzip -5 --verbose -- /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-volume.log 2024-02-18T11:56:48.831 INFO:teuthology.orchestra.run.smithi012.stderr: -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-18T11:56:48.831 INFO:teuthology.orchestra.run.smithi012.stderr: 82.8% -- replaced with /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph.cephadm.log.gz 2024-02-18T11:56:48.834 INFO:teuthology.orchestra.run.smithi158.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-02-18T11:56:48.837 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph.log: 86.0% -- replaced with /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph.log.gz 2024-02-18T11:56:48.837 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-osd.9.log 2024-02-18T11:56:48.837 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-mds.d.log 2024-02-18T11:56:48.837 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-mds.a.log 2024-02-18T11:56:48.839 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-mds.i.log: /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-osd.8.log 2024-02-18T11:56:48.839 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5/var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-mgr.z.log: --verbose -- /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-osd.5.log 2024-02-18T11:56:48.839 INFO:teuthology.orchestra.run.smithi188.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-18T11:56:48.839 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-mds.f.log 2024-02-18T11:56:48.839 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph.cephadm.log 2024-02-18T11:56:48.839 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-mds.f.log: /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-osd.5.log: 90.3% -- replaced with /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph.audit.log.gz 2024-02-18T11:56:48.839 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph.log 2024-02-18T11:56:48.839 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph.cephadm.log: 82.6% -- replaced with /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph.cephadm.log.gz 2024-02-18T11:56:48.839 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-osd.2.log 2024-02-18T11:56:48.839 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph.log: 94.4% -- replaced with /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-mds.l.log.gz 2024-02-18T11:56:48.839 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-mds.c.log 2024-02-18T11:56:48.839 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-osd.2.log: 86.0% -- replaced with /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph.log.gz 2024-02-18T11:56:48.839 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-volume.log 2024-02-18T11:56:48.839 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-mds.c.log: 92.9% -- replaced with /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-mgr.z.log.gz 2024-02-18T11:56:48.840 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-mon.c.log 2024-02-18T11:56:48.841 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-mds.d.log: 94.5% -- replaced with /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-mds.d.log.gz 2024-02-18T11:56:48.841 INFO:teuthology.orchestra.run.smithi158.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-18T11:56:48.841 INFO:teuthology.orchestra.run.smithi158.stderr:gzip -5 --verbose -- /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-osd.10.log 2024-02-18T11:56:48.842 INFO:teuthology.orchestra.run.smithi158.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph.audit.log 2024-02-18T11:56:48.842 INFO:teuthology.orchestra.run.smithi158.stderr:/var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-mds.b.log 2024-02-18T11:56:48.842 INFO:teuthology.orchestra.run.smithi158.stderr:/var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-osd.4.log 2024-02-18T11:56:48.845 INFO:teuthology.orchestra.run.smithi158.stderr:/var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-osd.1.log 2024-02-18T11:56:48.845 INFO:teuthology.orchestra.run.smithi158.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-18T11:56:48.846 INFO:teuthology.orchestra.run.smithi158.stderr:/var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-mds.k.log 2024-02-18T11:56:48.846 INFO:teuthology.orchestra.run.smithi158.stderr:/var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-mds.h.log 2024-02-18T11:56:48.846 INFO:teuthology.orchestra.run.smithi158.stderr:/var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-osd.7.log 2024-02-18T11:56:48.846 INFO:teuthology.orchestra.run.smithi158.stderr:/var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-mds.e.log 2024-02-18T11:56:48.846 INFO:teuthology.orchestra.run.smithi158.stderr:/var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-mgr.y.log 2024-02-18T11:56:48.846 INFO:teuthology.orchestra.run.smithi158.stderr:/var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph.cephadm.log 2024-02-18T11:56:48.846 INFO:teuthology.orchestra.run.smithi158.stderr:/var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph.log 2024-02-18T11:56:48.846 INFO:teuthology.orchestra.run.smithi158.stderr:/var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-volume.log 2024-02-18T11:56:48.846 INFO:teuthology.orchestra.run.smithi158.stderr:/var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-mon.b.log 2024-02-18T11:56:48.846 INFO:teuthology.orchestra.run.smithi158.stderr:/var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-volume.log: /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-mon.b.log: 94.4% -- replaced with /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-mds.k.log.gz 2024-02-18T11:56:48.846 INFO:teuthology.orchestra.run.smithi158.stderr: 92.9% -- replaced with /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-mgr.y.log.gz 2024-02-18T11:56:48.847 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-volume.log: 90.9% -- replaced with /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-volume.log.gz 2024-02-18T11:56:48.848 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-mds.a.log: 94.4% -- replaced with /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-mds.a.log.gz 2024-02-18T11:56:48.848 INFO:teuthology.orchestra.run.smithi012.stderr: 90.9% -- replaced with /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-volume.log.gz 2024-02-18T11:56:48.891 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-mon.c.log: 93.7% -- replaced with /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-mds.c.log.gz 2024-02-18T11:56:48.893 INFO:teuthology.orchestra.run.smithi188.stderr: 93.7% -- replaced with /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-mds.i.log.gz 2024-02-18T11:56:48.897 INFO:teuthology.orchestra.run.smithi188.stderr: 94.6% -- replaced with /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-mds.f.log.gz 2024-02-18T11:56:48.943 INFO:teuthology.orchestra.run.smithi158.stderr: 82.7% -- replaced with /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph.cephadm.log.gz 2024-02-18T11:56:49.055 INFO:teuthology.orchestra.run.smithi158.stderr: 90.3% -- replaced with /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph.audit.log.gz 2024-02-18T11:56:49.055 INFO:teuthology.orchestra.run.smithi158.stderr: 91.0% -- replaced with /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-volume.log.gz 2024-02-18T11:56:49.055 INFO:teuthology.orchestra.run.smithi158.stderr: 85.9% -- replaced with /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph.log.gz 2024-02-18T11:56:49.110 INFO:teuthology.orchestra.run.smithi012.stderr: 90.0% -- replaced with /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-mgr.x.log.gz 2024-02-18T11:56:49.157 INFO:teuthology.orchestra.run.smithi158.stderr: 94.5% -- replaced with /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-mds.e.log.gz 2024-02-18T11:56:49.189 INFO:teuthology.orchestra.run.smithi188.stderr: 92.0% -- replaced with /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-mon.c.log.gz 2024-02-18T11:56:49.761 INFO:teuthology.orchestra.run.smithi012.stderr: 90.4% -- replaced with /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-mon.a.log.gz 2024-02-18T11:56:50.320 INFO:teuthology.orchestra.run.smithi158.stderr: 92.3% -- replaced with /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-mon.b.log.gz 2024-02-18T11:56:54.500 INFO:teuthology.orchestra.run.smithi012.stderr: 93.2% -- replaced with /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-osd.9.log.gz 2024-02-18T11:56:55.371 INFO:teuthology.orchestra.run.smithi188.stderr: 93.1% -- replaced with /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-osd.2.log.gz 2024-02-18T11:56:56.193 INFO:teuthology.orchestra.run.smithi188.stderr: 93.0% -- replaced with /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-osd.8.log.gz 2024-02-18T11:56:56.695 INFO:teuthology.orchestra.run.smithi012.stderr: 93.3% -- replaced with /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-osd.6.log.gz 2024-02-18T11:56:57.322 INFO:teuthology.orchestra.run.smithi188.stderr: 93.0% -- replaced with /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-osd.5.log.gz 2024-02-18T11:56:57.610 INFO:teuthology.orchestra.run.smithi012.stderr: 93.2% -- replaced with /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-osd.3.log.gz 2024-02-18T11:56:57.956 INFO:teuthology.orchestra.run.smithi012.stderr: 93.4% -- replaced with /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-osd.0.log.gz 2024-02-18T11:56:57.958 INFO:teuthology.orchestra.run.smithi012.stderr: 2024-02-18T11:56:57.958 INFO:teuthology.orchestra.run.smithi012.stderr:real 0m9.152s 2024-02-18T11:56:57.958 INFO:teuthology.orchestra.run.smithi012.stderr:user 0m31.553s 2024-02-18T11:56:57.959 INFO:teuthology.orchestra.run.smithi012.stderr:sys 0m1.154s 2024-02-18T11:56:57.983 INFO:teuthology.orchestra.run.smithi188.stderr: 93.1% -- replaced with /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-osd.11.log.gz 2024-02-18T11:56:57.985 INFO:teuthology.orchestra.run.smithi188.stderr: 2024-02-18T11:56:57.985 INFO:teuthology.orchestra.run.smithi188.stderr:real 0m9.174s 2024-02-18T11:56:57.985 INFO:teuthology.orchestra.run.smithi188.stderr:user 0m30.931s 2024-02-18T11:56:57.985 INFO:teuthology.orchestra.run.smithi188.stderr:sys 0m1.175s 2024-02-18T11:57:12.244 INFO:teuthology.orchestra.run.smithi158.stderr: 93.3% -- replaced with /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-osd.4.log.gz 2024-02-18T11:57:16.717 INFO:teuthology.orchestra.run.smithi158.stderr: 93.1% -- replaced with /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-osd.10.log.gz 2024-02-18T11:57:23.081 INFO:teuthology.orchestra.run.smithi158.stderr: 93.0% -- replaced with /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-osd.1.log.gz 2024-02-18T11:57:30.041 INFO:teuthology.orchestra.run.smithi158.stderr: 93.1% -- replaced with /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-osd.7.log.gz 2024-02-18T11:58:08.670 INFO:teuthology.orchestra.run.smithi158.stderr: 97.4% -- replaced with /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-mds.h.log.gz 2024-02-18T12:01:23.383 INFO:teuthology.orchestra.run.smithi158.stderr: 94.4% -- replaced with /var/log/ceph/0c8bb0de-ce51-11ee-95ba-87774f69a715/ceph-mds.b.log.gz 2024-02-18T12:01:23.433 INFO:teuthology.orchestra.run.smithi158.stderr: 2024-02-18T12:01:23.433 INFO:teuthology.orchestra.run.smithi158.stderr:real 4m34.619s 2024-02-18T12:01:23.433 INFO:teuthology.orchestra.run.smithi158.stderr:user 4m9.368s 2024-02-18T12:01:23.433 INFO:teuthology.orchestra.run.smithi158.stderr:sys 0m10.433s 2024-02-18T12:01:23.434 INFO:tasks.cephadm:Archiving logs... 2024-02-18T12:01:23.434 DEBUG:teuthology.misc:Transferring archived files from smithi012:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-02-17_05:57:56-fs-wip-vshankar-testing-20240217.015652-testing-default-smithi/7564362/remote/smithi012/log 2024-02-18T12:01:23.435 DEBUG:teuthology.orchestra.run.smithi012:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-18T12:01:28.176 DEBUG:teuthology.misc:Transferring archived files from smithi158:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-02-17_05:57:56-fs-wip-vshankar-testing-20240217.015652-testing-default-smithi/7564362/remote/smithi158/log 2024-02-18T12:01:28.177 DEBUG:teuthology.orchestra.run.smithi158:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-18T12:02:05.780 DEBUG:teuthology.misc:Transferring archived files from smithi188:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-02-17_05:57:56-fs-wip-vshankar-testing-20240217.015652-testing-default-smithi/7564362/remote/smithi188/log 2024-02-18T12:02:05.781 DEBUG:teuthology.orchestra.run.smithi188:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-18T12:02:10.737 INFO:tasks.cephadm:Removing cluster... 2024-02-18T12:02:10.737 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 --force 2024-02-18T12:02:10.935 INFO:teuthology.orchestra.run.smithi012.stdout:Deleting cluster with fsid: 0c8bb0de-ce51-11ee-95ba-87774f69a715 2024-02-18T12:02:11.226 DEBUG:teuthology.orchestra.run.smithi158:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 --force 2024-02-18T12:02:11.466 INFO:teuthology.orchestra.run.smithi158.stdout:Deleting cluster with fsid: 0c8bb0de-ce51-11ee-95ba-87774f69a715 2024-02-18T12:02:12.007 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0c8bb0de-ce51-11ee-95ba-87774f69a715 --force 2024-02-18T12:02:12.198 INFO:teuthology.orchestra.run.smithi188.stdout:Deleting cluster with fsid: 0c8bb0de-ce51-11ee-95ba-87774f69a715 2024-02-18T12:02:12.496 INFO:tasks.cephadm:Removing cephadm ... 2024-02-18T12:02:12.496 DEBUG:teuthology.orchestra.run.smithi012:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-18T12:02:12.514 DEBUG:teuthology.orchestra.run.smithi158:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-18T12:02:12.530 DEBUG:teuthology.orchestra.run.smithi188:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-18T12:02:12.546 INFO:tasks.cephadm:Teardown complete 2024-02-18T12:02:12.546 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-02-18T12:02:12.558 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-02-18T12:02:12.558 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-18T12:02:12.561 DEBUG:teuthology.orchestra.run.smithi158:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-18T12:02:12.573 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-18T12:02:12.703 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-02-18T12:02:12.712 INFO:teuthology.task.clock:Checking final clock skew... 2024-02-18T12:02:12.713 DEBUG:teuthology.orchestra.run.smithi012:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-18T12:02:12.715 DEBUG:teuthology.orchestra.run.smithi158:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-18T12:02:12.718 DEBUG:teuthology.orchestra.run.smithi188:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-18T12:02:12.729 INFO:teuthology.orchestra.run.smithi012.stderr:bash: line 1: ntpq: command not found 2024-02-18T12:02:12.730 INFO:teuthology.orchestra.run.smithi158.stderr:bash: line 1: ntpq: command not found 2024-02-18T12:02:12.731 INFO:teuthology.orchestra.run.smithi188.stderr:bash: line 1: ntpq: command not found 2024-02-18T12:02:12.735 INFO:teuthology.orchestra.run.smithi012.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-18T12:02:12.735 INFO:teuthology.orchestra.run.smithi012.stdout:=============================================================================== 2024-02-18T12:02:12.735 INFO:teuthology.orchestra.run.smithi012.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 27 +3028us[+3028us] +/- 85ms 2024-02-18T12:02:12.735 INFO:teuthology.orchestra.run.smithi012.stdout:^* hv02.front.sepia.ceph.com 3 6 377 30 -1408us[-1344us] +/- 50ms 2024-02-18T12:02:12.735 INFO:teuthology.orchestra.run.smithi012.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 29 -629us[ -629us] +/- 85ms 2024-02-18T12:02:12.735 INFO:teuthology.orchestra.run.smithi012.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-02-18T12:02:12.736 INFO:teuthology.orchestra.run.smithi188.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-18T12:02:12.736 INFO:teuthology.orchestra.run.smithi188.stdout:=============================================================================== 2024-02-18T12:02:12.736 INFO:teuthology.orchestra.run.smithi188.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 28 +3001us[+3060us] +/- 85ms 2024-02-18T12:02:12.736 INFO:teuthology.orchestra.run.smithi188.stdout:^* hv02.front.sepia.ceph.com 3 6 377 27 -1431us[-1372us] +/- 50ms 2024-02-18T12:02:12.736 INFO:teuthology.orchestra.run.smithi188.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 25 -647us[ -647us] +/- 85ms 2024-02-18T12:02:12.736 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-02-18T12:02:12.792 INFO:teuthology.orchestra.run.smithi158.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-18T12:02:12.792 INFO:teuthology.orchestra.run.smithi158.stdout:=============================================================================== 2024-02-18T12:02:12.793 INFO:teuthology.orchestra.run.smithi158.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 30 +2953us[+2953us] +/- 85ms 2024-02-18T12:02:12.793 INFO:teuthology.orchestra.run.smithi158.stdout:^* hv02.front.sepia.ceph.com 3 6 377 32 -1468us[-1452us] +/- 50ms 2024-02-18T12:02:12.793 INFO:teuthology.orchestra.run.smithi158.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 28 -696us[ -696us] +/- 85ms 2024-02-18T12:02:12.793 INFO:teuthology.orchestra.run.smithi158.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-02-18T12:02:12.813 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-02-18T12:02:12.823 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-02-18T12:02:12.824 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-02-18T12:02:13.025 DEBUG:teuthology.orchestra.run.smithi012:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-18T12:02:13.111 DEBUG:teuthology.orchestra.run.smithi158:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-18T12:02:13.213 DEBUG:teuthology.orchestra.run.smithi188:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-18T12:02:13.287 DEBUG:teuthology.orchestra.run.smithi012:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-18T12:02:13.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-18T12:02:13.319 DEBUG:teuthology.orchestra.run.smithi158:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-18T12:02:13.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-18T12:02:13.350 DEBUG:teuthology.orchestra.run.smithi188:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-18T12:02:13.381 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-18T12:02:13.381 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-02-18T12:02:13.391 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-02-18T12:02:13.450 INFO:teuthology.task.internal:Duration was 2311.893479 seconds 2024-02-18T12:02:13.451 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-02-18T12:02:13.498 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-02-18T12:02:13.498 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-18T12:02:13.501 DEBUG:teuthology.orchestra.run.smithi158:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-18T12:02:13.503 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-18T12:02:13.545 INFO:teuthology.orchestra.run.smithi012.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-18T12:02:13.547 INFO:teuthology.orchestra.run.smithi188.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-18T12:02:13.557 INFO:teuthology.orchestra.run.smithi158.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-18T12:02:14.107 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-02-18T12:02:14.107 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi012.front.sepia.ceph.com 2024-02-18T12:02:14.107 DEBUG:teuthology.orchestra.run.smithi012:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-18T12:02:14.131 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi158.front.sepia.ceph.com 2024-02-18T12:02:14.131 DEBUG:teuthology.orchestra.run.smithi158:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-18T12:02:14.159 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi188.front.sepia.ceph.com 2024-02-18T12:02:14.160 DEBUG:teuthology.orchestra.run.smithi188:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-18T12:02:14.191 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-02-18T12:02:14.191 DEBUG:teuthology.orchestra.run.smithi012:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-18T12:02:14.193 DEBUG:teuthology.orchestra.run.smithi158:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-18T12:02:14.202 DEBUG:teuthology.orchestra.run.smithi188:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-18T12:02:14.261 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-02-18T12:02:14.261 DEBUG:teuthology.orchestra.run.smithi012:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-18T12:02:14.264 DEBUG:teuthology.orchestra.run.smithi158:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-18T12:02:14.271 DEBUG:teuthology.orchestra.run.smithi188:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-18T12:02:14.640 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-02-18T12:02:14.652 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-02-18T12:02:14.652 DEBUG:teuthology.orchestra.run.smithi012:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-18T12:02:14.678 DEBUG:teuthology.orchestra.run.smithi158:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-18T12:02:14.706 DEBUG:teuthology.orchestra.run.smithi188:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-18T12:02:14.734 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-02-18T12:02:14.746 DEBUG:teuthology.orchestra.run.smithi012:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-18T12:02:14.748 DEBUG:teuthology.orchestra.run.smithi158:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-18T12:02:14.750 DEBUG:teuthology.orchestra.run.smithi188:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-18T12:02:14.771 INFO:teuthology.orchestra.run.smithi012.stdout:kernel.core_pattern = core 2024-02-18T12:02:14.799 INFO:teuthology.orchestra.run.smithi188.stdout:kernel.core_pattern = core 2024-02-18T12:02:14.807 INFO:teuthology.orchestra.run.smithi158.stdout:kernel.core_pattern = core 2024-02-18T12:02:14.853 DEBUG:teuthology.orchestra.run.smithi012:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-18T12:02:14.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-18T12:02:14.867 DEBUG:teuthology.orchestra.run.smithi158:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-18T12:02:14.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-18T12:02:14.881 DEBUG:teuthology.orchestra.run.smithi188:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-18T12:02:14.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-18T12:02:14.894 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-02-18T12:02:14.905 INFO:teuthology.task.internal:Transferring archived files... 2024-02-18T12:02:14.906 DEBUG:teuthology.misc:Transferring archived files from smithi012:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-02-17_05:57:56-fs-wip-vshankar-testing-20240217.015652-testing-default-smithi/7564362/remote/smithi012 2024-02-18T12:02:14.907 DEBUG:teuthology.orchestra.run.smithi012:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-18T12:02:14.954 DEBUG:teuthology.misc:Transferring archived files from smithi158:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-02-17_05:57:56-fs-wip-vshankar-testing-20240217.015652-testing-default-smithi/7564362/remote/smithi158 2024-02-18T12:02:14.954 DEBUG:teuthology.orchestra.run.smithi158:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-18T12:02:14.993 DEBUG:teuthology.misc:Transferring archived files from smithi188:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-02-17_05:57:56-fs-wip-vshankar-testing-20240217.015652-testing-default-smithi/7564362/remote/smithi188 2024-02-18T12:02:14.993 DEBUG:teuthology.orchestra.run.smithi188:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-18T12:02:15.032 INFO:teuthology.task.internal:Removing archive directory... 2024-02-18T12:02:15.032 DEBUG:teuthology.orchestra.run.smithi012:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-18T12:02:15.034 DEBUG:teuthology.orchestra.run.smithi158:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-18T12:02:15.037 DEBUG:teuthology.orchestra.run.smithi188:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-18T12:02:15.080 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-02-18T12:02:15.093 INFO:teuthology.task.internal:Not uploading archives. 2024-02-18T12:02:15.093 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-02-18T12:02:15.122 INFO:teuthology.task.internal:Tidying up after the test... 2024-02-18T12:02:15.123 DEBUG:teuthology.orchestra.run.smithi012:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-18T12:02:15.125 DEBUG:teuthology.orchestra.run.smithi158:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-18T12:02:15.128 DEBUG:teuthology.orchestra.run.smithi188:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-18T12:02:15.141 INFO:teuthology.orchestra.run.smithi158.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 18 12:02 /home/ubuntu/cephtest 2024-02-18T12:02:15.141 INFO:teuthology.orchestra.run.smithi012.stdout: 265295 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 18 12:02 /home/ubuntu/cephtest 2024-02-18T12:02:15.141 INFO:teuthology.orchestra.run.smithi188.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 18 12:02 /home/ubuntu/cephtest 2024-02-18T12:02:15.143 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-02-18T12:02:15.174 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-02-18T12:02:15.314 INFO:teuthology.nuke:Checking targets against current locks 2024-02-18T12:02:15.341 DEBUG:teuthology.nuke:shortname: smithi012 2024-02-18T12:02:15.342 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-18T12:02:15.368 DEBUG:teuthology.nuke:shortname: smithi158 2024-02-18T12:02:15.368 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-18T12:02:15.373 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi012.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-02-17_05:57:56-fs-wip-vshankar-testing-20240217.015652-testing-default-smithi/7564362', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-18 11:14:53.493738', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPrh55wigUo1PV8Sb+YAd/BcDe3NzW1EBCcCr++pGUugPTHIrEASfgvqt8wLiFvHCFlwb40zj+IIA7WFOtaQ3y4='} 2024-02-18T12:02:15.392 DEBUG:teuthology.nuke:shortname: smithi188 2024-02-18T12:02:15.392 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-18T12:02:15.396 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi158.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-02-17_05:57:56-fs-wip-vshankar-testing-20240217.015652-testing-default-smithi/7564362', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-18 11:14:53.492560', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDJBuLtLzONUSej4uAtVBleRAiTjKIFNpfuX5dLDM0m0j1l2nOCx+EdZqn08ocQh6JQufNhmMjal/fpjbV3J+Fk='} 2024-02-18T12:02:15.417 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi188.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-02-17_05:57:56-fs-wip-vshankar-testing-20240217.015652-testing-default-smithi/7564362', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-18 11:14:53.490714', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBamJ0stpfhNmYvSj0ZmQp6uM1kWXi3t6wSf7VETIvnMOdClFcvgahcP/hcCcctGcvr6rYRthK9ke8H0znZLiWE='} 2024-02-18T12:02:15.430 INFO:teuthology.orchestra.console.smithi012:Power off 2024-02-18T12:02:15.430 DEBUG:teuthology.orchestra.console.smithi012:pexpect command: ipmitool -H smithi012.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-18T12:02:15.453 DEBUG:teuthology.orchestra.console.smithi012:power off output: Chassis Power Control: Down/Off 2024-02-18T12:02:15.453 DEBUG:teuthology.orchestra.console.smithi012:pexpect command: ipmitool -H smithi012.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-18T12:02:15.471 INFO:teuthology.orchestra.console.smithi188:Power off 2024-02-18T12:02:15.472 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-18T12:02:15.485 INFO:teuthology.orchestra.console.smithi158:Power off 2024-02-18T12:02:15.486 DEBUG:teuthology.orchestra.console.smithi158:pexpect command: ipmitool -H smithi158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-18T12:02:15.494 DEBUG:teuthology.orchestra.console.smithi012:check power output: Chassis Power is on 2024-02-18T12:02:15.494 DEBUG:teuthology.orchestra.console.smithi188:power off output: Chassis Power Control: Down/Off 2024-02-18T12:02:15.494 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-18T12:02:15.508 DEBUG:teuthology.orchestra.console.smithi158:power off output: Chassis Power Control: Down/Off 2024-02-18T12:02:15.508 DEBUG:teuthology.orchestra.console.smithi158:pexpect command: ipmitool -H smithi158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-18T12:02:15.518 DEBUG:teuthology.orchestra.console.smithi188:check power output: Chassis Power is on 2024-02-18T12:02:15.532 DEBUG:teuthology.orchestra.console.smithi158:check power output: Chassis Power is on 2024-02-18T12:02:19.495 DEBUG:teuthology.orchestra.console.smithi012:pexpect command: ipmitool -H smithi012.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-18T12:02:19.519 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-18T12:02:19.533 DEBUG:teuthology.orchestra.console.smithi158:pexpect command: ipmitool -H smithi158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-18T12:02:19.606 DEBUG:teuthology.orchestra.console.smithi012:check power output: Chassis Power is on 2024-02-18T12:02:19.631 DEBUG:teuthology.orchestra.console.smithi188:check power output: Chassis Power is on 2024-02-18T12:02:19.647 DEBUG:teuthology.orchestra.console.smithi158:check power output: Chassis Power is on 2024-02-18T12:02:23.611 DEBUG:teuthology.orchestra.console.smithi012:pexpect command: ipmitool -H smithi012.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-18T12:02:23.632 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-18T12:02:23.648 DEBUG:teuthology.orchestra.console.smithi158:pexpect command: ipmitool -H smithi158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-18T12:02:23.722 DEBUG:teuthology.orchestra.console.smithi012:check power output: Chassis Power is on 2024-02-18T12:02:23.741 DEBUG:teuthology.orchestra.console.smithi188:check power output: Chassis Power is off 2024-02-18T12:02:23.757 DEBUG:teuthology.orchestra.console.smithi158:check power output: Chassis Power is on 2024-02-18T12:02:23.842 INFO:teuthology.orchestra.console.smithi188:Power off completed 2024-02-18T12:02:27.724 DEBUG:teuthology.orchestra.console.smithi012:pexpect command: ipmitool -H smithi012.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-18T12:02:27.758 DEBUG:teuthology.orchestra.console.smithi158:pexpect command: ipmitool -H smithi158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-18T12:02:27.837 DEBUG:teuthology.orchestra.console.smithi012:check power output: Chassis Power is off 2024-02-18T12:02:27.867 DEBUG:teuthology.orchestra.console.smithi158:check power output: Chassis Power is off 2024-02-18T12:02:27.939 INFO:teuthology.orchestra.console.smithi012:Power off completed 2024-02-18T12:02:27.968 INFO:teuthology.orchestra.console.smithi158:Power off completed 2024-02-18T12:02:28.074 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/no 5-workunit/suites/iogen}} duration: 2311.8934786319733 failure_reason: '"2024-02-18T11:32:48.888403+0000 mon.a (mon.0) 190 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c" in cluster log' flavor: default owner: scheduled_vshankar@teuthology success: false 2024-02-18T12:02:28.074 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-18T12:02:28.165 INFO:teuthology.run:FAIL